2020-11-16T21:55:07.121 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-16T21:55:07.123 DEBUG:teuthology.run:Teuthology command: teuthology --lock --archive /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936 --description multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc} --verbose --block --name pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi --owner scheduled_pdonnell@teuthology -- /tmp/teuthology-worker._qr_iub1.tmp 2020-11-16T21:55:07.164 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T21:55:07.225 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936 branch: wip-pdonnell-testing-20201116.174825 description: multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc} email: pdonnell@redhat.com first_in_suite: false job_id: '5631936' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-pdonnell-testing-20201116.174825 ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 fuse default permissions: false rados mon op timeout: 15m rados osd op timeout: 15m global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Scrub error on inode log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6959059e6aae2fb5241382a4eeeca62785867604 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 6959059e6aae2fb5241382a4eeeca62785867604 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20201116.174825 sha1: 6959059e6aae2fb5241382a4eeeca62785867604 owner: scheduled_pdonnell@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mgr.x - mds.b - mds.c - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - client.1 sha1: 6959059e6aae2fb5241382a4eeeca62785867604 sleep_before_teardown: 0 suite: multimds suite_branch: wip-pdonnell-testing-20201116.174825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 6959059e6aae2fb5241382a4eeeca62785867604 tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - check-counter: counters: mds: - mds.exported - mds.imported - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology_branch: master timestamp: 2020-11-16_20:59:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.24835 2020-11-16T21:55:07.226 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa; will attempt to use it 2020-11-16T21:55:07.226 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa/tasks 2020-11-16T21:55:07.227 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-16T21:55:07.228 INFO:teuthology.task.internal:Checking packages... 2020-11-16T21:55:07.249 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '6959059e6aae2fb5241382a4eeeca62785867604' 2020-11-16T21:55:07.249 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-16T21:55:07.249 INFO:teuthology.packaging:ref: None 2020-11-16T21:55:07.250 INFO:teuthology.packaging:tag: None 2020-11-16T21:55:07.250 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20201116.174825 2020-11-16T21:55:07.250 INFO:teuthology.packaging:sha1: 6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T21:55:07.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-pdonnell-testing-20201116.174825 2020-11-16T21:55:07.416 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7226-g6959059e-1bionic 2020-11-16T21:55:07.417 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-16T21:55:07.427 INFO:teuthology.task.internal:no buildpackages task found 2020-11-16T21:55:07.427 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-16T21:55:07.453 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-16T21:55:07.453 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T21:55:07.513 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:17.531 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:27.549 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:37.567 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:47.582 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:55:57.595 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:07.609 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:17.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:27.647 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:37.666 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:47.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-11-16T21:56:57.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-11-16T21:57:07.718 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:17.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:27.747 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:37.767 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:47.778 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:57:57.795 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:07.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:17.831 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:27.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:37.866 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:47.878 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-11-16T21:58:57.891 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 3, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936'} 2020-11-16T21:58:57.913 DEBUG:teuthology.lock.ops:locked ubuntu@smithi185.front.sepia.ceph.com, ubuntu@smithi096.front.sepia.ceph.com, ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T21:58:57.988 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4970868d90>, signals=[15]) 2020-11-16T21:58:58.205 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of ubuntu 18.04 2020-11-16T21:58:58.208 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of ubuntu 18.04 2020-11-16T21:58:58.215 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of ubuntu 18.04 2020-11-16T21:58:58.516 INFO:teuthology.orchestra.console:Power off smithi096 2020-11-16T21:58:58.517 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T21:58:58.534 INFO:teuthology.orchestra.console:Power off smithi185 2020-11-16T21:58:58.535 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi185.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T21:58:58.549 INFO:teuthology.orchestra.console:Power off smithi062 2020-11-16T21:58:58.550 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T21:58:58.559 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi185.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:58:58.567 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:58:58.577 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:58:58.686 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2020-11-16T21:58:58.688 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2020-11-16T21:58:58.788 INFO:teuthology.orchestra.console:Power on smithi185 2020-11-16T21:58:58.788 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi185.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T21:58:58.798 INFO:teuthology.orchestra.console:Power on smithi096 2020-11-16T21:58:58.799 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T21:58:58.810 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi185.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:58:58.819 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:02.600 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:02.830 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi185.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:02.840 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:03.043 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2020-11-16T21:59:03.050 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2020-11-16T21:59:03.144 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2020-11-16T21:59:03.150 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2020-11-16T21:59:06.713 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:19.300 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2020-11-16T21:59:19.401 INFO:teuthology.orchestra.console:Power on smithi062 2020-11-16T21:59:19.401 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T21:59:19.422 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:23.447 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:27.559 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T21:59:27.770 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2020-11-16T21:59:27.871 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2020-11-16T22:02:04.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:05.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:13.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:13.842 ERROR:teuthology.orchestra.connection:Error connecting to smithi062.front.sepia.ceph.com Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/connection.py", line 108, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect passphrase, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 764, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 740, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1580, in auth_publickey return self.auth_handler.wait_for_response(my_event) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/auth_handler.py", line 250, in wait_for_response raise e paramiko.ssh_exception.AuthenticationException: Authentication failed. 2020-11-16T22:02:29.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:29.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:38.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:38.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:47.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:47.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:56.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:02:56.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:03:05.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:03:14.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:03:20.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:03:45.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:03:54.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:02.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:02.681 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:04:02.683 INFO:teuthology.orchestra.run.smithi096:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:04:03.043 INFO:teuthology.provision.fog.smithi096:Node is ready 2020-11-16T22:04:03.043 INFO:teuthology.orchestra.run.smithi096:> hostname 2020-11-16T22:04:03.092 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2020-11-16T22:04:03.093 INFO:teuthology.orchestra.run.smithi096:> grep smithi096.front.sepia.ceph.com /etc/hosts 2020-11-16T22:04:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2020-11-16T22:04:03.140 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-11-16T22:04:03.252 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hosts 2020-11-16T22:04:03.266 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/172.21.15.96/172.21.15.96/g' /etc/hosts 2020-11-16T22:04:03.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:03.325 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hostname 2020-11-16T22:04:03.381 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-11-16T22:04:03.435 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2020-11-16T22:04:03.436 DEBUG:teuthology.parallel:result is None 2020-11-16T22:04:12.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:12.441 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:04:12.441 INFO:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:04:12.814 INFO:teuthology.provision.fog.smithi062:Node is ready 2020-11-16T22:04:12.815 INFO:teuthology.orchestra.run.smithi062:> hostname 2020-11-16T22:04:12.860 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2020-11-16T22:04:12.861 INFO:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts 2020-11-16T22:04:12.910 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2020-11-16T22:04:12.910 INFO:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2020-11-16T22:04:13.023 INFO:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts 2020-11-16T22:04:13.034 INFO:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts 2020-11-16T22:04:13.109 INFO:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname 2020-11-16T22:04:13.164 INFO:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2020-11-16T22:04:13.221 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2020-11-16T22:04:13.222 DEBUG:teuthology.parallel:result is None 2020-11-16T22:04:20.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:20.796 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T22:04:20.797 INFO:teuthology.orchestra.run.smithi185:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T22:04:21.119 INFO:teuthology.provision.fog.smithi185:Node is ready 2020-11-16T22:04:21.119 INFO:teuthology.orchestra.run.smithi185:> hostname 2020-11-16T22:04:21.165 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2020-11-16T22:04:21.166 INFO:teuthology.orchestra.run.smithi185:> grep smithi185.front.sepia.ceph.com /etc/hosts 2020-11-16T22:04:21.214 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2020-11-16T22:04:21.214 INFO:teuthology.orchestra.run.smithi185:> sudo hostname smithi185 2020-11-16T22:04:21.367 INFO:teuthology.orchestra.run.smithi185:> sudo sed -i -e 's/smithi185.front.sepia.ceph.com/smithi185/g' /etc/hosts 2020-11-16T22:04:21.383 INFO:teuthology.orchestra.run.smithi185:> sudo sed -i -e 's/172.21.15.185/172.21.15.185/g' /etc/hosts 2020-11-16T22:04:21.438 INFO:teuthology.orchestra.run.smithi185:> sudo sed -i -e 's/smithi185.front.sepia.ceph.com/smithi185/g' /etc/hostname 2020-11-16T22:04:21.493 INFO:teuthology.orchestra.run.smithi185:> sudo hostname smithi185 2020-11-16T22:04:21.550 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2020-11-16T22:04:21.550 DEBUG:teuthology.parallel:result is None 2020-11-16T22:04:21.718 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2020-11-16T22:04:21.922 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:04:21.938 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2020-11-16T22:04:21.939 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:04:21.951 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2020-11-16T22:04:21.951 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T22:04:21.965 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:04:21.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:04:21.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:22.041 INFO:teuthology.orchestra.run.smithi185:> true 2020-11-16T22:04:22.231 INFO:teuthology.orchestra.run.smithi185:> uname -m 2020-11-16T22:04:22.277 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2020-11-16T22:04:22.278 INFO:teuthology.orchestra.run.smithi185:> cat /etc/os-release 2020-11-16T22:04:22.325 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Ubuntu" 2020-11-16T22:04:22.326 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:22.326 INFO:teuthology.orchestra.run.smithi185.stdout:ID=ubuntu 2020-11-16T22:04:22.326 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE=debian 2020-11-16T22:04:22.326 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:22.326 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="18.04" 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:22.327 INFO:teuthology.orchestra.run.smithi185.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:22.328 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2020-11-16T22:04:22.349 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:04:22.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:04:22.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:22.425 INFO:teuthology.orchestra.run.smithi096:> true 2020-11-16T22:04:22.659 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-11-16T22:04:22.706 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-11-16T22:04:22.707 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-11-16T22:04:22.754 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2020-11-16T22:04:22.754 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="18.04" 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:22.755 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:22.756 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:22.756 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-11-16T22:04:22.771 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2020-11-16T22:04:22.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:04:22.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:22.850 INFO:teuthology.orchestra.run.smithi062:> true 2020-11-16T22:04:23.086 INFO:teuthology.orchestra.run.smithi062:> uname -m 2020-11-16T22:04:23.133 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2020-11-16T22:04:23.133 INFO:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2020-11-16T22:04:23.181 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2020-11-16T22:04:23.181 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="18.04" 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:23.182 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:23.183 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:23.183 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:23.183 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:23.183 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2020-11-16T22:04:23.198 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-11-16T22:04:23.245 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuzY3IcuYwznKE0zVDwYqPet1njx9kdFtD64h4NQk/zw9+xuDbhlL2OkwyhWwM/sccB0Pd2eAA3T9DJ/6a/aa+P9/Th1NpgiCMmObrXOV2bpjJNeMP32YQxpF6VznvHurtM0KD/4143QuOPmKEvCyQ9y25LlksWu896WmUtcD/o71i+2/qk3g0dQu1SCWXAAqu8CynJbSUrWvK2o5L/z+M7qZER9r3TUXwtvu/FnKlgm6iJCUSxFuFRfvLQT2mdV5vCL+iwaFRC/OFCwPzXEYpYmg14ePfno2udmsWIg30FzW+bH0YTqFuTPLbXi7Y6PljKcdNHWhzmUYU0vFwz3mb smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzjvIZwUJ9GPhyZcPgODoG47eKuYPW5cY/P45M3uons0W9I8NkMymG1KWSSFO26xCnAT1VU7mXSS7BRrFKBno+3dzDJnc/ReYU45HEgrA/08JenVvvUvngKYkzIzYtvIdYIZc38kNaI6WBMp8QehgyfVLnqMZOvsqss4mykZU7N+FoyAjBSH9we51opWLo1chcffaMNS43Iaz3hChrh6F01uUg5JYy+EJu6DvDIjtdH5R3UpfDPXxmls1XY0yvd4lWnZKTvxoZR8IUKIxWgk2wVobiDTsvllHMQXnpt0WoLcbvOYxcPkWDx0+VG4k8+V0cANE53w56La877vmgYcsN smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCf2u2VpfI2OiGTlXbLsu5ah2Tn+IfD7QHUuqb8apbpZ/qrIxfqGvWWkCbsXA5sp2l0Nd120DAz5CRiY78zSD9zO8nUnYQYaTE/pYHsnpL7FAZ0dpF6r3Wx9kFYj5HwwUubBjZYlaaYHoX8oj8DQ2I8WpC1nzseAeeO0ENup3WCrrlbdWvjSQDBCSsL1um+2uQDHOFFNs9c7Dq+ol0Y6BxvOhJVJOlNuQQc3Hjw9fq0PrzLyCmTz+27MxtQtm8v1Z+HECsaLi3MmRAXaNPTR83FDiH5Fn97OR1KrJqg9AE9WtIZ7OKkRYo7L1s+jeMH9N8ISTFerXVljuzipatSbyih 2020-11-16T22:04:23.245 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T22:04:23.300 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-16T22:04:23.306 INFO:teuthology.task.internal:Saving configuration 2020-11-16T22:04:23.323 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-16T22:04:23.330 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-16T22:04:23.347 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 21:58:57.910826', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEKBnJeGKAoH54RV+yqiGqrFaVYPijnTH2D92eO/hO7f', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi185.front.sepia.ceph.com'} 2020-11-16T22:04:23.363 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 21:58:57.910857', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJu5LdwpCUpFOXhX8l5hJAy81JMBVm3s4/IwYoC2SZCq', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi096.front.sepia.ceph.com'} 2020-11-16T22:04:23.378 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 21:58:57.910873', 'description': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILVi1TtkeR9Tds6Hi9bErHAoYSD/wMT/Py0fTEXdVZcQ', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi062.front.sepia.ceph.com'} 2020-11-16T22:04:23.378 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-16T22:04:23.389 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2020-11-16T22:04:23.390 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'mds.b', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-11-16T22:04:23.594 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['client.0', 'client.1'] 2020-11-16T22:04:23.594 INFO:teuthology.run_tasks:Running task console_log... 2020-11-16T22:04:23.887 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f497442f048>, signals=[15]) 2020-11-16T22:04:23.888 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-16T22:04:24.098 INFO:teuthology.task.internal:Opening connections... 2020-11-16T22:04:24.099 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:04:24.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:24.178 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:04:24.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:24.252 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:04:24.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:04:24.327 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-16T22:04:24.335 INFO:teuthology.orchestra.run.smithi185:> uname -m 2020-11-16T22:04:24.353 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2020-11-16T22:04:24.354 INFO:teuthology.orchestra.run.smithi185:> cat /etc/os-release 2020-11-16T22:04:24.401 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Ubuntu" 2020-11-16T22:04:24.401 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:ID=ubuntu 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE=debian 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="18.04" 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:24.402 INFO:teuthology.orchestra.run.smithi185.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:24.403 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:24.403 INFO:teuthology.orchestra.run.smithi185.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:24.403 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:24.403 INFO:teuthology.orchestra.run.smithi185.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:24.403 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2020-11-16T22:04:24.417 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-11-16T22:04:24.446 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-11-16T22:04:24.447 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-11-16T22:04:24.494 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2020-11-16T22:04:24.494 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:24.494 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2020-11-16T22:04:24.494 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="18.04" 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:24.496 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:24.496 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:24.496 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-11-16T22:04:24.509 INFO:teuthology.orchestra.run.smithi062:> uname -m 2020-11-16T22:04:24.521 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2020-11-16T22:04:24.522 INFO:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2020-11-16T22:04:24.569 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2020-11-16T22:04:24.569 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T22:04:24.569 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2020-11-16T22:04:24.569 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="18.04" 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T22:04:24.570 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T22:04:24.571 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=bionic 2020-11-16T22:04:24.571 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=bionic 2020-11-16T22:04:24.571 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2020-11-16T22:04:24.584 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-16T22:04:24.802 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-16T22:04:25.018 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-16T22:04:25.019 INFO:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:04:25.021 INFO:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:04:25.023 INFO:teuthology.orchestra.run.smithi185:> test '!' -e /home/ubuntu/cephtest 2020-11-16T22:04:25.028 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-16T22:04:25.238 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-16T22:04:25.238 INFO:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:04:25.241 INFO:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:04:25.243 INFO:teuthology.orchestra.run.smithi185:> test -z $(ls -A /var/lib/ceph) 2020-11-16T22:04:25.251 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-16T22:04:25.510 INFO:teuthology.run_tasks:Running task kernel... 2020-11-16T22:04:25.730 DEBUG:teuthology.task.kernel:normalized config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}} 2020-11-16T22:04:25.730 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2020-11-16T22:04:25.731 INFO:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2020-11-16T22:04:25.782 INFO:teuthology.orchestra.run.smithi062.stdout:Ubuntu 2020-11-16T22:04:25.788 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:04:25.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=testing 2020-11-16T22:04:26.147 DEBUG:teuthology.task.kernel:sha1 for client.0 is ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:04:26.147 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9"... 2020-11-16T22:04:26.148 INFO:teuthology.orchestra.run.smithi062:> uname -r 2020-11-16T22:04:26.154 INFO:teuthology.orchestra.run.smithi062.stdout:4.15.0-123-generic 2020-11-16T22:04:26.154 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-123-generic vs ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:04:26.155 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2020-11-16T22:04:26.156 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2020-11-16T22:04:26.156 INFO:teuthology.orchestra.run.smithi062:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2020-11-16T22:04:26.200 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2020-11-16T22:04:26.306 INFO:teuthology.orchestra.run.smithi062.stdout:Initialized empty Git repository in /lib/firmware/updates/.git/ 2020-11-16T22:04:26.308 INFO:teuthology.orchestra.run.smithi062:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2020-11-16T22:04:26.386 INFO:teuthology.orchestra.run.smithi062:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2020-11-16T22:04:26.444 INFO:teuthology.orchestra.run.smithi062:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master 2020-11-16T22:04:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2020-11-16T22:04:36.613 INFO:teuthology.orchestra.run.smithi062.stderr: * [new branch] main -> origin/main 2020-11-16T22:04:36.613 INFO:teuthology.orchestra.run.smithi062.stderr: * [new branch] master -> origin/master 2020-11-16T22:04:36.637 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190312 -> 20190312 2020-11-16T22:04:36.637 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190416 -> 20190416 2020-11-16T22:04:36.638 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190514 -> 20190514 2020-11-16T22:04:36.638 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190618 -> 20190618 2020-11-16T22:04:36.638 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190717 -> 20190717 2020-11-16T22:04:36.638 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190815 -> 20190815 2020-11-16T22:04:36.639 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190923 -> 20190923 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20191022 -> 20191022 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20191215 -> 20191215 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200122 -> 20200122 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200316 -> 20200316 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200421 -> 20200421 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200519 -> 20200519 2020-11-16T22:04:36.640 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200619 -> 20200619 2020-11-16T22:04:36.641 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200721 -> 20200721 2020-11-16T22:04:36.641 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200817 -> 20200817 2020-11-16T22:04:36.641 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200918 -> 20200918 2020-11-16T22:04:36.641 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20201022 -> 20201022 2020-11-16T22:04:38.871 INFO:teuthology.orchestra.run.smithi062.stderr:Checking out files: 52% (1240/2348) Checking out files: 53% (1245/2348) Checking out files: 54% (1268/2348) Checking out files: 55% (1292/2348) Checking out files: 56% (1315/2348) Checking out files: 57% (1339/2348) Checking out files: 58% (1362/2348) Checking out files: 59% (1386/2348) Checking out files: 60% (1409/2348) Checking out files: 61% (1433/2348) Checking out files: 62% (1456/2348) Checking out files: 63% (1480/2348) Checking out files: 64% (1503/2348) Checking out files: 65% (1527/2348) Checking out files: 66% (1550/2348) Checking out files: 67% (1574/2348) Checking out files: 68% (1597/2348) Checking out files: 69% (1621/2348) Checking out files: 70% (1644/2348) Checking out files: 71% (1668/2348) Checking out files: 72% (1691/2348) Checking out files: 73% (1715/2348) Checking out files: 74% (1738/2348) Checking out files: 75% (1761/2348) Checking out files: 76% (1785/2348) Checking out files: 77% (1808/2348) Checking out files: 78% (1832/2348) Checking out files: 78% (1843/2348) Checking out files: 79% (1855/2348) Checking out files: 80% (1879/2348) Checking out files: 81% (1902/2348) Checking out files: 82% (1926/2348) Checking out files: 83% (1949/2348) Checking out files: 84% (1973/2348) Checking out files: 85% (1996/2348) Checking out files: 86% (2020/2348) Checking out files: 87% (2043/2348) Checking out files: 88% (2067/2348) Checking out files: 89% (2090/2348) Checking out files: 90% (2114/2348) Checking out files: 91% (2137/2348) Checking out files: 92% (2161/2348) Checking out files: 93% (2184/2348) Checking out files: 94% (2208/2348) Checking out files: 95% (2231/2348) Checking out files: 96% (2255/2348) Checking out files: 97% (2278/2348) Checking out files: 98% (2302/2348) Checking out files: 99% (2325/2348) Checking out files: 100% (2348/2348) Checking out files: 100% (2348/2348), done. 2020-11-16T22:04:38.874 INFO:teuthology.orchestra.run.smithi062.stdout:HEAD is now at 2ea8667 rtlwifi: v88.2 firmware files for RTL8192CU 2020-11-16T22:04:38.884 INFO:teuthology.task.kernel:Downloading kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-16T22:04:38.885 INFO:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2020-11-16T22:04:38.975 INFO:teuthology.orchestra.run.smithi062.stdout:Ubuntu 2020-11-16T22:04:38.979 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:04:38.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-16T22:04:39.151 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:04:39.151 INFO:teuthology.orchestra.run.smithi062:> rm -f /tmp/linux-image.deb && echo linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ --input-file=- 2020-11-16T22:04:39.153 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2020-11-16T22:04:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2020-11-16 22:04:40 URL:https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb [33925808/33925808] -> "/tmp/linux-image.deb" [1] 2020-11-16T22:04:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2020-11-16 22:04:40-- 2020-11-16T22:04:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 1.3s 2020-11-16T22:04:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 32M in 1.2s (27.1 MB/s) 2020-11-16T22:04:40.498 INFO:teuthology.task.kernel:Installing kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-16T22:04:40.499 INFO:teuthology.orchestra.run.smithi062:> sudo dpkg -i /tmp/linux-image.deb 2020-11-16T22:04:40.724 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-image-5.10.0-rc2-ceph-gac4f7b07f403. 2020-11-16T22:04:40.765 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:04:40.768 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack /tmp/linux-image.deb ... 2020-11-16T22:04:40.840 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-16T22:04:43.838 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-16T22:04:44.072 INFO:teuthology.orchestra.run.smithi062.stderr:update-initramfs: Generating /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:01.474 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub' 2020-11-16T22:05:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2020-11-16T22:05:01.786 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:01.825 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T22:05:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T22:05:03.896 INFO:teuthology.orchestra.run.smithi062.stderr:done 2020-11-16T22:05:04.029 INFO:teuthology.orchestra.run.smithi062:> dpkg-deb -c /tmp/linux-image.deb 2020-11-16T22:05:04.038 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./ 2020-11-16T22:05:04.038 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./boot/ 2020-11-16T22:05:04.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 5164573 2020-11-13 14:15 ./boot/System.map-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:04.099 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 153248 2020-11-13 14:15 ./boot/config-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:04.102 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9847776 2020-11-13 14:15 ./boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:04.247 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/ 2020-11-16T22:05:04.247 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postinst.d/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postrm.d/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/preinst.d/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/prerm.d/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/ 2020-11-16T22:05:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:05:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/ 2020-11-16T22:05:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/ 2020-11-16T22:05:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/ 2020-11-16T22:05:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ 2020-11-16T22:05:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 488985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/aesni-intel.ko 2020-11-16T22:05:04.250 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/blowfish-x86_64.ko 2020-11-16T22:05:04.250 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2020-11-16T22:05:04.251 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2020-11-16T22:05:04.251 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-x86_64.ko 2020-11-16T22:05:04.252 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2020-11-16T22:05:04.252 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 143041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2020-11-16T22:05:04.253 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crc32-pclmul.ko 2020-11-16T22:05:04.253 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2020-11-16T22:05:04.253 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2020-11-16T22:05:04.254 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2020-11-16T22:05:04.254 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/glue_helper.ko 2020-11-16T22:05:04.254 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2020-11-16T22:05:04.255 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx2.ko 2020-11-16T22:05:04.256 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2020-11-16T22:05:04.257 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha1-ssse3.ko 2020-11-16T22:05:04.257 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha256-ssse3.ko 2020-11-16T22:05:04.258 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha512-ssse3.ko 2020-11-16T22:05:04.259 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2020-11-16T22:05:04.259 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2020-11-16T22:05:04.259 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64.ko 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/ 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/ 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/ 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpuid.ko 2020-11-16T22:05:04.260 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/msr.ko 2020-11-16T22:05:04.261 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/ 2020-11-16T22:05:04.261 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 247537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-amd.ko 2020-11-16T22:05:04.263 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 826857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-intel.ko 2020-11-16T22:05:04.272 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1834553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm.ko 2020-11-16T22:05:04.295 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/ 2020-11-16T22:05:04.295 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 135385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/oprofile.ko 2020-11-16T22:05:04.296 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ 2020-11-16T22:05:04.296 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/af_alg.ko 2020-11-16T22:05:04.297 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_hash.ko 2020-11-16T22:05:04.297 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_skcipher.ko 2020-11-16T22:05:04.297 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ansi_cprng.ko 2020-11-16T22:05:04.298 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/anubis.ko 2020-11-16T22:05:04.299 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/arc4.ko 2020-11-16T22:05:04.299 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/ 2020-11-16T22:05:04.299 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2020-11-16T22:05:04.299 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/ 2020-11-16T22:05:04.299 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_memcpy.ko 2020-11-16T22:05:04.300 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_pq.ko 2020-11-16T22:05:04.300 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_raid6_recov.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_tx.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_xor.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/raid6test.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authenc.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authencesn.ko 2020-11-16T22:05:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blake2b_generic.ko 2020-11-16T22:05:04.302 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_common.ko 2020-11-16T22:05:04.302 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_generic.ko 2020-11-16T22:05:04.302 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/camellia_generic.ko 2020-11-16T22:05:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast5_generic.ko 2020-11-16T22:05:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast6_generic.ko 2020-11-16T22:05:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast_common.ko 2020-11-16T22:05:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ccm.ko 2020-11-16T22:05:04.303 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cmac.ko 2020-11-16T22:05:04.304 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crc32_generic.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cryptd.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_engine.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_simd.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_user.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cts.ko 2020-11-16T22:05:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/des_generic.ko 2020-11-16T22:05:04.306 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/dh_generic.ko 2020-11-16T22:05:04.306 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/echainiv.ko 2020-11-16T22:05:04.306 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/essiv.ko 2020-11-16T22:05:04.307 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/fcrypt.ko 2020-11-16T22:05:04.307 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/khazad.ko 2020-11-16T22:05:04.307 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lrw.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4hc.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/md4.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/michael_mic.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcbc.ko 2020-11-16T22:05:04.308 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcrypt.ko 2020-11-16T22:05:04.309 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd128.ko 2020-11-16T22:05:04.309 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd160.ko 2020-11-16T22:05:04.309 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd256.ko 2020-11-16T22:05:04.310 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd320.ko 2020-11-16T22:05:04.310 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/salsa20_generic.ko 2020-11-16T22:05:04.310 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/seed.ko 2020-11-16T22:05:04.310 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/serpent_generic.ko 2020-11-16T22:05:04.310 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 170649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tcrypt.ko 2020-11-16T22:05:04.311 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tea.ko 2020-11-16T22:05:04.311 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tgr192.ko 2020-11-16T22:05:04.312 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_common.ko 2020-11-16T22:05:04.312 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_generic.ko 2020-11-16T22:05:04.312 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/vmac.ko 2020-11-16T22:05:04.313 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/wp512.ko 2020-11-16T22:05:04.313 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xcbc.ko 2020-11-16T22:05:04.314 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xor.ko 2020-11-16T22:05:04.314 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xts.ko 2020-11-16T22:05:04.314 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xxhash_generic.ko 2020-11-16T22:05:04.314 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ 2020-11-16T22:05:04.315 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ 2020-11-16T22:05:04.315 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_extlog.ko 2020-11-16T22:05:04.315 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_ipmi.ko 2020-11-16T22:05:04.315 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_pad.ko 2020-11-16T22:05:04.316 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/ 2020-11-16T22:05:04.316 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/einj.ko 2020-11-16T22:05:04.316 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ec_sys.ko 2020-11-16T22:05:04.316 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbs.ko 2020-11-16T22:05:04.317 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbshc.ko 2020-11-16T22:05:04.317 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/video.ko 2020-11-16T22:05:04.317 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ 2020-11-16T22:05:04.317 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/acard-ahci.ko 2020-11-16T22:05:04.318 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci.ko 2020-11-16T22:05:04.318 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci_platform.ko 2020-11-16T22:05:04.318 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci.ko 2020-11-16T22:05:04.319 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci_platform.ko 2020-11-16T22:05:04.319 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_acpi.ko 2020-11-16T22:05:04.319 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ali.ko 2020-11-16T22:05:04.320 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_amd.ko 2020-11-16T22:05:04.320 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_artop.ko 2020-11-16T22:05:04.320 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atiixp.ko 2020-11-16T22:05:04.321 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atp867x.ko 2020-11-16T22:05:04.321 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd640.ko 2020-11-16T22:05:04.322 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd64x.ko 2020-11-16T22:05:04.322 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cypress.ko 2020-11-16T22:05:04.322 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_efar.ko 2020-11-16T22:05:04.322 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt366.ko 2020-11-16T22:05:04.322 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt37x.ko 2020-11-16T22:05:04.323 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x2n.ko 2020-11-16T22:05:04.323 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x3.ko 2020-11-16T22:05:04.324 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it8213.ko 2020-11-16T22:05:04.324 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it821x.ko 2020-11-16T22:05:04.324 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_jmicron.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_legacy.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_marvell.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_mpiix.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_netcell.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ninja32.ko 2020-11-16T22:05:04.325 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87410.ko 2020-11-16T22:05:04.326 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87415.ko 2020-11-16T22:05:04.326 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_oldpiix.ko 2020-11-16T22:05:04.326 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_opti.ko 2020-11-16T22:05:04.326 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_optidma.ko 2020-11-16T22:05:04.326 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pcmcia.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc2027x.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc202xx_old.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_piccolo.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_platform.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_radisys.ko 2020-11-16T22:05:04.328 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rdc.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rz1000.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sch.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_serverworks.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sil680.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sl82c105.ko 2020-11-16T22:05:04.329 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_triflex.ko 2020-11-16T22:05:04.330 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_via.ko 2020-11-16T22:05:04.330 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pdc_adma.ko 2020-11-16T22:05:04.330 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_inic162x.ko 2020-11-16T22:05:04.330 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_mv.ko 2020-11-16T22:05:04.331 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_nv.ko 2020-11-16T22:05:04.331 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_promise.ko 2020-11-16T22:05:04.331 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_qstor.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil24.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sis.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_svw.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sx4.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_uli.ko 2020-11-16T22:05:04.332 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_via.ko 2020-11-16T22:05:04.333 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_vsc.ko 2020-11-16T22:05:04.333 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ 2020-11-16T22:05:04.333 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ks0108.ko 2020-11-16T22:05:04.334 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/ 2020-11-16T22:05:04.334 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 123697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/bcma.ko 2020-11-16T22:05:04.334 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/ 2020-11-16T22:05:04.334 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/ 2020-11-16T22:05:04.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 115129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/aoe.ko 2020-11-16T22:05:04.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/cryptoloop.ko 2020-11-16T22:05:04.335 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/ 2020-11-16T22:05:04.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1050545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/drbd.ko 2020-11-16T22:05:04.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 176513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/floppy.ko 2020-11-16T22:05:04.352 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/ 2020-11-16T22:05:04.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 107353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/mtip32xx.ko 2020-11-16T22:05:04.353 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/nbd.ko 2020-11-16T22:05:04.354 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 92265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/null_blk.ko 2020-11-16T22:05:04.355 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/pktcdvd.ko 2020-11-16T22:05:04.356 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 260233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rbd.ko 2020-11-16T22:05:04.359 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/ 2020-11-16T22:05:04.360 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/rsxx.ko 2020-11-16T22:05:04.361 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 140897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/skd.ko 2020-11-16T22:05:04.363 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/sx8.ko 2020-11-16T22:05:04.363 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/umem.ko 2020-11-16T22:05:04.364 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/ 2020-11-16T22:05:04.364 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/xen-blkback.ko 2020-11-16T22:05:04.365 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/ 2020-11-16T22:05:04.366 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/zram.ko 2020-11-16T22:05:04.366 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ 2020-11-16T22:05:04.366 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/applicom.ko 2020-11-16T22:05:04.367 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hangcheck-timer.ko 2020-11-16T22:05:04.367 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/ 2020-11-16T22:05:04.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/amd-rng.ko 2020-11-16T22:05:04.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/intel-rng.ko 2020-11-16T22:05:04.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/timeriomem-rng.ko 2020-11-16T22:05:04.368 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/via-rng.ko 2020-11-16T22:05:04.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/virtio-rng.ko 2020-11-16T22:05:04.369 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ 2020-11-16T22:05:04.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_devintf.ko 2020-11-16T22:05:04.369 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 138969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2020-11-16T22:05:04.370 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2020-11-16T22:05:04.371 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_si.ko 2020-11-16T22:05:04.372 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_ssif.ko 2020-11-16T22:05:04.373 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2020-11-16T22:05:04.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/lp.ko 2020-11-16T22:05:04.374 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/ 2020-11-16T22:05:04.374 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/mwave.ko 2020-11-16T22:05:04.375 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/nvram.ko 2020-11-16T22:05:04.376 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/ 2020-11-16T22:05:04.376 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4000_cs.ko 2020-11-16T22:05:04.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4040_cs.ko 2020-11-16T22:05:04.377 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/synclink_cs.ko 2020-11-16T22:05:04.378 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ppdev.ko 2020-11-16T22:05:04.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/raw.ko 2020-11-16T22:05:04.379 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tlclk.ko 2020-11-16T22:05:04.380 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/ 2020-11-16T22:05:04.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_atmel.ko 2020-11-16T22:05:04.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2020-11-16T22:05:04.380 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2020-11-16T22:05:04.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2020-11-16T22:05:04.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_infineon.ko 2020-11-16T22:05:04.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_nsc.ko 2020-11-16T22:05:04.381 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/xen-tpmfront.ko 2020-11-16T22:05:04.382 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/ 2020-11-16T22:05:04.382 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_core.ko 2020-11-16T22:05:04.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_pcie.ko 2020-11-16T22:05:04.383 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/ 2020-11-16T22:05:04.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-palmas.ko 2020-11-16T22:05:04.383 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-s2mps11.ko 2020-11-16T22:05:04.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-si5351.ko 2020-11-16T22:05:04.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-twl6040.ko 2020-11-16T22:05:04.384 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-wm831x.ko 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/ 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/p4-clockmod.ko 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/speedstep-lib.ko 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ 2020-11-16T22:05:04.385 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp-crypto.ko 2020-11-16T22:05:04.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 203753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp.ko 2020-11-16T22:05:04.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-aes.ko 2020-11-16T22:05:04.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-sha.ko 2020-11-16T22:05:04.389 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/ 2020-11-16T22:05:04.389 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/ 2020-11-16T22:05:04.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 279065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2020-11-16T22:05:04.392 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/ 2020-11-16T22:05:04.392 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2020-11-16T22:05:04.393 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/ 2020-11-16T22:05:04.393 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/virtio_crypto.ko 2020-11-16T22:05:04.393 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/ 2020-11-16T22:05:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/dca.ko 2020-11-16T22:05:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ 2020-11-16T22:05:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/ 2020-11-16T22:05:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/dw_dmac.ko 2020-11-16T22:05:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ 2020-11-16T22:05:04.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ioatdma.ko 2020-11-16T22:05:04.395 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ 2020-11-16T22:05:04.395 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/amd64_edac_mod.ko 2020-11-16T22:05:04.396 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/e752x_edac.ko 2020-11-16T22:05:04.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/edac_mce_amd.ko 2020-11-16T22:05:04.397 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3000_edac.ko 2020-11-16T22:05:04.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3200_edac.ko 2020-11-16T22:05:04.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5000_edac.ko 2020-11-16T22:05:04.398 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5100_edac.ko 2020-11-16T22:05:04.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5400_edac.ko 2020-11-16T22:05:04.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7300_edac.ko 2020-11-16T22:05:04.399 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7core_edac.ko 2020-11-16T22:05:04.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i82975x_edac.ko 2020-11-16T22:05:04.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ie31200_edac.ko 2020-11-16T22:05:04.400 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/sb_edac.ko 2020-11-16T22:05:04.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/x38_edac.ko 2020-11-16T22:05:04.401 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/ 2020-11-16T22:05:04.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-gpio.ko 2020-11-16T22:05:04.401 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max14577.ko 2020-11-16T22:05:04.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max77693.ko 2020-11-16T22:05:04.402 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max8997.ko 2020-11-16T22:05:04.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-palmas.ko 2020-11-16T22:05:04.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-rt8973a.ko 2020-11-16T22:05:04.403 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-sm5502.ko 2020-11-16T22:05:04.404 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/ 2020-11-16T22:05:04.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/dmi-sysfs.ko 2020-11-16T22:05:04.404 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/ 2020-11-16T22:05:04.404 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/efi-pstore.ko 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/iscsi_ibft.ko 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/ 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5520.ko 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5588.ko 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-amd8111.ko 2020-11-16T22:05:04.405 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-arizona.ko 2020-11-16T22:05:04.406 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9052.ko 2020-11-16T22:05:04.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9055.ko 2020-11-16T22:05:04.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-dln2.ko 2020-11-16T22:05:04.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-f7188x.ko 2020-11-16T22:05:04.407 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-generic.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ich.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-janz-ttl.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-kempld.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-lp3943.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max7300.ko 2020-11-16T22:05:04.408 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max730x.ko 2020-11-16T22:05:04.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max732x.ko 2020-11-16T22:05:04.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ml-ioh.ko 2020-11-16T22:05:04.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pca953x.ko 2020-11-16T22:05:04.409 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pcf857x.ko 2020-11-16T22:05:04.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-rdc321x.ko 2020-11-16T22:05:04.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch.ko 2020-11-16T22:05:04.411 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch311x.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-tps65912.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl4030.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl6040.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-viperboard.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-vx855.ko 2020-11-16T22:05:04.412 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm831x.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8350.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8994.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/ 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-a4tech.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-apple.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-appleir.ko 2020-11-16T22:05:04.413 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-aureal.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-axff.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-belkin.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cherry.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-chicony.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cp2112.ko 2020-11-16T22:05:04.414 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cypress.ko 2020-11-16T22:05:04.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-dr.ko 2020-11-16T22:05:04.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elecom.ko 2020-11-16T22:05:04.415 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elo.ko 2020-11-16T22:05:04.416 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-emsff.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ezkey.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gaff.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-generic.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gt683r.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gyration.ko 2020-11-16T22:05:04.417 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-kbd.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-mouse.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtekff.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-hyperv.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-icade.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kensington.ko 2020-11-16T22:05:04.418 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-keytouch.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kye.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lcpower.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-led.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lenovo.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lg-g15.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-dj.ko 2020-11-16T22:05:04.419 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-hidpp.ko 2020-11-16T22:05:04.420 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech.ko 2020-11-16T22:05:04.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-magicmouse.ko 2020-11-16T22:05:04.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-microsoft.ko 2020-11-16T22:05:04.421 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-monterey.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-multitouch.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ntrig.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ortek.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-penmount.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-petalynx.ko 2020-11-16T22:05:04.422 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-picolcd.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-pl.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-plantronics.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-primax.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-rmi.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-arvo.ko 2020-11-16T22:05:04.423 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-common.ko 2020-11-16T22:05:04.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-isku.ko 2020-11-16T22:05:04.424 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kone.ko 2020-11-16T22:05:04.425 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-koneplus.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-konepure.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kovaplus.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-lua.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-pyra.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-ryos.ko 2020-11-16T22:05:04.426 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-savu.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-saitek.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-samsung.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sensor-hub.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sjoy.ko 2020-11-16T22:05:04.427 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sony.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-speedlink.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-steelseries.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sunplus.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tivo.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tmff.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-topseed.ko 2020-11-16T22:05:04.428 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-twinhan.ko 2020-11-16T22:05:04.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-uclogic.ko 2020-11-16T22:05:04.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-waltop.ko 2020-11-16T22:05:04.429 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-wiimote.ko 2020-11-16T22:05:04.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-xinmo.ko 2020-11-16T22:05:04.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zpff.ko 2020-11-16T22:05:04.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zydacron.ko 2020-11-16T22:05:04.430 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 284097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid.ko 2020-11-16T22:05:04.431 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/ 2020-11-16T22:05:04.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/i2c-hid.ko 2020-11-16T22:05:04.432 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/uhid.ko 2020-11-16T22:05:04.432 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/ 2020-11-16T22:05:04.433 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 125177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbhid.ko 2020-11-16T22:05:04.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbkbd.ko 2020-11-16T22:05:04.434 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbmouse.ko 2020-11-16T22:05:04.435 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 222905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/wacom.ko 2020-11-16T22:05:04.437 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/ 2020-11-16T22:05:04.437 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_balloon.ko 2020-11-16T22:05:04.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_utils.ko 2020-11-16T22:05:04.438 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 277809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_vmbus.ko 2020-11-16T22:05:04.441 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ 2020-11-16T22:05:04.442 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru.ko 2020-11-16T22:05:04.442 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru3.ko 2020-11-16T22:05:04.443 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/acpi_power_meter.ko 2020-11-16T22:05:04.443 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7414.ko 2020-11-16T22:05:04.443 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7418.ko 2020-11-16T22:05:04.444 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adc128d818.ko 2020-11-16T22:05:04.444 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1021.ko 2020-11-16T22:05:04.444 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1025.ko 2020-11-16T22:05:04.444 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1026.ko 2020-11-16T22:05:04.445 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1029.ko 2020-11-16T22:05:04.445 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1031.ko 2020-11-16T22:05:04.446 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm9240.ko 2020-11-16T22:05:04.446 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ads7828.ko 2020-11-16T22:05:04.446 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7410.ko 2020-11-16T22:05:04.447 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7411.ko 2020-11-16T22:05:04.447 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7462.ko 2020-11-16T22:05:04.447 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7470.ko 2020-11-16T22:05:04.448 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7475.ko 2020-11-16T22:05:04.448 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7x10.ko 2020-11-16T22:05:04.449 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/amc6821.ko 2020-11-16T22:05:04.449 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/applesmc.ko 2020-11-16T22:05:04.449 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asb100.ko 2020-11-16T22:05:04.450 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asc7621.ko 2020-11-16T22:05:04.450 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asus_atk0110.ko 2020-11-16T22:05:04.451 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/atxp1.ko 2020-11-16T22:05:04.451 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/coretemp.ko 2020-11-16T22:05:04.452 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9052-hwmon.ko 2020-11-16T22:05:04.452 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9055-hwmon.ko 2020-11-16T22:05:04.452 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dell-smm-hwmon.ko 2020-11-16T22:05:04.452 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 87601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dme1737.ko 2020-11-16T22:05:04.453 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds1621.ko 2020-11-16T22:05:04.454 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds620.ko 2020-11-16T22:05:04.454 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc1403.ko 2020-11-16T22:05:04.454 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc2103.ko 2020-11-16T22:05:04.454 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc6w201.ko 2020-11-16T22:05:04.455 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71805f.ko 2020-11-16T22:05:04.455 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71882fg.ko 2020-11-16T22:05:04.456 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f75375s.ko 2020-11-16T22:05:04.456 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fam15h_power.ko 2020-11-16T22:05:04.456 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fschmd.ko 2020-11-16T22:05:04.457 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g760a.ko 2020-11-16T22:05:04.457 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g762.ko 2020-11-16T22:05:04.457 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl518sm.ko 2020-11-16T22:05:04.458 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl520sm.ko 2020-11-16T22:05:04.458 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hih6130.ko 2020-11-16T22:05:04.458 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hwmon-vid.ko 2020-11-16T22:05:04.459 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5500_temp.ko 2020-11-16T22:05:04.459 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5k_amb.ko 2020-11-16T22:05:04.459 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmaem.ko 2020-11-16T22:05:04.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmpex.ko 2020-11-16T22:05:04.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina209.ko 2020-11-16T22:05:04.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina2xx.ko 2020-11-16T22:05:04.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/it87.ko 2020-11-16T22:05:04.461 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/jc42.ko 2020-11-16T22:05:04.461 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k10temp.ko 2020-11-16T22:05:04.462 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k8temp.ko 2020-11-16T22:05:04.462 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lineage-pem.ko 2020-11-16T22:05:04.462 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm63.ko 2020-11-16T22:05:04.463 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm73.ko 2020-11-16T22:05:04.463 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm75.ko 2020-11-16T22:05:04.463 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm77.ko 2020-11-16T22:05:04.463 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm78.ko 2020-11-16T22:05:04.463 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm80.ko 2020-11-16T22:05:04.464 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm83.ko 2020-11-16T22:05:04.464 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm85.ko 2020-11-16T22:05:04.464 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm87.ko 2020-11-16T22:05:04.465 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm90.ko 2020-11-16T22:05:04.465 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm92.ko 2020-11-16T22:05:04.465 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 104761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm93.ko 2020-11-16T22:05:04.466 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95234.ko 2020-11-16T22:05:04.467 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95241.ko 2020-11-16T22:05:04.467 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95245.ko 2020-11-16T22:05:04.467 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc2945.ko 2020-11-16T22:05:04.468 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4151.ko 2020-11-16T22:05:04.468 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4215.ko 2020-11-16T22:05:04.468 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4222.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4245.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4260.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4261.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max16065.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1619.ko 2020-11-16T22:05:04.469 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1668.ko 2020-11-16T22:05:04.470 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max197.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6639.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6642.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6650.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6697.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mc13783-adc.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mcp3021.ko 2020-11-16T22:05:04.471 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2020-11-16T22:05:04.472 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6683.ko 2020-11-16T22:05:04.472 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6775.ko 2020-11-16T22:05:04.472 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct7802.ko 2020-11-16T22:05:04.473 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ntc_thermistor.ko 2020-11-16T22:05:04.473 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87360.ko 2020-11-16T22:05:04.473 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87427.ko 2020-11-16T22:05:04.474 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pcf8591.ko 2020-11-16T22:05:04.474 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ 2020-11-16T22:05:04.474 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/adm1275.ko 2020-11-16T22:05:04.475 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/lm25066.ko 2020-11-16T22:05:04.475 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ltc2978.ko 2020-11-16T22:05:04.475 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max16064.ko 2020-11-16T22:05:04.476 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max34440.ko 2020-11-16T22:05:04.476 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max8688.ko 2020-11-16T22:05:04.476 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus.ko 2020-11-16T22:05:04.476 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2020-11-16T22:05:04.477 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/tps40422.ko 2020-11-16T22:05:04.477 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9000.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9200.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/zl6100.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/powr1220.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5627.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5636.ko 2020-11-16T22:05:04.478 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch56xx-common.ko 2020-11-16T22:05:04.479 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht15.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht21.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/shtc1.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sis5595.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smm665.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47b397.ko 2020-11-16T22:05:04.480 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m1.ko 2020-11-16T22:05:04.481 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m192.ko 2020-11-16T22:05:04.481 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/thmc50.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp102.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp103.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp401.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp421.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via-cputemp.ko 2020-11-16T22:05:04.482 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via686a.ko 2020-11-16T22:05:04.483 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt1211.ko 2020-11-16T22:05:04.483 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt8231.ko 2020-11-16T22:05:04.483 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627ehf.ko 2020-11-16T22:05:04.483 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627hf.ko 2020-11-16T22:05:04.484 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83781d.ko 2020-11-16T22:05:04.484 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83791d.ko 2020-11-16T22:05:04.484 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83792d.ko 2020-11-16T22:05:04.485 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83793.ko 2020-11-16T22:05:04.486 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83795.ko 2020-11-16T22:05:04.487 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l785ts.ko 2020-11-16T22:05:04.487 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l786ng.ko 2020-11-16T22:05:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm831x-hwmon.ko 2020-11-16T22:05:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm8350-hwmon.ko 2020-11-16T22:05:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/ 2020-11-16T22:05:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/ 2020-11-16T22:05:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2020-11-16T22:05:04.489 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2020-11-16T22:05:04.490 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/ 2020-11-16T22:05:04.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1535.ko 2020-11-16T22:05:04.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1563.ko 2020-11-16T22:05:04.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2020-11-16T22:05:04.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2020-11-16T22:05:04.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756.ko 2020-11-16T22:05:04.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd8111.ko 2020-11-16T22:05:04.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2020-11-16T22:05:04.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2020-11-16T22:05:04.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-core.ko 2020-11-16T22:05:04.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2020-11-16T22:05:04.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-dln2.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-gpio.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-i801.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-isch.ko 2020-11-16T22:05:04.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ismt.ko 2020-11-16T22:05:04.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-kempld.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ocores.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-parport.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2020-11-16T22:05:04.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-piix4.ko 2020-11-16T22:05:04.496 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2020-11-16T22:05:04.496 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-scmi.ko 2020-11-16T22:05:04.497 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-simtec.ko 2020-11-16T22:05:04.497 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis5595.ko 2020-11-16T22:05:04.497 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis630.ko 2020-11-16T22:05:04.497 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis96x.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-via.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viapro.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viperboard.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-xiic.ko 2020-11-16T22:05:04.498 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-mux.ko 2020-11-16T22:05:04.499 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-smbus.ko 2020-11-16T22:05:04.499 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-stub.ko 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/ 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ 2020-11-16T22:05:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ 2020-11-16T22:05:04.501 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 259017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_cm.ko 2020-11-16T22:05:04.501 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 848849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_core.ko 2020-11-16T22:05:04.511 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_umad.ko 2020-11-16T22:05:04.512 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 309825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_uverbs.ko 2020-11-16T22:05:04.516 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/iw_cm.ko 2020-11-16T22:05:04.517 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 263657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_cm.ko 2020-11-16T22:05:04.520 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_ucm.ko 2020-11-16T22:05:04.521 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ 2020-11-16T22:05:04.521 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/ 2020-11-16T22:05:04.521 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 512473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2020-11-16T22:05:04.527 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/ 2020-11-16T22:05:04.528 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 420329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2020-11-16T22:05:04.533 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/ 2020-11-16T22:05:04.533 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 721305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2020-11-16T22:05:04.543 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ 2020-11-16T22:05:04.543 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 287121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2020-11-16T22:05:04.547 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ 2020-11-16T22:05:04.547 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 162257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2020-11-16T22:05:04.549 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/ 2020-11-16T22:05:04.549 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 177449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2020-11-16T22:05:04.551 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ 2020-11-16T22:05:04.551 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ 2020-11-16T22:05:04.551 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 272041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2020-11-16T22:05:04.554 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ 2020-11-16T22:05:04.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 105369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2020-11-16T22:05:04.556 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ 2020-11-16T22:05:04.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 114745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2020-11-16T22:05:04.557 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ 2020-11-16T22:05:04.557 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2020-11-16T22:05:04.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ 2020-11-16T22:05:04.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 131153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2020-11-16T22:05:04.560 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ 2020-11-16T22:05:04.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/evbug.ko 2020-11-16T22:05:04.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ff-memless.ko 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/emu10k1-gp.ko 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/fm801-gp.ko 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/gameport.ko 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/lightning.ko 2020-11-16T22:05:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ns558.ko 2020-11-16T22:05:04.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/input-polldev.ko 2020-11-16T22:05:04.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/joydev.ko 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/ 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5520-keys.ko 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5588-keys.ko 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5589-keys.ko 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2020-11-16T22:05:04.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys.ko 2020-11-16T22:05:04.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2020-11-16T22:05:04.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lkkbd.ko 2020-11-16T22:05:04.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8323.ko 2020-11-16T22:05:04.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8333.ko 2020-11-16T22:05:04.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/matrix_keypad.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/max7359_keypad.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mcs_touchkey.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/newtonkbd.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/opencores-kbd.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt1070.ko 2020-11-16T22:05:04.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt2160.ko 2020-11-16T22:05:04.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/samsung-keypad.ko 2020-11-16T22:05:04.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/stowaway.ko 2020-11-16T22:05:04.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/sunkbd.ko 2020-11-16T22:05:04.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca6416-keypad.ko 2020-11-16T22:05:04.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca8418_keypad.ko 2020-11-16T22:05:04.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/twl4030_keypad.ko 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/xtkbd.ko 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/matrix-keymap.ko 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/ 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/appletouch.ko 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/bcm5974.ko 2020-11-16T22:05:04.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/cyapatp.ko 2020-11-16T22:05:04.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/elan_i2c.ko 2020-11-16T22:05:04.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/gpio_mouse.ko 2020-11-16T22:05:04.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 337393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/psmouse.ko 2020-11-16T22:05:04.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/sermouse.ko 2020-11-16T22:05:04.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_i2c.ko 2020-11-16T22:05:04.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_usb.ko 2020-11-16T22:05:04.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/vsxxxaa.ko 2020-11-16T22:05:04.576 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/ 2020-11-16T22:05:04.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/rmi_core.ko 2020-11-16T22:05:04.578 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ 2020-11-16T22:05:04.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/altera_ps2.ko 2020-11-16T22:05:04.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/arc_ps2.ko 2020-11-16T22:05:04.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ct82c710.ko 2020-11-16T22:05:04.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/hyperv-keyboard.ko 2020-11-16T22:05:04.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/parkbd.ko 2020-11-16T22:05:04.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/pcips2.ko 2020-11-16T22:05:04.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ps2mult.ko 2020-11-16T22:05:04.580 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serio_raw.ko 2020-11-16T22:05:04.580 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serport.ko 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/sparse-keymap.ko 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/ 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/ 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/iommu_v2.ko 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/ 2020-11-16T22:05:04.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-88pm860x.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-adp5520.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-bd2802.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-blinkm.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-clevo-mail.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da903x.ko 2020-11-16T22:05:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da9052.ko 2020-11-16T22:05:04.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-gpio.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3530.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3533.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm355x.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3642.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp3944.ko 2020-11-16T22:05:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp8788.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-max8997.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-mc13783.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-menf21bmc.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca9532.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca955x.ko 2020-11-16T22:05:04.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca963x.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-ss4200.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-tca6507.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm831x-status.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm8350.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-audio.ko 2020-11-16T22:05:04.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-camera.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-timer.ko 2020-11-16T22:05:04.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-transient.ko 2020-11-16T22:05:04.589 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/ 2020-11-16T22:05:04.589 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/ 2020-11-16T22:05:04.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 633113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/bcache.ko 2020-11-16T22:05:04.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bio-prison.ko 2020-11-16T22:05:04.593 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bufio.ko 2020-11-16T22:05:04.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache-smq.ko 2020-11-16T22:05:04.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 146689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache.ko 2020-11-16T22:05:04.596 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-crypt.ko 2020-11-16T22:05:04.597 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-delay.ko 2020-11-16T22:05:04.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-era.ko 2020-11-16T22:05:04.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-flakey.ko 2020-11-16T22:05:04.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log-userspace.ko 2020-11-16T22:05:04.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log.ko 2020-11-16T22:05:04.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-mirror.ko 2020-11-16T22:05:04.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-multipath.ko 2020-11-16T22:05:04.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-queue-length.ko 2020-11-16T22:05:04.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-raid.ko 2020-11-16T22:05:04.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-region-hash.ko 2020-11-16T22:05:04.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-round-robin.ko 2020-11-16T22:05:04.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-service-time.ko 2020-11-16T22:05:04.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-snapshot.ko 2020-11-16T22:05:04.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-switch.ko 2020-11-16T22:05:04.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 182441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-thin-pool.ko 2020-11-16T22:05:04.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-verity.ko 2020-11-16T22:05:04.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-zero.ko 2020-11-16T22:05:04.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/faulty.ko 2020-11-16T22:05:04.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/linear.ko 2020-11-16T22:05:04.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/multipath.ko 2020-11-16T22:05:04.608 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/ 2020-11-16T22:05:04.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 179553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2020-11-16T22:05:04.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid0.ko 2020-11-16T22:05:04.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 111753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid1.ko 2020-11-16T22:05:04.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 156737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid10.ko 2020-11-16T22:05:04.613 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 385993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid456.ko 2020-11-16T22:05:04.618 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/ 2020-11-16T22:05:04.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/ 2020-11-16T22:05:04.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 194137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptbase.ko 2020-11-16T22:05:04.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptctl.ko 2020-11-16T22:05:04.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptfc.ko 2020-11-16T22:05:04.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptlan.ko 2020-11-16T22:05:04.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 129617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptsas.ko 2020-11-16T22:05:04.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptscsih.ko 2020-11-16T22:05:04.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptspi.ko 2020-11-16T22:05:04.626 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ 2020-11-16T22:05:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm800.ko 2020-11-16T22:05:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm805.ko 2020-11-16T22:05:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm80x.ko 2020-11-16T22:05:04.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ab3100-otp.ko 2020-11-16T22:05:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/arizona-i2c.ko 2020-11-16T22:05:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/bcm590xx.ko 2020-11-16T22:05:04.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/cros_ec_dev.ko 2020-11-16T22:05:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/dln2.ko 2020-11-16T22:05:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/htc-pasic3.ko 2020-11-16T22:05:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/janz-cmodio.ko 2020-11-16T22:05:04.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/kempld-core.ko 2020-11-16T22:05:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-core.ko 2020-11-16T22:05:04.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-ctrlbank.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lp3943.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_ich.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_sch.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/max8907.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-core.ko 2020-11-16T22:05:04.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-i2c.ko 2020-11-16T22:05:04.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/menf21bmc.ko 2020-11-16T22:05:04.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-adc.ko 2020-11-16T22:05:04.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-gpio.ko 2020-11-16T22:05:04.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633.ko 2020-11-16T22:05:04.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/rdc321x-southbridge.ko 2020-11-16T22:05:04.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/retu-mfd.ko 2020-11-16T22:05:04.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/si476x-core.ko 2020-11-16T22:05:04.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/sm501.ko 2020-11-16T22:05:04.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ti_am335x_tscadc.ko 2020-11-16T22:05:04.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps65010.ko 2020-11-16T22:05:04.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps6507x.ko 2020-11-16T22:05:04.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/viperboard.ko 2020-11-16T22:05:04.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/vx855.ko 2020-11-16T22:05:04.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wl1273-core.ko 2020-11-16T22:05:04.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wm8994.ko 2020-11-16T22:05:04.635 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ 2020-11-16T22:05:04.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot-i2c.ko 2020-11-16T22:05:04.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot.ko 2020-11-16T22:05:04.636 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/ 2020-11-16T22:05:04.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2020-11-16T22:05:04.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds9802als.ko 2020-11-16T22:05:04.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds990x.ko 2020-11-16T22:05:04.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/bh1770glc.ko 2020-11-16T22:05:04.637 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/ 2020-11-16T22:05:04.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2020-11-16T22:05:04.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/core.ko 2020-11-16T22:05:04.638 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/ 2020-11-16T22:05:04.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/cb710.ko 2020-11-16T22:05:04.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ds1682.ko 2020-11-16T22:05:04.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/dummy-irq.ko 2020-11-16T22:05:04.639 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/ 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/echo.ko 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/ 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/at24.ko 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom.ko 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2020-11-16T22:05:04.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/max6875.ko 2020-11-16T22:05:04.641 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/enclosure.ko 2020-11-16T22:05:04.641 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/ 2020-11-16T22:05:04.641 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 151177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/genwqe_card.ko 2020-11-16T22:05:04.641 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hmc6352.ko 2020-11-16T22:05:04.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hpilo.ko 2020-11-16T22:05:04.642 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ 2020-11-16T22:05:04.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ibmasm.ko 2020-11-16T22:05:04.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ics932s401.ko 2020-11-16T22:05:04.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29003.ko 2020-11-16T22:05:04.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29020.ko 2020-11-16T22:05:04.643 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/ 2020-11-16T22:05:04.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2020-11-16T22:05:04.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2020-11-16T22:05:04.644 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/ 2020-11-16T22:05:04.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 121553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-me.ko 2020-11-16T22:05:04.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-txe.ko 2020-11-16T22:05:04.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 279201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei.ko 2020-11-16T22:05:04.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/phantom.ko 2020-11-16T22:05:04.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/pvpanic.ko 2020-11-16T22:05:04.649 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/ 2020-11-16T22:05:04.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/st_drv.ko 2020-11-16T22:05:04.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_7xx1.ko 2020-11-16T22:05:04.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_core.ko 2020-11-16T22:05:04.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tsl2550.ko 2020-11-16T22:05:04.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_balloon.ko 2020-11-16T22:05:04.651 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/ 2020-11-16T22:05:04.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 179385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2020-11-16T22:05:04.654 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ 2020-11-16T22:05:04.654 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/ 2020-11-16T22:05:04.654 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 433953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/bonding.ko 2020-11-16T22:05:04.659 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/ 2020-11-16T22:05:04.659 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/ 2020-11-16T22:05:04.660 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/b53_common.ko 2020-11-16T22:05:04.660 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/bcm-sf2.ko 2020-11-16T22:05:04.661 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6060.ko 2020-11-16T22:05:04.661 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/ 2020-11-16T22:05:04.661 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 346377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2020-11-16T22:05:04.665 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dummy.ko 2020-11-16T22:05:04.665 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/eql.ko 2020-11-16T22:05:04.666 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ 2020-11-16T22:05:04.666 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/ 2020-11-16T22:05:04.666 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2020-11-16T22:05:04.666 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2020-11-16T22:05:04.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 123353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c59x.ko 2020-11-16T22:05:04.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/typhoon.ko 2020-11-16T22:05:04.669 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ 2020-11-16T22:05:04.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/8390.ko 2020-11-16T22:05:04.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2020-11-16T22:05:04.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2020-11-16T22:05:04.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2020-11-16T22:05:04.671 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/ 2020-11-16T22:05:04.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/starfire.ko 2020-11-16T22:05:04.672 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/ 2020-11-16T22:05:04.672 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/et131x.ko 2020-11-16T22:05:04.673 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/ 2020-11-16T22:05:04.673 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/acenic.ko 2020-11-16T22:05:04.674 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/ 2020-11-16T22:05:04.674 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/altera_tse.ko 2020-11-16T22:05:04.674 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/ 2020-11-16T22:05:04.674 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/amd8111e.ko 2020-11-16T22:05:04.675 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2020-11-16T22:05:04.675 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/pcnet32.ko 2020-11-16T22:05:04.676 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/ 2020-11-16T22:05:04.676 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/ 2020-11-16T22:05:04.677 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2020-11-16T22:05:04.677 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/ 2020-11-16T22:05:04.678 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2020-11-16T22:05:04.679 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/ 2020-11-16T22:05:04.679 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2020-11-16T22:05:04.680 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/ 2020-11-16T22:05:04.680 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2020-11-16T22:05:04.681 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2020-11-16T22:05:04.682 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/ 2020-11-16T22:05:04.682 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/b44.ko 2020-11-16T22:05:04.683 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 167025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2020-11-16T22:05:04.685 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2020-11-16T22:05:04.686 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1528577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2020-11-16T22:05:04.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 130113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/cnic.ko 2020-11-16T22:05:04.708 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/ 2020-11-16T22:05:04.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 129345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2020-11-16T22:05:04.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 314625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/tg3.ko 2020-11-16T22:05:04.715 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/ 2020-11-16T22:05:04.715 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/ 2020-11-16T22:05:04.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 330209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2020-11-16T22:05:04.719 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/ 2020-11-16T22:05:04.719 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/ 2020-11-16T22:05:04.719 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 187345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2020-11-16T22:05:04.721 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2020-11-16T22:05:04.721 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 349953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2020-11-16T22:05:04.726 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2020-11-16T22:05:04.940 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1039081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2020-11-16T22:05:04.940 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 165321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/ 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/ 2020-11-16T22:05:04.941 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 205121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/ 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/ 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 97801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de4x5.ko 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2020-11-16T22:05:04.942 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2020-11-16T22:05:04.943 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2020-11-16T22:05:04.943 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2020-11-16T22:05:04.943 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/ 2020-11-16T22:05:04.943 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/dl2k.ko 2020-11-16T22:05:04.943 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/sundance.ko 2020-11-16T22:05:05.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dnet.ko 2020-11-16T22:05:05.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ec_bhf.ko 2020-11-16T22:05:05.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/ 2020-11-16T22:05:05.146 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/ 2020-11-16T22:05:05.146 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 287161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ethoc.ko 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fealnx.ko 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/ 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ 2020-11-16T22:05:05.147 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e100.ko 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/ 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 293081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/ 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 549313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/ 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 247433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/ 2020-11-16T22:05:05.148 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 892665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/ 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 250857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/ 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 509433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/igb.ko 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/ 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2020-11-16T22:05:05.149 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 123905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 743497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/jme.ko 2020-11-16T22:05:05.150 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/ 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 103057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/skge.ko 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 130193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/sky2.ko 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/ 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/ 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 739097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2020-11-16T22:05:05.151 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 273433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2020-11-16T22:05:05.354 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/ 2020-11-16T22:05:05.354 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2020-11-16T22:05:05.354 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 920849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2020-11-16T22:05:05.354 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ 2020-11-16T22:05:05.354 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8842.ko 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8851_mll.ko 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/ 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/ 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ 2020-11-16T22:05:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/ 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 156657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/s2io.ko 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/ 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 221913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/vxge.ko 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/ 2020-11-16T22:05:05.356 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2020-11-16T22:05:05.357 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/ 2020-11-16T22:05:05.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2020-11-16T22:05:05.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2020-11-16T22:05:05.558 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/ 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/ 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 242793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 706337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/ 2020-11-16T22:05:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/r6040.ko 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/ 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139cp.ko 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139too.ko 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/atp.ko 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 173241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/r8169.ko 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/ 2020-11-16T22:05:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/ 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 104689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/ 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 824569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/sfc.ko 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/ 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/sc92031.ko 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/ 2020-11-16T22:05:05.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis190.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis900.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/ 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/epic100.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2020-11-16T22:05:05.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/ 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/ 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 381753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/ 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/cassini.ko 2020-11-16T22:05:05.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 176753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/niu.ko 2020-11-16T22:05:05.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sungem.ko 2020-11-16T22:05:05.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sunhme.ko 2020-11-16T22:05:05.766 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/ 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/ 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/tlan.ko 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/ 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-rhine.ko 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-velocity.ko 2020-11-16T22:05:05.767 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/ 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5100.ko 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5300.ko 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/ 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/ 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/defxx.ko 2020-11-16T22:05:05.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/ 2020-11-16T22:05:05.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 236713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/skfp.ko 2020-11-16T22:05:05.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/geneve.ko 2020-11-16T22:05:05.769 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/ 2020-11-16T22:05:05.970 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 227313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/hv_netvsc.ko 2020-11-16T22:05:05.970 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/ 2020-11-16T22:05:05.970 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/fakelb.ko 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ifb.ko 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ipvlan.ko 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvlan.ko 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvtap.ko 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/ 2020-11-16T22:05:05.971 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bitbang.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-gpio.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mii.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/netconsole.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/nlmon.ko 2020-11-16T22:05:05.972 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/ 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/pcs-xpcs.ko 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/amd.ko 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm-phy-lib.ko 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm7xxx.ko 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm87xx.ko 2020-11-16T22:05:05.973 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/broadcom.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/cicada.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/davicom.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/et1011c.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/icplus.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/lxt.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/marvell.ko 2020-11-16T22:05:05.974 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/micrel.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/national.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/phylink.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/qsemi.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/realtek.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/smsc.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ste10Xp.ko 2020-11-16T22:05:05.975 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/vitesse.ko 2020-11-16T22:05:05.976 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/ 2020-11-16T22:05:05.976 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/plip.ko 2020-11-16T22:05:05.976 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ 2020-11-16T22:05:05.976 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/bsd_comp.ko 2020-11-16T22:05:06.182 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_async.ko 2020-11-16T22:05:06.182 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_deflate.ko 2020-11-16T22:05:06.182 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_mppe.ko 2020-11-16T22:05:06.182 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_synctty.ko 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppoe.ko 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppox.ko 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pptp.ko 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sb1000.ko 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/ 2020-11-16T22:05:06.183 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/slip.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sungem_phy.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/tap.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/ 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_activebackup.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_broadcast.ko 2020-11-16T22:05:06.184 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_loadbalance.ko 2020-11-16T22:05:06.185 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_random.ko 2020-11-16T22:05:06.185 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_roundrobin.ko 2020-11-16T22:05:06.185 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/veth.ko 2020-11-16T22:05:06.185 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/ 2020-11-16T22:05:06.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 116905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/vmxnet3.ko 2020-11-16T22:05:06.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 171137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vxlan.ko 2020-11-16T22:05:06.386 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/ 2020-11-16T22:05:06.386 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 143721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/xen-netback.ko 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ntb.ko 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/ 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/ 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 209665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme-core.ko 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 92745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme.ko 2020-11-16T22:05:06.387 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/ 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 130825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport.ko 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_ax88796.ko 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_cs.ko 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_pc.ko 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_serial.ko 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/ 2020-11-16T22:05:06.388 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/ 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/pci-stub.ko 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/xen-pcifront.ko 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/ 2020-11-16T22:05:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/i82092.ko 2020-11-16T22:05:06.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 150113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia.ko 2020-11-16T22:05:06.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_core.ko 2020-11-16T22:05:06.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2020-11-16T22:05:06.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pd6729.ko 2020-11-16T22:05:06.390 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 111129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/yenta_socket.ko 2020-11-16T22:05:06.458 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/ 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/ 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/ 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/ 2020-11-16T22:05:06.459 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/ 2020-11-16T22:05:06.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2020-11-16T22:05:06.460 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/ 2020-11-16T22:05:06.460 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/ 2020-11-16T22:05:06.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2020-11-16T22:05:06.460 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec.ko 2020-11-16T22:05:06.666 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2020-11-16T22:05:06.666 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2020-11-16T22:05:06.666 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acer-wmi.ko 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acerhdf.ko 2020-11-16T22:05:06.667 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/alienware-wmi.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/apple-gmux.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-laptop.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-nb-wmi.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-wmi.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/classmate-laptop.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dcdbas.ko 2020-11-16T22:05:06.668 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-smo8800.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-wmi-aio.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell_rbu.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-laptop.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-wmi.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-laptop.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-tablet.ko 2020-11-16T22:05:06.669 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hdaps.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wireless.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wmi.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp_accel.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ibm_rtl.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-rst.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-smartconnect.ko 2020-11-16T22:05:06.670 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_ips.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_menlow.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/msi-wmi.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/mxm-wmi.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/panasonic-laptop.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-laptop.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-q10.ko 2020-11-16T22:05:06.671 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 260993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/thinkpad_acpi.ko 2020-11-16T22:05:06.672 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/topstar-laptop.ko 2020-11-16T22:05:06.672 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2020-11-16T22:05:06.874 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_haps.ko 2020-11-16T22:05:06.874 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi-bmof.ko 2020-11-16T22:05:06.874 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi.ko 2020-11-16T22:05:06.874 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/ 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_battery.ko 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_charger.ko 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq2415x_charger.ko 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24190_charger.ko 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24735-charger.ko 2020-11-16T22:05:06.875 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9030_battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9052-battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2760_battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2780_battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2781_battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2782_battery.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/gpio-charger.ko 2020-11-16T22:05:06.876 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/isp1704_charger.ko 2020-11-16T22:05:06.877 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/lp8727_charger.ko 2020-11-16T22:05:06.877 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max14577_charger.ko 2020-11-16T22:05:07.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17040_battery.ko 2020-11-16T22:05:07.078 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17042_battery.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8903_charger.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8925_power.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pcf50633-charger.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pda_power.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/sbs-battery.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/smb347-charger.ko 2020-11-16T22:05:07.079 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/test_power.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/tps65090-charger.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_backup.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_power.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm8350_power.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/ 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_common.ko 2020-11-16T22:05:07.080 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_msr.ko 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/ 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/ 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-gpio.ko 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-ldisc.ko 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps_parport.ko 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/pps_core.ko 2020-11-16T22:05:07.081 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp.ko 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp_kvm.ko 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/ 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm80x.ko 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm860x.ko 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ab3100.ko 2020-11-16T22:05:07.082 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq32k.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq4802.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9052.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9055.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9063.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1286.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1307.ko 2020-11-16T22:05:07.083 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1374.ko 2020-11-16T22:05:07.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1511.ko 2020-11-16T22:05:07.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1553.ko 2020-11-16T22:05:07.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1672.ko 2020-11-16T22:05:07.084 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1742.ko 2020-11-16T22:05:07.286 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds2404.ko 2020-11-16T22:05:07.286 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds3232.ko 2020-11-16T22:05:07.286 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-em3027.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-fm3130.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl12022.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl1208.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-lp8788.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m41t80.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t35.ko 2020-11-16T22:05:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t59.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t86.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max6900.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8907.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8925.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8997.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8998.ko 2020-11-16T22:05:07.288 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-mc13xxx.ko 2020-11-16T22:05:07.289 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-msm6242.ko 2020-11-16T22:05:07.289 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-palmas.ko 2020-11-16T22:05:07.289 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf2127.ko 2020-11-16T22:05:07.289 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf50633.ko 2020-11-16T22:05:07.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf85063.ko 2020-11-16T22:05:07.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8523.ko 2020-11-16T22:05:07.490 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8563.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8583.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rc5t583.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rp5c01.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rs5c372.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rv3029c2.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8025.ko 2020-11-16T22:05:07.491 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8581.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s35390a.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s5m.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-stk17ta8.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps6586x.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps65910.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps80031.ko 2020-11-16T22:05:07.492 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-v3020.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm831x.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm8350.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-x1205.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-9xxx.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-sas.ko 2020-11-16T22:05:07.493 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-xxxx.ko 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 132001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/BusLogic.ko 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/a100u2w.ko 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/ 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 286449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/aacraid.ko 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/advansys.ko 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/ 2020-11-16T22:05:07.494 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 346097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 297009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/ 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 194297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/aic94xx.ko 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/am53c974.ko 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/ 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 116369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/arcmsr.ko 2020-11-16T22:05:07.495 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/atp870u.ko 2020-11-16T22:05:07.496 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/ 2020-11-16T22:05:07.496 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 265265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2020-11-16T22:05:07.496 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/ 2020-11-16T22:05:07.496 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1164361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/bfa.ko 2020-11-16T22:05:07.698 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/ 2020-11-16T22:05:07.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 252833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2020-11-16T22:05:07.698 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/ 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/bnx2i.ko 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ch.ko 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/ 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 312289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/csiostor.ko 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/ 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/ 2020-11-16T22:05:07.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/ 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 153137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dc395x.ko 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dmx3191d.ko 2020-11-16T22:05:07.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dpt_i2o.ko 2020-11-16T22:05:07.701 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/ 2020-11-16T22:05:07.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 213561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/esas2r.ko 2020-11-16T22:05:07.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esp_scsi.ko 2020-11-16T22:05:07.701 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/ 2020-11-16T22:05:07.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/fcoe.ko 2020-11-16T22:05:07.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 162897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/libfcoe.ko 2020-11-16T22:05:07.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fdomain.ko 2020-11-16T22:05:07.902 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/ 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 280281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/fnic.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 107993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/gdth.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 215137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hpsa.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hptiop.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hv_storvsc.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/imm.ko 2020-11-16T22:05:07.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/initio.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 294193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ipr.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ips.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/ 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 335225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/isci.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_tcp.ko 2020-11-16T22:05:07.904 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/ 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 321337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/libfc.ko 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi.ko 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi_tcp.ko 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/ 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 199625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/libsas.ko 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/ 2020-11-16T22:05:07.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2047225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/lpfc.ko 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/ 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 334937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid.ko 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/ 2020-11-16T22:05:07.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 652873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/ 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 122993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/mvsas.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvumi.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/ 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2020-11-16T22:05:07.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/ 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 420593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/pm80xx.ko 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 137097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pmcraid.ko 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ppa.ko 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla1280.ko 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/ 2020-11-16T22:05:07.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1702009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/ 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 610673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qlogicfas408.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/raid_class.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 238049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_debug.ko 2020-11-16T22:05:07.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_fc.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 265049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_iscsi.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 95017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_sas.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_spi.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_srp.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ses.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 118033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/st.ko 2020-11-16T22:05:07.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/stex.ko 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/ 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 166257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 404305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-core.ko 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pci.ko 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pltfrm.ko 2020-11-16T22:05:07.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/virtio_scsi.ko 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/vmw_pvscsi.ko 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/wd719x.ko 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/xen-scsifront.ko 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ssb.ko 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/ 2020-11-16T22:05:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/ 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 747873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/ 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/tcm_loop.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_file.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_iblock.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 931017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_mod.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_pscsi.ko 2020-11-16T22:05:07.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 108177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_user.ko 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/ 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/tcm_fc.ko 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/ 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/ 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/ 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2020-11-16T22:05:07.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_powerclamp.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2020-11-16T22:05:07.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/cyclades.ko 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ipwireless.ko 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/isicom.ko 2020-11-16T22:05:07.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/moxa.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/mxser.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_gsm.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_hdlc.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracerouter.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracesink.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/nozomi.ko 2020-11-16T22:05:07.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/rocket.ko 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/ 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/ 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/8250_dw.ko 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/serial_cs.ko 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_jtaguart.ko 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_uart.ko 2020-11-16T22:05:07.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/arc_uart.ko 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/fsl_lpuart.ko 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/ 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/jsm.ko 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/rp2.ko 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/sc16is7xx.ko 2020-11-16T22:05:07.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink.ko 2020-11-16T22:05:08.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink_gt.ko 2020-11-16T22:05:08.122 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclinkmp.ko 2020-11-16T22:05:08.122 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/ 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_aec.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_cif.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_dmem_genirq.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_mf624.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_netx.ko 2020-11-16T22:05:08.123 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pci_generic.ko 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pdrv_genirq.ko 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_sercos3.ko 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/ 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/ 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/c67x00.ko 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ 2020-11-16T22:05:08.124 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc.ko 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/ 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-acm.ko 2020-11-16T22:05:08.125 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-wdm.ko 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usblp.ko 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usbtmc.ko 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ulpi.ko 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/ 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/dwc2_pci.ko 2020-11-16T22:05:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/ 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-haps.ko 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-pci.ko 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 432665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3.ko 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/ 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/ 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_ether.ko 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_serial.ko 2020-11-16T22:05:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 92065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2020-11-16T22:05:08.128 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2020-11-16T22:05:08.129 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/ 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ether.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2020-11-16T22:05:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_hid.ko 2020-11-16T22:05:08.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2020-11-16T22:05:08.131 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2020-11-16T22:05:08.334 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2020-11-16T22:05:08.334 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_printer.ko 2020-11-16T22:05:08.334 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_serial.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_zero.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 143505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/libcomposite.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/ 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2020-11-16T22:05:08.335 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/ 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 160057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc_pci.ko 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/fotg210-udc.ko 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/goku_udc.ko 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/gr_udc.ko 2020-11-16T22:05:08.336 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_udc.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2272.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2280.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pch_udc.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2020-11-16T22:05:08.337 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/udc-core.ko 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/bcma-hcd.ko 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 128681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/fotg210-hcd.ko 2020-11-16T22:05:08.338 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/isp116x-hcd.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/oxu210hp-hcd.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/r8a66597-hcd.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811-hcd.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811_cs.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ssb-hcd.ko 2020-11-16T22:05:08.339 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 97993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/u132-hcd.ko 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/xhci-plat-hcd.ko 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/ 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/mdc800.ko 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/microtek.ko 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/adutux.ko 2020-11-16T22:05:08.340 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/appledisplay.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cypress_cy7c63.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cytherm.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ehset.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi26.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi62.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ezusb.ko 2020-11-16T22:05:08.341 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ftdi-elan.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/idmouse.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/iowarrior.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/isight_firmware.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ldusb.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/legousbtower.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/lvstest.ko 2020-11-16T22:05:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/ 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/trancevibrator.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usb3503.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usblcd.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbsevseg.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbtest.ko 2020-11-16T22:05:08.343 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/uss720.ko 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/yurex.ko 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/ 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/usbmon.ko 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/ 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 293313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/musb_hdrc.ko 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/ 2020-11-16T22:05:08.344 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-generic.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-isp1301.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-tahvo.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/aircable.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ark3116.ko 2020-11-16T22:05:08.345 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/belkin_sa.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ch341.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cp210x.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cyberjack.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cypress_m8.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/digi_acceleport.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/empeg.ko 2020-11-16T22:05:08.346 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/f81232.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 174849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ftdi_sio.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/garmin_gps.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 140417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_edgeport.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_ti.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipaq.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipw.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ir-usb.ko 2020-11-16T22:05:08.347 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/iuu_phoenix.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan_pda.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kl5kusb105.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kobil_sct.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mct_u232.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/metro-usb.ko 2020-11-16T22:05:08.348 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7720.ko 2020-11-16T22:05:08.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7840.ko 2020-11-16T22:05:08.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mxuport.ko 2020-11-16T22:05:08.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/navman.ko 2020-11-16T22:05:08.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/omninet.ko 2020-11-16T22:05:08.349 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/opticon.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 176777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/option.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/oti6858.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/pl2303.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcaux.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcserial.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/quatech2.ko 2020-11-16T22:05:08.350 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/safe_serial.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/sierra.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/spcp8x5.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ssu100.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/symbolserial.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb-serial-simple.ko 2020-11-16T22:05:08.351 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_debug.ko 2020-11-16T22:05:08.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_wwan.ko 2020-11-16T22:05:08.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 118233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usbserial.ko 2020-11-16T22:05:08.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/visor.ko 2020-11-16T22:05:08.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/whiteheat.ko 2020-11-16T22:05:08.352 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/wishbone-serial.ko 2020-11-16T22:05:08.554 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/xsens_mt.ko 2020-11-16T22:05:08.554 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ 2020-11-16T22:05:08.554 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/uas.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-alauda.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-cypress.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-datafab.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-eneub6250.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-freecom.ko 2020-11-16T22:05:08.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-isd200.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-jumpshot.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-karma.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-onetouch.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-realtek.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr09.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr55.ko 2020-11-16T22:05:08.556 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-usbat.ko 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 173801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/usb-storage.ko 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/ 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-core.ko 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-host.ko 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 107913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/vhci-hcd.ko 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/ 2020-11-16T22:05:08.557 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/ 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 149025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/vfio-pci.ko 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio.ko 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_iommu_type1.ko 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_virqfd.ko 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/ 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 99057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost.ko 2020-11-16T22:05:08.558 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_iotlb.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_net.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_scsi.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/ 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/ 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/88pm860x_bl.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/aat2870_bl.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp5520_bl.ko 2020-11-16T22:05:08.559 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8860_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8870_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/apple_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/as3711_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/bd6107.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da903x_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da9052_bl.ko 2020-11-16T22:05:08.560 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/gpio_backlight.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/kb3886_bl.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lcd.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3533_bl.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3639_bl.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lv5207lp.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/max8925_bl.ko 2020-11-16T22:05:08.561 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pandora_bl.ko 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pcf50633-backlight.ko 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/platform_lcd.ko 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/wm831x_bl.ko 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/ 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/virtio_mem.ko 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/ 2020-11-16T22:05:08.562 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds1wm.ko 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2482.ko 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2490.ko 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/matrox_w1.ko 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/w1-gpio.ko 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/ 2020-11-16T22:05:08.563 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2406.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2408.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2413.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2423.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2431.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2433.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2780.ko 2020-11-16T22:05:08.564 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2781.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds28e04.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_smem.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_therm.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 99985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/wire.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/acquirewdt.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/advantechwdt.ko 2020-11-16T22:05:08.565 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim1535_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim7101_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/cpu5wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9052_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9055_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9063_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/dw_wdt.ko 2020-11-16T22:05:08.566 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/eurotechwdt.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/f71808e_wdt.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/hpwdt.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/i6300esb.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_vendor_support.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_wdt.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ib700wdt.ko 2020-11-16T22:05:08.567 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ibmasr.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ie6xx_wdt.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it8712f_wdt.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it87_wdt.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/kempld_wdt.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/machzwd.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/mena21_wdt.ko 2020-11-16T22:05:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/menf21bmc_wdt.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/nv_tco.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/of_xilinx_wdt.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pc87413_wdt.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_pci.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_usb.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/retu_wdt.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc60xxwdt.ko 2020-11-16T22:05:08.569 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_epx_c3.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sc1200wdt.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sch311x_wdt.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/smsc37b787_wdt.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/softdog.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sp5100_tco.ko 2020-11-16T22:05:08.570 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/twl4030_wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/via_wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83627hf_wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83877f_wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83977f_wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wafer5823wdt.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wdt_pci.ko 2020-11-16T22:05:08.571 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm831x_wdt.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm8350_wdt.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/xen_wdt.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/ 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-evtchn.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntalloc.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntdev.ko 2020-11-16T22:05:08.572 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/ 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 178577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-privcmd.ko 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-scsiback.ko 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/ 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/xenfs.ko 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ 2020-11-16T22:05:08.573 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/ 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/9p.ko 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/ 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/adfs.ko 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/ 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 162417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/affs.ko 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/ 2020-11-16T22:05:08.574 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 972889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/kafs.ko 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/ 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/autofs4.ko 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/ 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/befs.ko 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/ 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/bfs.ko 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/binfmt_misc.ko 2020-11-16T22:05:08.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/ 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 3192737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/btrfs.ko 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/ 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 194401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/cachefiles.ko 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1125913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ceph.ko 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/ 2020-11-16T22:05:08.576 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2585001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/cifs.ko 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/ 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/coda.ko 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/ 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/configfs.ko 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/ 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/cramfs.ko 2020-11-16T22:05:08.577 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/ 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 438425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/dlm.ko 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/ 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/efs.ko 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/ 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1367417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/f2fs.ko 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/ 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/msdos.ko 2020-11-16T22:05:08.578 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/ 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/freevxfs.ko 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/ 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 317913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/fscache.ko 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/ 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/cuse.ko 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/ 2020-11-16T22:05:08.579 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1193977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/gfs2.ko 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/ 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 141505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/hfs.ko 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/ 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 229553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/hfsplus.ko 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/ 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 193625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/hpfs.ko 2020-11-16T22:05:08.580 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/ 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/isofs.ko 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/ 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 413889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/jfs.ko 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/ 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 268617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/lockd.ko 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/ 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/minix.ko 2020-11-16T22:05:08.581 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/ 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/ 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 87177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/ 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/ 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 123673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2020-11-16T22:05:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 899209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfs.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv2.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv3.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1849921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv4.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/ 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/grace.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_acl.ko 2020-11-16T22:05:08.583 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_ssc.ko 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/ 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1361097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/nfsd.ko 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/ 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 557369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/nilfs2.ko 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/ 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-celtic.ko 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-centeuro.ko 2020-11-16T22:05:08.584 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-croatian.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-cyrillic.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-gaelic.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-greek.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-iceland.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-inuit.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-roman.ko 2020-11-16T22:05:08.585 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-romanian.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-turkish.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_ascii.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1250.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1251.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1255.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp737.ko 2020-11-16T22:05:08.586 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp775.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp850.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp852.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp855.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp857.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp860.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp861.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp862.ko 2020-11-16T22:05:08.587 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp863.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp864.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp865.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp866.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp869.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp874.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp932.ko 2020-11-16T22:05:08.588 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 141825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp936.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 164961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp949.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 117337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp950.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_euc-jp.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-1.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-13.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-14.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-15.ko 2020-11-16T22:05:08.589 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-2.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-3.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-4.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-5.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-6.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-7.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-9.ko 2020-11-16T22:05:08.590 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-r.ko 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-ru.ko 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-u.ko 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_utf8.ko 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 212417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ntfs.ko 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ 2020-11-16T22:05:08.591 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 235897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 435145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2561401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2.ko 2020-11-16T22:05:08.592 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_user.ko 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stackglue.ko 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/ 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/omfs.ko 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/ 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 288641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/overlay.ko 2020-11-16T22:05:08.593 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ramoops.ko 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/ 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/qnx4.ko 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/ 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/qnx6.ko 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/ 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_tree.ko 2020-11-16T22:05:08.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v1.ko 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v2.ko 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/ 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 521233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/reiserfs.ko 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/ 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/romfs.ko 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/ 2020-11-16T22:05:08.595 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 125585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/squashfs.ko 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/ 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/sysv.ko 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/ 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 249097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/udf.ko 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 157929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ufs.ko 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/ 2020-11-16T22:05:08.596 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 3641945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/xfs.ko 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/ 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/ 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/locktorture.ko 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/ 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/test_udelay.ko 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/torture.ko 2020-11-16T22:05:08.597 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc-itu-t.ko 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc64.ko 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc7.ko 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc8.ko 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/ 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libarc4.ko 2020-11-16T22:05:08.598 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libdes.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/interval_tree_test.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/libcrc32c.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lru_cache.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/ 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4_compress.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4hc_compress.ko 2020-11-16T22:05:08.599 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/ 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/cordic.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/memory-notifier-error-inject.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/notifier-error-inject.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/percpu_test.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/pm-notifier-error-inject.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/ 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 141225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/raid6_pq.ko 2020-11-16T22:05:08.600 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/rbtree_test.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/ 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/reed_solomon.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-kstrtox.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-string_helpers.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1714593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_bpf.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_firmware.ko 2020-11-16T22:05:08.601 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 5545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_module.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_user_copy.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_bm.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_fsm.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_kmp.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/ 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/xz_dec_test.ko 2020-11-16T22:05:08.602 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/ 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 259025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/zstd_compress.ko 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/ 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/hwpoison-inject.ko 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/ 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/6lowpan.ko 2020-11-16T22:05:08.603 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_dest.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_fragment.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_hop.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_ipv6.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_mobility.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_routing.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_udp.ko 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/ 2020-11-16T22:05:08.604 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/garp.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/mrp.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/p8022.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/psnap.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/stp.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/ 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/8021q.ko 2020-11-16T22:05:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/ 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 208001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet.ko 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_rdma.ko 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_virtio.ko 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/ 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 647129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/batman-adv.ko 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/ 2020-11-16T22:05:08.606 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/br_netfilter.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 610185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/bridge.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_802_3.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_among.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arp.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arpreply.ko 2020-11-16T22:05:08.607 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_dnat.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip6.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_limit.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_log.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark_m.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_nflog.ko 2020-11-16T22:05:08.608 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_pkttype.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_redirect.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_snat.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_stp.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_vlan.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_broute.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_filter.ko 2020-11-16T22:05:08.609 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_nat.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtables.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nf_log_bridge.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/ 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 836073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/libceph.ko 2020-11-16T22:05:08.610 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/ 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/pktgen.ko 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/ 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 201017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp.ko 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_diag.ko 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv4.ko 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv6.ko 2020-11-16T22:05:08.611 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/ 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 263865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/decnet.ko 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/ 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/dn_rtmsg.ko 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/ 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/dsa_core.ko 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_brcm.ko 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_dsa.ko 2020-11-16T22:05:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_edsa.ko 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_trailer.ko 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/ 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/hsr.ko 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2020-11-16T22:05:08.613 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 220801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154_socket.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ah4.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/esp4.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/fou.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/gre.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/inet_diag.ko 2020-11-16T22:05:08.614 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_gre.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_tunnel.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_vti.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipcomp.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipip.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arp_tables.ko 2020-11-16T22:05:08.615 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arpt_mangle.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arptable_filter.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ip_tables.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ECN.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2020-11-16T22:05:08.616 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ah.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_filter.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_mangle.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_nat.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_raw.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_security.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2020-11-16T22:05:08.617 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_flow_table_ipv4.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_arp.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_ipv4.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2020-11-16T22:05:08.618 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_bic.ko 2020-11-16T22:05:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_dctcp.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_diag.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_highspeed.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_htcp.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_hybla.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_illinois.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_lp.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_scalable.ko 2020-11-16T22:05:08.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_vegas.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_veno.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_westwood.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_yeah.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tunnel4.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_diag.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_tunnel.ko 2020-11-16T22:05:08.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/xfrm4_tunnel.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ah6.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/esp6.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/fou6.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_gre.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_tunnel.ko 2020-11-16T22:05:08.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_udp_tunnel.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_vti.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ipcomp6.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/mip6.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6_tables.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2020-11-16T22:05:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ah.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_frag.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_mh.ko 2020-11-16T22:05:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rt.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_filter.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_nat.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_raw.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_security.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2020-11-16T22:05:08.625 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_flow_table_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_log_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2020-11-16T22:05:08.626 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/sit.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/tunnel6.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/xfrm6_tunnel.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/ 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/af_key.ko 2020-11-16T22:05:08.627 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/ 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_core.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_debugfs.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_eth.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip6.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_netlink.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ppp.ko 2020-11-16T22:05:08.628 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/ 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/lapb.ko 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/ 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc.ko 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 263433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc2.ko 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/ 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 218937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/mac802154.ko 2020-11-16T22:05:08.629 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/ 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/mpls_gso.ko 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 135633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set.ko 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2020-11-16T22:05:08.630 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 103425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 104705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 108489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2020-11-16T22:05:08.631 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_list_set.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 384017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs.ko 2020-11-16T22:05:08.632 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2020-11-16T22:05:08.633 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conncount.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 414313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack.ko 2020-11-16T22:05:08.634 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_amanda.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_broadcast.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_ftp.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 118865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_h323.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_irc.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 105089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netlink.ko 2020-11-16T22:05:08.635 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_pptp.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sane.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sip.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_snmp.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_tftp.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_dup_netdev.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table_inet.ko 2020-11-16T22:05:08.636 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_log_common.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_amanda.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_ftp.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_irc.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_sip.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_tftp.ko 2020-11-16T22:05:08.637 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_synproxy_core.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 492617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_tables.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_acct.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_cttimeout.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_log.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_osf.ko 2020-11-16T22:05:08.638 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_queue.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_chain_nat.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_compat.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_counter.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_ct.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_dup_netdev.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fib.ko 2020-11-16T22:05:08.639 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fwd_netdev.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_hash.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_limit.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_log.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_masq.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_nat.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_numgen.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_osf.ko 2020-11-16T22:05:08.640 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_queue.ko 2020-11-16T22:05:08.641 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_redir.ko 2020-11-16T22:05:08.641 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject.ko 2020-11-16T22:05:08.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject_inet.ko 2020-11-16T22:05:08.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_socket.ko 2020-11-16T22:05:08.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_synproxy.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_tproxy.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/x_tables.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_AUDIT.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CHECKSUM.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CLASSIFY.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CONNSECMARK.ko 2020-11-16T22:05:08.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CT.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_DSCP.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HL.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HMARK.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_IDLETIMER.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LED.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LOG.ko 2020-11-16T22:05:08.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_MASQUERADE.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NETMAP.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFLOG.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFQUEUE.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_RATEEST.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_REDIRECT.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_SECMARK.ko 2020-11-16T22:05:08.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPMSS.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TEE.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TPROXY.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TRACE.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_addrtype.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_bpf.ko 2020-11-16T22:05:08.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cgroup.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cluster.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_comment.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connbytes.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlabel.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlimit.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connmark.ko 2020-11-16T22:05:08.847 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_conntrack.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cpu.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dccp.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_devgroup.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dscp.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ecn.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_esp.ko 2020-11-16T22:05:08.848 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hashlimit.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_helper.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hl.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipcomp.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_iprange.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipvs.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_l2tp.ko 2020-11-16T22:05:08.849 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_length.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_limit.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mac.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mark.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_multiport.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nat.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nfacct.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_osf.ko 2020-11-16T22:05:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_owner.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_physdev.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_pkttype.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_policy.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_quota.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_rateest.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_realm.ko 2020-11-16T22:05:08.851 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_recent.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_sctp.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_set.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_socket.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_state.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_statistic.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_string.ko 2020-11-16T22:05:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpmss.ko 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpudp.ko 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_time.ko 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_u32.ko 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/ 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/netlink_diag.ko 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/ 2020-11-16T22:05:08.853 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/nsh.ko 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/ 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 367929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/openvswitch.ko 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-geneve.ko 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-gre.ko 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-vxlan.ko 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/ 2020-11-16T22:05:08.854 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/af_packet_diag.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/ 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 123433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/phonet.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/pn_pep.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/ 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 239729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 158753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_rdma.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_tcp.ko 2020-11-16T22:05:08.855 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/ 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 698057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/rxrpc.ko 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/ 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_csum.ko 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_gact.ko 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_ipt.ko 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_mirred.ko 2020-11-16T22:05:08.856 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_nat.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_pedit.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_police.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_simple.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_skbedit.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_vlan.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_basic.ko 2020-11-16T22:05:08.857 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_bpf.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_cgroup.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_flow.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_fw.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_route.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp6.ko 2020-11-16T22:05:08.858 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_tcindex.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_u32.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_cmp.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_ipset.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_meta.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_nbyte.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_text.ko 2020-11-16T22:05:08.859 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_u32.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_cbq.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_choke.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_codel.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_drr.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_dsmark.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq_codel.ko 2020-11-16T22:05:08.860 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_gred.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hfsc.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hhf.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_htb.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_ingress.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_mqprio.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_multiq.ko 2020-11-16T22:05:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_netem.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_pie.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_plug.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_prio.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_qfq.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_red.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfb.ko 2020-11-16T22:05:08.862 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfq.ko 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_tbf.ko 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_teql.ko 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/ 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 879937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp.ko 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp_diag.ko 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/ 2020-11-16T22:05:08.863 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/ 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 334985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1415977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/sunrpc.ko 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/ 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 644937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/ 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/diag.ko 2020-11-16T22:05:08.864 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1118753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/tipc.ko 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/ 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/unix_diag.ko 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/ 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock.ko 2020-11-16T22:05:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_diag.ko 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_loopback.ko 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/ 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 126641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/x25.ko 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/ 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_algo.ko 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_ipcomp.ko 2020-11-16T22:05:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_user.ko 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/ 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/ 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/irqbypass.ko 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 614204 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 599988 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias.bin 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7746 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin 2020-11-16T22:05:08.867 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10228 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.bin 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63287 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.modinfo 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 146392 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 235760 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep.bin 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 210 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.devname 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.order 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 574 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.softdep 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 238541 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols 2020-11-16T22:05:08.868 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 289923 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols.bin 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/ 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/ 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/ 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 178 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/changelog.Debian.gz 2020-11-16T22:05:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 692 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/copyright 2020-11-16T22:05:08.870 DEBUG:teuthology.task.kernel:get_image_version: 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:08.871 INFO:teuthology.task.kernel:searching for kernel 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:08.871 INFO:teuthology.orchestra.run.smithi062:> egrep '(submenu|menuentry.*5.10.0-rc2-ceph-gac4f7b07f403).*{' /boot/grub/grub.cfg 2020-11-16T22:05:08.879 INFO:teuthology.orchestra.run.smithi062.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:05:08.880 INFO:teuthology.orchestra.run.smithi062.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:05:08.880 INFO:teuthology.orchestra.run.smithi062.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-recovery-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-16T22:05:08.880 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2020-11-16T22:05:08.881 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:08.881 INFO:teuthology.orchestra.run.smithi062:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2020-11-16T22:05:08.924 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi062.front.sepia.ceph.com to complete... 2020-11-16T22:05:09.509 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub' 2020-11-16T22:05:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2020-11-16T22:05:09.772 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:09.782 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:05:10.079 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T22:05:10.091 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T22:05:11.416 INFO:teuthology.orchestra.run.smithi062.stderr:done 2020-11-16T22:05:42.555 INFO:teuthology.misc:Re-opening connections... 2020-11-16T22:05:42.555 INFO:teuthology.misc:trying to connect to ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:05:42.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:05:42.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:05:42.632 INFO:teuthology.orchestra.run.smithi185:> true 2020-11-16T22:05:42.875 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:05:42.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:05:42.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:01.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2020-11-16T22:06:01.279 DEBUG:teuthology.misc:waited 18.723511219024658 2020-11-16T22:06:02.279 INFO:teuthology.misc:trying to connect to ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:06:02.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:02.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:02.352 INFO:teuthology.orchestra.run.smithi096:> true 2020-11-16T22:06:02.591 DEBUG:teuthology.misc:waited 20.035336017608643 2020-11-16T22:06:03.592 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:06:03.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:03.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:06.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2020-11-16T22:06:06.654 DEBUG:teuthology.misc:waited 24.09915828704834 2020-11-16T22:06:07.656 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:06:07.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:07.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:09.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2020-11-16T22:06:09.726 DEBUG:teuthology.misc:waited 27.171120643615723 2020-11-16T22:06:10.727 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:06:10.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:10.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:12.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2020-11-16T22:06:12.798 DEBUG:teuthology.misc:waited 30.243127822875977 2020-11-16T22:06:13.799 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:06:13.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:14.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:06:15.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2020-11-16T22:06:15.871 DEBUG:teuthology.misc:waited 33.31534147262573 2020-11-16T22:06:16.872 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:06:16.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:06:16.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:07:16.875 DEBUG:teuthology.orchestra.remote:timed out 2020-11-16T22:07:16.878 DEBUG:teuthology.misc:waited 94.32237935066223 2020-11-16T22:07:17.879 INFO:teuthology.misc:trying to connect to ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:07:17.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:07:17.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:07:17.943 INFO:teuthology.orchestra.run.smithi062:> true 2020-11-16T22:07:18.445 DEBUG:teuthology.misc:waited 95.88971710205078 2020-11-16T22:07:19.446 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2020-11-16T22:07:19.446 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.10.0-rc2-ceph-gac4f7b07f403"... 2020-11-16T22:07:19.447 INFO:teuthology.orchestra.run.smithi062:> uname -r 2020-11-16T22:07:19.463 INFO:teuthology.orchestra.run.smithi062.stdout:5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:07:19.464 DEBUG:teuthology.task.kernel:current kernel version is 5.10.0-rc2-ceph-gac4f7b07f403 vs 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-16T22:07:19.464 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-16T22:07:19.464 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-16T22:07:20.466 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-16T22:07:20.474 INFO:teuthology.task.internal:Creating test directory... 2020-11-16T22:07:20.475 INFO:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:07:20.481 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:07:20.483 INFO:teuthology.orchestra.run.smithi185:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T22:07:20.492 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-16T22:07:20.503 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-16T22:07:20.518 INFO:teuthology.task.internal:Creating archive directory... 2020-11-16T22:07:20.518 INFO:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:07:20.538 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:07:20.540 INFO:teuthology.orchestra.run.smithi185:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T22:07:20.653 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-16T22:07:20.661 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-16T22:07:20.662 INFO:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:07:20.667 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:07:20.670 INFO:teuthology.orchestra.run.smithi185:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T22:07:20.691 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.695 INFO:teuthology.orchestra.run.smithi185.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.710 INFO:teuthology.orchestra.run.smithi185.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.711 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.717 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.749 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T22:07:20.753 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-16T22:07:20.761 INFO:teuthology.task.internal:Configuring sudo... 2020-11-16T22:07:20.762 INFO:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:07:20.768 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:07:20.770 INFO:teuthology.orchestra.run.smithi185:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T22:07:20.794 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-16T22:07:20.805 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-16T22:07:20.805 INFO:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:07:20.846 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:07:20.848 INFO:teuthology.orchestra.run.smithi185:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T22:07:20.858 INFO:teuthology.orchestra.run.smithi185:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:07:20.902 INFO:teuthology.orchestra.run.smithi185:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:07:20.950 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:07:20.950 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:20.996 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:21.016 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:07:21.016 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:07:21.016 INFO:teuthology.orchestra.run.smithi185.stderr:152 bytes copied, 3.3261e-05 s, 4.6 MB/s 2020-11-16T22:07:21.017 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:07:21.024 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:07:21.070 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:07:21.070 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:21.116 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:21.141 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:07:21.141 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:07:21.141 INFO:teuthology.orchestra.run.smithi096.stderr:152 bytes copied, 3.1216e-05 s, 4.9 MB/s 2020-11-16T22:07:21.142 INFO:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T22:07:21.159 INFO:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T22:07:21.175 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:07:21.175 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:21.234 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T22:07:21.265 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:07:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:07:21.266 INFO:teuthology.orchestra.run.smithi062.stderr:152 bytes copied, 8.548e-05 s, 1.8 MB/s 2020-11-16T22:07:21.269 INFO:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2020-11-16T22:07:21.274 INFO:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2020-11-16T22:07:21.276 INFO:teuthology.orchestra.run.smithi185:> sudo service rsyslog restart 2020-11-16T22:07:21.364 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-16T22:07:21.445 INFO:teuthology.task.internal:Starting timer... 2020-11-16T22:07:21.445 INFO:teuthology.run_tasks:Running task pcp... 2020-11-16T22:07:21.455 INFO:teuthology.run_tasks:Running task selinux... 2020-11-16T22:07:21.467 DEBUG:teuthology.task.selinux:Excluding smithi185: OS 'ubuntu' does not support SELinux 2020-11-16T22:07:21.468 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2020-11-16T22:07:21.468 DEBUG:teuthology.task.selinux:Excluding smithi062: OS 'ubuntu' does not support SELinux 2020-11-16T22:07:21.468 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-16T22:07:21.468 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-11-16T22:07:21.468 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-16T22:07:21.469 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-16T22:07:21.691 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-11-16T22:07:21.691 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-16T22:07:25.726 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-16T22:07:25.728 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi185.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-16T22:07:26.226 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-16T22:07:26.997 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-16T22:07:29.188 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-16T22:07:29.205 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:29 +0000 (0:00:00.062) 0:00:00.062 ******* 2020-11-16T22:07:29.218 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:29 +0000 (0:00:00.012) 0:00:00.075 ******* 2020-11-16T22:07:29.231 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:29 +0000 (0:00:00.012) 0:00:00.088 ******* 2020-11-16T22:07:30.888 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi185.front.sepia.ceph.com] 2020-11-16T22:07:30.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:30 +0000 (0:00:01.726) 0:00:01.815 ******* 2020-11-16T22:07:30.968 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] 2020-11-16T22:07:31.038 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.080) 0:00:01.895 ******* 2020-11-16T22:07:31.202 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-16T22:07:31.204 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:07:31.238 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.199) 0:00:02.095 ******* 2020-11-16T22:07:31.272 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.034) 0:00:02.129 ******* 2020-11-16T22:07:31.647 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi062.front.sepia.ceph.com] 2020-11-16T22:07:31.694 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-16T22:07:31.696 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} changed: [smithi185.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T22:07:31.732 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.459) 0:00:02.589 ******* 2020-11-16T22:07:31.762 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.030) 0:00:02.619 ******* 2020-11-16T22:07:31.793 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.030) 0:00:02.650 ******* 2020-11-16T22:07:31.913 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-16T22:07:31.915 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:07:31.953 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:31 +0000 (0:00:00.160) 0:00:02.811 ******* 2020-11-16T22:07:32.083 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-16T22:07:32.085 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.072033", "end": "2020-11-16 22:07:32.073321", "rc": 0, "start": "2020-11-16 22:07:32.001288", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-16T22:07:32.118 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.164) 0:00:02.975 ******* 2020-11-16T22:07:32.125 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.082279", "end": "2020-11-16 22:07:32.085187", "rc": 0, "start": "2020-11-16 22:07:32.002908", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-16T22:07:32.159 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.041) 0:00:03.016 ******* 2020-11-16T22:07:32.315 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-16T22:07:32.317 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T22:07:32.355 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.195) 0:00:03.212 ******* 2020-11-16T22:07:32.362 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-16T22:07:32.363 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:07:32.398 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.043) 0:00:03.255 ******* 2020-11-16T22:07:32.429 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.031) 0:00:03.286 ******* 2020-11-16T22:07:32.643 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-16T22:07:32.645 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.104812", "end": "2020-11-16 22:07:32.565232", "rc": 0, "start": "2020-11-16 22:07:32.460420", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-16T22:07:32.682 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.253) 0:00:03.540 ******* 2020-11-16T22:07:32.844 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-16T22:07:32.845 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:07:32.883 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:32 +0000 (0:00:00.200) 0:00:03.740 ******* 2020-11-16T22:07:33.025 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-16T22:07:33.027 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605564452.4117181-236591290070686/source", "state": "file", "uid": 0} changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605564452.444589-88947943854941/source", "state": "file", "uid": 0} 2020-11-16T22:07:33.060 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:33 +0000 (0:00:00.176) 0:00:03.917 ******* 2020-11-16T22:07:33.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:33 +0000 (0:00:00.031) 0:00:03.949 ******* 2020-11-16T22:07:33.367 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-16T22:07:33.368 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-16T22:07:33.571 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-11-16T22:07:33.573 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-11-16T22:07:33.779 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605564452.8959293-117962039176612/source", "state": "file", "uid": 0} Monday 16 November 2020 22:07:33 +0000 (0:00:00.443) 0:00:04.392 ******* TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/ 2020-11-16T22:07:33.781 INFO:teuthology.task.ansible.out:7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-16T22:07:33.982 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts mat 2020-11-16T22:07:33.984 INFO:teuthology.task.ansible.out:ched PLAY [testnodes] *************************************************************** 2020-11-16T22:07:33.986 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:33 +0000 (0:00:00.451) 0:00:04.844 ******* 2020-11-16T22:07:34.016 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:34 +0000 (0:00:00.029) 0:00:04.873 ******* 2020-11-16T22:07:34.047 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:34 +0000 (0:00:00.031) 0:00:04.905 ******* 2020-11-16T22:07:34.793 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-16T22:07:34.795 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] 2020-11-16T22:07:34.871 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-16T22:07:34.873 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:34 +0000 (0:00:00.823) 0:00:05.729 ******* 2020-11-16T22:07:35.228 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-16T22:07:35.229 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.163921", "end": "2020-11-16 22:07:35.205212", "rc": 0, "start": "2020-11-16 22:07:35.041291", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:07:35.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.389) 0:00:06.118 ******* 2020-11-16T22:07:35.403 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T22:07:35.405 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.015079", "end": "2020-11-16 22:07:35.392773", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:07:35.377694", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:07:35.441 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.179) 0:00:06.298 ******* 2020-11-16T22:07:35.468 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:07:35.469 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T22:07:35.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.061) 0:00:06.359 ******* 2020-11-16T22:07:35.528 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:07:35.530 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.563 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.060) 0:00:06.420 ******* 2020-11-16T22:07:35.589 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T22:07:35.591 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.623 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.060) 0:00:06.481 ******* 2020-11-16T22:07:35.650 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T22:07:35.652 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.684 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.060) 0:00:06.541 ******* 2020-11-16T22:07:35.711 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T22:07:35.712 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.059) 0:00:06.601 ******* 2020-11-16T22:07:35.770 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T22:07:35.772 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.774 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi185.front.sepia.ceph.com] 2020-11-16T22:07:35.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.068) 0:00:06.669 ******* 2020-11-16T22:07:35.881 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:35 +0000 (0:00:00.068) 0:00:06.738 ******* 2020-11-16T22:07:35.887 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:07:35.889 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.890 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr 2020-11-16T22:07:35.892 INFO:teuthology.task.ansible.out:+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_l 2020-11-16T22:07:35.893 INFO:teuthology.task.ansible.out:oop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.903 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.905 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:35.915 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:36.123 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23 2020-11-16T22:07:36.125 INFO:teuthology.task.ansible.out:cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansib 2020-11-16T22:07:36.327 INFO:teuthology.task.ansible.out:le_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (ite 2020-11-16T22:07:36.329 INFO:teuthology.task.ansible.out:m={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9 2020-11-16T22:07:36.535 INFO:teuthology.task.ansible.out:aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3 2020-11-16T22:07:36.537 INFO:teuthology.task.ansible.out:489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "s 2020-11-16T22:07:36.743 INFO:teuthology.task.ansible.out:kip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzept 2020-11-16T22:07:36.745 INFO:teuthology.task.ansible.out:znuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:36.946 INFO:teuthology.task.ansible.out: skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3 2020-11-16T22:07:36.948 INFO:teuthology.task.ansible.out:b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0 2020-11-16T22:07:36.949 INFO:teuthology.task.ansible.out:Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn' 2020-11-16T22:07:37.155 INFO:teuthology.task.ansible.out:: 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4U 2020-11-16T22:07:37.157 INFO:teuthology.task.ansible.out:kibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d47 2020-11-16T22:07:37.359 INFO:teuthology.task.ansible.out:29c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) 2020-11-16T22:07:37.361 INFO:teuthology.task.ansible.out:=> {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkoga 2020-11-16T22:07:37.563 INFO:teuthology.task.ansible.out:n@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA d 2020-11-16T22:07:37.565 INFO:teuthology.task.ansible.out:a2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional resu 2020-11-16T22:07:37.771 INFO:teuthology.task.ansible.out:lt was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:37.772 INFO:teuthology.task.ansible.out: 2020-11-16T22:07:37.773 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi185.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.156187", "end": "2020-11-16 22:07:36.191840", "rc": 0, "start": "2020-11-16 22:07:36.035653", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:37.774 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:37.774 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jz 2020-11-16T22:07:37.776 INFO:teuthology.task.ansible.out:f2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": fal 2020-11-16T22:07:37.979 INFO:teuthology.task.ansible.out:se, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields 2020-11-16T22:07:37.981 INFO:teuthology.task.ansible.out: IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f" 2020-11-16T22:07:38.183 INFO:teuthology.task.ansible.out:}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "ch 2020-11-16T22:07:38.185 INFO:teuthology.task.ansible.out:anged": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4ire 2020-11-16T22:07:38.387 INFO:teuthology.task.ansible.out:sis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af 2020-11-16T22:07:38.389 INFO:teuthology.task.ansible.out:66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was Fa 2020-11-16T22:07:38.591 INFO:teuthology.task.ansible.out:lse"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn 2020-11-16T22:07:38.593 INFO:teuthology.task.ansible.out:": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.co 2020-11-16T22:07:38.799 INFO:teuthology.task.ansible.out:m PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:38.800 INFO:teuthology.task.ansible.out: 2020-11-16T22:07:38.800 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:38.805 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi062.front.sepia.ceph.com] 2020-11-16T22:07:38.814 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:07:38.815 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:39.018 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (i 2020-11-16T22:07:39.020 INFO:teuthology.task.ansible.out:tem={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacB 2020-11-16T22:07:39.021 INFO:teuthology.task.ansible.out:ook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 2020-11-16T22:07:39.227 INFO:teuthology.task.ansible.out:'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5 2020-11-16T22:07:39.229 INFO:teuthology.task.ansible.out:Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:38 +0000 (0:00:03.016) 0:00:09.754 ******* Monday 16 November 2020 22:07:38 +0000 (0:00:00.037) 0:00:09.791 ******* 2020-11-16T22:07:42.992 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-16T22:07:43.040 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:43 +0000 (0:00:04.105) 0:00:13.897 ******* 2020-11-16T22:07:43.072 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:43 +0000 (0:00:00.031) 0:00:13.929 ******* 2020-11-16T22:07:43.099 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-16T22:07:43.101 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:43.141 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:43 +0000 (0:00:00.068) 0:00:13.998 ******* 2020-11-16T22:07:47.244 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T22:07:47.246 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.088534", "end": "2020-11-16 22:07:43.196614", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:07:43.108080", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:07:47.282 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:04.140) 0:00:18.139 ******* 2020-11-16T22:07:47.289 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-16T22:07:47.290 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.041) 0:00:18.181 ******* 2020-11-16T22:07:47.332 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T22:07:47.334 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T22:07:47.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.042) 0:00:18.223 ******* 2020-11-16T22:07:47.372 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-16T22:07:47.374 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.408 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.041) 0:00:18.265 ******* 2020-11-16T22:07:47.414 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:07:47.415 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.416 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi062.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:04.198326", "end": "2020-11-16 22:07:47.392120", "rc": 0, "start": "2020-11-16 22:07:43.193794", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T22:07:47.455 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.047) 0:00:18.312 ******* 2020-11-16T22:07:47.486 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.031) 0:00:18.343 ******* 2020-11-16T22:07:47.494 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T22:07:47.495 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.528 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.042) 0:00:18.386 ******* 2020-11-16T22:07:47.536 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T22:07:47.538 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.570 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.041) 0:00:18.427 ******* 2020-11-16T22:07:47.577 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-16T22:07:47.578 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.612 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.042) 0:00:18.470 ******* 2020-11-16T22:07:47.619 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T22:07:47.620 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.655 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.042) 0:00:18.512 ******* 2020-11-16T22:07:47.663 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:07:47.665 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.665 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.018623", "end": "2020-11-16 22:07:47.583080", "failed_when_result": false, "rc": 0, "start": "2020-11-16 22:07:47.564457", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.666 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.667 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.707 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.051) 0:00:18.564 ******* 2020-11-16T22:07:47.714 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.716 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.717 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.751 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.044) 0:00:18.608 ******* 2020-11-16T22:07:47.758 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.760 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.760 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.761 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T22:07:47.762 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.763 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.061) 0:00:18.669 ******* 2020-11-16T22:07:47.821 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.863 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_v 2020-11-16T22:07:47.864 INFO:teuthology.task.ansible.out:ar": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:47 +0000 (0:00:00.049) 0:00:18.719 ******* 2020-11-16T22:07:47.869 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T22:07:47.869 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.871 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.871 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.871 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.872 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.872 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.872 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.873 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.050) 0:00:18.770 ******* 2020-11-16T22:07:47.920 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.925 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T22:07:47.927 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.927 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ 2020-11-16T22:07:47.929 INFO:teuthology.task.ansible.out:ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcN 2020-11-16T22:07:47.930 INFO:teuthology.task.ansible.out:E/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.962 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:47 +0000 (0:00:00.049) 0:00:18.820 ******* 2020-11-16T22:07:47.970 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.972 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:47.972 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYl 2020-11-16T22:07:47.974 INFO:teuthology.task.ansible.out:hLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreD 2020-11-16T22:07:47.976 INFO:teuthology.task.ansible.out:KF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item" 2020-11-16T22:07:47.977 INFO:teuthology.task.ansible.out:, "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed14 2020-11-16T22:07:47.979 INFO:teuthology.task.ansible.out:9cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.016 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:48 +0000 (0:00:00.053) 0:00:18.873 ******* 2020-11-16T22:07:48.024 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.025 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.026 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f 2020-11-16T22:07:48.027 INFO:teuthology.task.ansible.out:0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.028 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6r 2020-11-16T22:07:48.029 INFO:teuthology.task.ansible.out:xqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d34 2020-11-16T22:07:48.031 INFO:teuthology.task.ansible.out:89b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.040 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.042 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.080 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:48 +0000 (0:00:00.064) 0:00:18.938 ******* 2020-11-16T22:07:48.088 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.090 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.092 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d 2020-11-16T22:07:48.093 INFO:teuthology.task.ansible.out:2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.094 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"a 2020-11-16T22:07:48.095 INFO:teuthology.task.ansible.out:nsible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.105 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.107 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.158 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:48 +0000 (0:00:00.077) 0:00:19.016 ******* 2020-11-16T22:07:48.167 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.169 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.169 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.171 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.171 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.172 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "cha 2020-11-16T22:07:48.173 INFO:teuthology.task.ansible.out:nged": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b54 2020-11-16T22:07:48.175 INFO:teuthology.task.ansible.out:94847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.185 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.186 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.225 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:48 +0000 (0:00:00.066) 0:00:19.083 ******* 2020-11-16T22:07:48.233 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.235 INFO:teuthology.task.ansible.out: skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.236 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.237 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.238 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.240 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.240 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.242 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.253 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.254 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.256 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.267 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.269 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.279 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.281 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.291 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.293 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.293 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "ite 2020-11-16T22:07:48.295 INFO:teuthology.task.ansible.out:m": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.305 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.306 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.307 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": f 2020-11-16T22:07:48.308 INFO:teuthology.task.ansible.out:alse, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.318 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.318 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.319 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.320 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.331 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.332 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"a 2020-11-16T22:07:48.334 INFO:teuthology.task.ansible.out:nsible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.344 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.344 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.346 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.346 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a 2020-11-16T22:07:48.348 INFO:teuthology.task.ansible.out:27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.358 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.359 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.360 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": 2020-11-16T22:07:48.361 INFO:teuthology.task.ansible.out:false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.370 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.372 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.383 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.383 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.384 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.385 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.396 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.398 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.409 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.410 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.411 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover 2020-11-16T22:07:48.412 INFO:teuthology.task.ansible.out:", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.422 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.424 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.435 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.436 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.437 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.438 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.438 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.439 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a 2020-11-16T22:07:48.441 INFO:teuthology.task.ansible.out:6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.450 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.451 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.462 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.463 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.464 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn" 2020-11-16T22:07:48.465 INFO:teuthology.task.ansible.out:: "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.475 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.476 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.487 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.489 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.489 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.490 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9 2020-11-16T22:07:48.491 INFO:teuthology.task.ansible.out:sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.502 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.503 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.514 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.514 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.516 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:07:48.516 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-16T22:07:48.518 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.722 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:48 +0000 (0:00:00.335) 0:00:19.419 ******* skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5 2020-11-16T22:07:48.724 INFO:teuthology.task.ansible.out:g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmz 2020-11-16T22:07:48.725 INFO:teuthology.task.ansible.out:sOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd 2020-11-16T22:07:48.727 INFO:teuthology.task.ansible.out:1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd39 2020-11-16T22:07:48.728 INFO:teuthology.task.ansible.out:0302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was F 2020-11-16T22:07:48.931 INFO:teuthology.task.ansible.out:alse"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'pc 2020-11-16T22:07:48.933 INFO:teuthology.task.ansible.out:uzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca 2020-11-16T22:07:48.934 INFO:teuthology.task.ansible.out:6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi096.front.sepia.ceph.com] => { "msg": "Host smithi096.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da 2020-11-16T22:07:48.936 INFO:teuthology.task.ansible.out:2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:48 +0000 (0:00:00.085) 0:00:19.504 ******* Monday 16 November 2020 22:07:48 +0000 (0:00:00.033) 0:00:19.538 ******* skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.937 INFO:teuthology.task.ansible.out: 2020-11-16T22:07:48.938 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.938 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.939 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "b 2020-11-16T22:07:48.940 INFO:teuthology.task.ansible.out:england@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.941 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:48.942 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home 2020-11-16T22:07:48.943 INFO:teuthology.task.ansible.out:/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.146 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb8 2020-11-16T22:07:49.148 INFO:teuthology.task.ansible.out:88\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} TASK [users : Create all users without sudo access.] *************************** TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": 2020-11-16T22:07:49.149 INFO:teuthology.task.ansible.out:"Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622 2020-11-16T22:07:49.151 INFO:teuthology.task.ansible.out:f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@mileston 2020-11-16T22:07:49.152 INFO:teuthology.task.ansible.out:es utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc0 2020-11-16T22:07:49.154 INFO:teuthology.task.ansible.out:01c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062. 2020-11-16T22:07:49.155 INFO:teuthology.task.ansible.out:front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@m 2020-11-16T22:07:49.157 INFO:teuthology.task.ansible.out:acprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4 2020-11-16T22:07:49.158 INFO:teuthology.task.ansible.out:f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.159 INFO:teuthology.task.ansible.out: 2020-11-16T22:07:49.160 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.160 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.161 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"na 2020-11-16T22:07:49.162 INFO:teuthology.task.ansible.out:me": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [sm 2020-11-16T22:07:49.163 INFO:teuthology.task.ansible.out:ithi062.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorind 2020-11-16T22:07:49.165 INFO:teuthology.task.ansible.out:a", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.536) 0:00:20.074 ******* 2020-11-16T22:07:49.249 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.031) 0:00:20.106 ******* 2020-11-16T22:07:49.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.030) 0:00:20.140 ******* 2020-11-16T22:07:49.290 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-16T22:07:49.291 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T22:07:49.291 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.047) 0:00:20.188 ******* 2020-11-16T22:07:49.362 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.031) 0:00:20.219 ******* 2020-11-16T22:07:49.369 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-16T22:07:49.371 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.416 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.053) 0:00:20.273 ******* 2020-11-16T22:07:49.452 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.036) 0:00:20.310 ******* 2020-11-16T22:07:49.460 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:07:49.461 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Clone the keys repo] ********************************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.049) 0:00:20.359 ******* 2020-11-16T22:07:49.535 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.032) 0:00:20.392 ******* 2020-11-16T22:07:49.542 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.544 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.583 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.048) 0:00:20.440 ******* 2020-11-16T22:07:49.617 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.033) 0:00:20.474 ******* 2020-11-16T22:07:49.623 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T22:07:49.625 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.051) 0:00:20.525 ******* 2020-11-16T22:07:49.701 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.032) 0:00:20.558 ******* 2020-11-16T22:07:49.903 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:49 +0000 (0:00:00.047) 0:00:20.606 ******* Monday 16 November 2020 22:07:49 +0000 (0:00:00.031) 0:00:20.638 ******* TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:49 +0000 (0:00:00.044) 0:00:20.682 ******* skipping: [smithi062.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:49.905 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:49 +0000 (0:00:00.056) 0:00:20.738 ******* skipping: [smithi062.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 2020-11-16T22:07:50.107 INFO:teuthology.task.ansible.out:6)] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:50.109 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:49 +0000 (0:00:00.057) 0:00:20.796 ******* TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:50.311 INFO:teuthology.task.ansible.out: skipping: [smithi062.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:49 +0000 (0:00:00.047) 0:00:20.843 ******* skipping: [smithi062.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] = 2020-11-16T22:07:50.313 INFO:teuthology.task.ansible.out:> (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:50 +0000 (0:00:00.049) 0:00:20.893 ******* skipping: [smithi062.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:50 +0000 (0:00:00.053) 0:00:20.946 ******* skipping: [smithi185.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "ite 2020-11-16T22:07:50.515 INFO:teuthology.task.ansible.out:m": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi185.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => 2020-11-16T22:07:50.517 INFO:teuthology.task.ansible.out: (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:50 +0000 (0:00:00.048) 0:00:20.994 ******* skipping: [smithi185.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping 2020-11-16T22:07:50.719 INFO:teuthology.task.ansible.out:: [smithi185.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:50 +0000 (0:00:00.058) 0:00:21.052 ******* 2020-11-16T22:07:54.258 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T22:07:54.260 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.261 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.261 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.262 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.263 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.263 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.466 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:54 +0000 (0:00:04.105) 0:00:25.158 ******* TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi096.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:07:54 +0000 (0:00:00.044) 2020-11-16T22:07:54.468 INFO:teuthology.task.ansible.out: 0:00:25.202 ******* TASK [common : Set have_entitlements] ****************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:54 +0000 (0:00:00.061) 0:00:25.264 ******* TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:54 +0000 (0:00:00.059) 0:00:25.324 ******* 2020-11-16T22:07:54.494 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:54.526 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.059) 0:00:25.384 ******* 2020-11-16T22:07:54.542 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:07:54.544 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-16T22:07:54.586 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.059) 0:00:25.443 ******* 2020-11-16T22:07:54.592 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:07:54.594 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.628 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.041) 0:00:25.485 ******* 2020-11-16T22:07:54.634 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T22:07:54.636 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} TASK [common : Log the OS name, version and release] *************************** ok: [smithi062.front.sepia.ceph.com] => { "msg": "Host smithi062.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-16T22:07:54.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.051) 0:00:25.537 ******* Monday 16 November 2020 22:07:54 +0000 (0:00:00.033) 0:00:25.570 ******* TASK [common : Register with subscription-manager.] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Including major version specific variables.] ******************** ok: [smithi062.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} Monday 16 November 2020 22:07:54 +0000 (0:00:00.047) 0:00:25.618 ******* Monday 16 November 2020 22:07:54 +0000 (0:00:00.030) 0:00:25.649 ******* TASK [common : Log the OS name, version and release] *************************** ok: [smithi185.front.sepia.ceph.com] => { "msg": "Host smithi185.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.840 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.047) 0:00:25.696 ******* 2020-11-16T22:07:54.871 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.031) 0:00:25.728 ******* 2020-11-16T22:07:54.876 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T22:07:54.878 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.919 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.048) 0:00:25.776 ******* 2020-11-16T22:07:54.950 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.031) 0:00:25.808 ******* 2020-11-16T22:07:54.957 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T22:07:54.958 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:54.997 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:54 +0000 (0:00:00.047) 0:00:25.855 ******* 2020-11-16T22:07:55.033 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.035) 0:00:25.890 ******* 2020-11-16T22:07:55.040 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T22:07:55.042 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.246 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.148) 0:00:26.039 ******* Monday 16 November 2020 22:07:55 +0000 (0:00:00.031) 0:00:26.070 ******* TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.259 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.046) 0:00:26.116 ******* 2020-11-16T22:07:55.296 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.036) 0:00:26.153 ******* 2020-11-16T22:07:55.301 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T22:07:55.303 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.050) 0:00:26.204 ******* 2020-11-16T22:07:55.377 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.030) 0:00:26.234 ******* 2020-11-16T22:07:55.383 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T22:07:55.385 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.427 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.050) 0:00:26.285 ******* 2020-11-16T22:07:55.458 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.030) 0:00:26.316 ******* 2020-11-16T22:07:55.464 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T22:07:55.466 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.505 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.046) 0:00:26.362 ******* 2020-11-16T22:07:55.535 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.030) 0:00:26.393 ******* 2020-11-16T22:07:55.541 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T22:07:55.542 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi062.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:55.580 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.045) 0:00:26.438 ******* 2020-11-16T22:07:55.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.033) 0:00:26.471 ******* 2020-11-16T22:07:55.620 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T22:07:55.622 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set have_entitlements] ****************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.045) 0:00:26.517 ******* 2020-11-16T22:07:55.691 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.031) 0:00:26.548 ******* 2020-11-16T22:07:55.699 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T22:07:55.701 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.738 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.047) 0:00:26.596 ******* 2020-11-16T22:07:55.769 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.030) 0:00:26.627 ******* 2020-11-16T22:07:55.779 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:55.817 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.047) 0:00:26.674 ******* 2020-11-16T22:07:55.848 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.030) 0:00:26.705 ******* 2020-11-16T22:07:55.854 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T22:07:55.856 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:55.894 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.046) 0:00:26.751 ******* 2020-11-16T22:07:55.925 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.031) 0:00:26.783 ******* 2020-11-16T22:07:55.933 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T22:07:55.934 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Register with subscription-manager.] **************************** skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:55.973 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:55 +0000 (0:00:00.047) 0:00:26.831 ******* 2020-11-16T22:07:56.005 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.031) 0:00:26.862 ******* 2020-11-16T22:07:56.011 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-16T22:07:56.013 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.047) 0:00:26.910 ******* 2020-11-16T22:07:56.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.037) 0:00:26.947 ******* 2020-11-16T22:07:56.095 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-16T22:07:56.097 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : List CDN releases available to system] ************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.046) 0:00:26.994 ******* 2020-11-16T22:07:56.168 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.031) 0:00:27.025 ******* 2020-11-16T22:07:56.177 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T22:07:56.178 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.218 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.049) 0:00:27.075 ******* 2020-11-16T22:07:56.250 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.032) 0:00:27.107 ******* 2020-11-16T22:07:56.258 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-16T22:07:56.260 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.299 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.049) 0:00:27.157 ******* 2020-11-16T22:07:56.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.031) 0:00:27.188 ******* 2020-11-16T22:07:56.338 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T22:07:56.339 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:56.379 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.047) 0:00:27.236 ******* 2020-11-16T22:07:56.411 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.032) 0:00:27.269 ******* 2020-11-16T22:07:56.418 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-16T22:07:56.420 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.460 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.048) 0:00:27.317 ******* 2020-11-16T22:07:56.493 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.033) 0:00:27.350 ******* 2020-11-16T22:07:56.500 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-16T22:07:56.502 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.542 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.048) 0:00:27.399 ******* 2020-11-16T22:07:56.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.039) 0:00:27.439 ******* 2020-11-16T22:07:56.589 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T22:07:56.590 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.629 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.048) 0:00:27.487 ******* 2020-11-16T22:07:56.662 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.032) 0:00:27.519 ******* 2020-11-16T22:07:56.670 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T22:07:56.672 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.048) 0:00:27.568 ******* 2020-11-16T22:07:56.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.030) 0:00:27.599 ******* 2020-11-16T22:07:56.748 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-16T22:07:56.749 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.788 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.046) 0:00:27.646 ******* 2020-11-16T22:07:56.819 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.030) 0:00:27.676 ******* 2020-11-16T22:07:56.826 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-16T22:07:56.827 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:56.866 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.046) 0:00:27.723 ******* 2020-11-16T22:07:56.898 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.031) 0:00:27.755 ******* 2020-11-16T22:07:56.904 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-16T22:07:56.906 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:07:56.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.046) 0:00:27.801 ******* 2020-11-16T22:07:56.974 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:56 +0000 (0:00:00.029) 0:00:27.831 ******* 2020-11-16T22:07:56.981 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-16T22:07:56.982 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.047) 0:00:27.878 ******* 2020-11-16T22:07:57.058 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.036) 0:00:27.915 ******* 2020-11-16T22:07:57.064 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-16T22:07:57.066 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.105 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.047) 0:00:27.962 ******* 2020-11-16T22:07:57.137 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.031) 0:00:27.994 ******* 2020-11-16T22:07:57.144 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T22:07:57.145 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.347 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.046) 0:00:28.041 ******* Monday 16 November 2020 22:07:57 +0000 (0:00:00.031) 0:00:28.072 ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.048) 0:00:28.120 ******* Monday 16 November 2020 22:07:57 +0000 (0:00:00.031) 0:00:28.152 ******* TASK [common : Register w 2020-11-16T22:07:57.349 INFO:teuthology.task.ansible.out:ith subscription-manager.] **************************** skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi062.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.052) 0:00:28.205 ******* 2020-11-16T22:07:57.379 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.031) 0:00:28.237 ******* 2020-11-16T22:07:57.386 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:07:57.388 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.590 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.049) 0:00:28.286 ******* Monday 16 November 2020 22:07:57 +0000 (0:00:00.030) 0:00:28.317 ******* TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.048) 0:00:28.365 ******* TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi096.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirr 2020-11-16T22:07:57.592 INFO:teuthology.task.ansible.out:orlist", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.041) 0:00:28.407 ******* TASK [common : List CDN releases available to system] ************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.041) 0:00:28.448 ******* 2020-11-16T22:07:57.597 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:07:57.599 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.631 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.040) 0:00:28.489 ******* 2020-11-16T22:07:57.637 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T22:07:57.640 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get the current timezone] *************************************** ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.022971", "end": "2020-11-16 22:07:57.569700", "rc": 0, "start": "2020-11-16 22:07:57.546729", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:07:57.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.047) 0:00:28.536 ******* 2020-11-16T22:07:57.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.032) 0:00:28.568 ******* 2020-11-16T22:07:57.717 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:07:57.719 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.757 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.045) 0:00:28.614 ******* 2020-11-16T22:07:57.763 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T22:07:57.765 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:57.798 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:57 +0000 (0:00:00.040) 0:00:28.655 ******* 2020-11-16T22:07:57.804 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:07:58.007 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014385", "end": "2020-11-16 22:07:57.766093", "rc": 0, "start": "2020-11-16 22:07:57.751708", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.057) 0:00:28.713 ******* Monday 16 November 2020 22:07:57 +0000 (0:00:00.031) 0:00:28.744 ******* TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get the current timezone] *************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:57 +0000 (0:00:00.047) 0:00:28.792 ******* Monday 16 November 2020 22:07:57 +0000 (0:00:00.040) 0:00:28.832 ******* TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.024 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.049) 0:00:28.881 ******* 2020-11-16T22:07:58.055 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.030) 0:00:28.912 ******* 2020-11-16T22:07:58.062 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T22:07:58.064 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.048) 0:00:28.961 ******* 2020-11-16T22:07:58.135 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.031) 0:00:28.992 ******* 2020-11-16T22:07:58.141 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:07:58.143 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.182 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.047) 0:00:29.039 ******* 2020-11-16T22:07:58.213 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.030) 0:00:29.070 ******* 2020-11-16T22:07:58.219 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T22:07:58.220 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.422 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.045) 0:00:29.116 ******* Monday 16 November 2020 22:07:58 +0000 (0:00:00.031) 0:00:29.147 ******* TASK [common : Get the current timezone] *************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:58 +0000 (0:00:00.042) 0:00:29.190 ******* TASK [common : Store list of enabled repos] ************************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:58 +0000 (0:00:00.041) 0:00:29.231 ******* TASK [common : Set the timezone] *********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:58 +0000 (0:00:00.039) 0:00:29.271 ******* TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.453 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.039) 0:00:29.310 ******* 2020-11-16T22:07:58.460 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T22:07:58.461 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.494 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.041) 0:00:29.351 ******* 2020-11-16T22:07:58.501 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-16T22:07:58.502 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.536 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.042) 0:00:29.394 ******* 2020-11-16T22:07:58.542 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T22:07:58.544 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.044) 0:00:29.438 ******* 2020-11-16T22:07:58.588 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T22:07:58.589 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.126107", "end": "2020-11-16 22:07:58.501108", "rc": 0, "start": "2020-11-16 22:07:58.375001", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:07:58.629 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.047) 0:00:29.486 ******* 2020-11-16T22:07:58.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.031) 0:00:29.517 ******* 2020-11-16T22:07:58.668 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:07:58.670 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.874 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.047) 0:00:29.565 ******* Monday 16 November 2020 22:07:58 +0000 (0:00:00.030) 0:00:29.596 ******* TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:07:58 +0000 (0:00:00.048) 0:00:29.644 ******* TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:58 +0000 (0:00:00.060) 0:00:29.705 ******* 2020-11-16T22:07:58.875 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.907 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.059) 0:00:29.764 ******* 2020-11-16T22:07:58.934 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-16T22:07:58.935 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:58.966 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:58 +0000 (0:00:00.058) 0:00:29.823 ******* 2020-11-16T22:07:58.993 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:07:58.995 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.104958", "end": "2020-11-16 22:07:58.930388", "rc": 0, "start": "2020-11-16 22:07:58.825430", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:07:59.032 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.066) 0:00:29.890 ******* 2020-11-16T22:07:59.039 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T22:07:59.041 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.242 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.043) 0:00:29.933 ******* TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi185.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:59 +0000 (0:00:00.071) 0:00:30.005 ******* TASK [common : Clean yum cache] ************************************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 N 2020-11-16T22:07:59.244 INFO:teuthology.task.ansible.out:ovember 2020 22:07:59 +0000 (0:00:00.064) 0:00:30.070 ******* 2020-11-16T22:07:59.419 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:07:59.421 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.065421", "end": "2020-11-16 22:07:59.401683", "rc": 0, "start": "2020-11-16 22:07:59.336262", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T22:07:59.464 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.251) 0:00:30.322 ******* 2020-11-16T22:07:59.491 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T22:07:59.493 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.525 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.060) 0:00:30.382 ******* 2020-11-16T22:07:59.532 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:07:59.534 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.642167", "end": "2020-11-16 22:07:59.500390", "rc": 0, "start": "2020-11-16 22:07:58.858223", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:07:59.571 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.046) 0:00:30.429 ******* 2020-11-16T22:07:59.578 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T22:07:59.579 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.611 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.040) 0:00:30.469 ******* 2020-11-16T22:07:59.617 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:07:59.618 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.039) 0:00:30.509 ******* 2020-11-16T22:07:59.660 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T22:07:59.662 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.694 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.043) 0:00:30.552 ******* 2020-11-16T22:07:59.899 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:59 +0000 (0:00:00.071) 0:00:30.624 ******* TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:07:59 +0000 (0:00:00.065) 0:00:30.689 ******* TASK [common : Clean apt cache] ************************************************ changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.054896", "end": "2020-11-16 22:07:59.829883", "rc": 0, "start": "2020-11-16 22:07:59.774987", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Monday 16 November 2020 22:07:59 +0000 (0:00:00.042) 0:00:30.732 ******* TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.916 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.041) 0:00:30.774 ******* 2020-11-16T22:07:59.943 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T22:07:59.945 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:07:59.980 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:07:59 +0000 (0:00:00.063) 0:00:30.838 ******* 2020-11-16T22:08:00.506 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T22:08:00.507 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.406783", "end": "2020-11-16 22:08:00.500409", "rc": 0, "start": "2020-11-16 22:08:00.093626", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:00.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:00 +0000 (0:00:00.564) 0:00:31.402 ******* 2020-11-16T22:08:00.572 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T22:08:00.573 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:00.605 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:00 +0000 (0:00:00.060) 0:00:31.462 ******* 2020-11-16T22:08:00.800 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T22:08:00.802 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.066906", "end": "2020-11-16 22:08:00.787026", "rc": 0, "start": "2020-11-16 22:08:00.720120", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:00.836 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:00 +0000 (0:00:00.230) 0:00:31.693 ******* 2020-11-16T22:08:06.011 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:08:06.013 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605564484, "cache_updated": true, "changed": true} 2020-11-16T22:08:06.045 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:06 +0000 (0:00:05.209) 0:00:36.902 ******* 2020-11-16T22:08:06.781 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605564484, "cache_updated": true, "changed": true} 2020-11-16T22:08:06.816 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:06 +0000 (0:00:00.770) 0:00:37.673 ******* 2020-11-16T22:08:06.926 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:08:06.928 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:06.961 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:06 +0000 (0:00:00.145) 0:00:37.818 ******* 2020-11-16T22:08:06.987 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:08:06.989 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:07.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:07 +0000 (0:00:00.060) 0:00:37.878 ******* 2020-11-16T22:08:07.333 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:08:07.335 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:08:07.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:07 +0000 (0:00:00.345) 0:00:38.224 ******* 2020-11-16T22:08:07.657 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T22:08:07.658 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:07.691 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:07 +0000 (0:00:00.325) 0:00:38.549 ******* 2020-11-16T22:08:07.718 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:08:07.720 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:07.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:07 +0000 (0:00:00.061) 0:00:38.610 ******* 2020-11-16T22:08:07.863 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T22:08:07.864 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605564485, "cache_updated": true, "changed": true} 2020-11-16T22:08:07.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:07 +0000 (0:00:00.149) 0:00:38.760 ******* 2020-11-16T22:08:08.066 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T22:08:08.068 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:08:08.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:08 +0000 (0:00:00.199) 0:00:38.960 ******* 2020-11-16T22:08:08.192 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:08:08.194 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:08.227 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:08 +0000 (0:00:00.124) 0:00:39.085 ******* 2020-11-16T22:08:08.545 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:08.547 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:08:08.807 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:08:08.953 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:08:08.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:08 +0000 (0:00:00.766) 0:00:39.851 ******* 2020-11-16T22:08:09.000 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T22:08:09.002 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:09.039 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.045) 0:00:39.897 ******* 2020-11-16T22:08:09.045 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T22:08:09.047 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:09.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.047) 0:00:39.944 ******* 2020-11-16T22:08:09.094 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:09.096 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:08:09.133 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.046) 0:00:39.990 ******* 2020-11-16T22:08:09.253 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:08:09.255 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:08:09.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.154) 0:00:40.145 ******* 2020-11-16T22:08:09.367 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:09.368 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:08:09.570 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T22:08:09.589 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.301) 0:00:40.447 ******* 2020-11-16T22:08:09.597 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:09.599 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:08:09.630 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:09.632 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:08:09.840 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:09.841 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:08:09.883 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:09.885 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:08:09.919 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:09 +0000 (0:00:00.329) 0:00:40.776 ******* 2020-11-16T22:08:10.039 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:08:10.041 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:08:10.074 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.154) 0:00:40.931 ******* 2020-11-16T22:08:10.100 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:10.102 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:08:10.137 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.063) 0:00:40.994 ******* 2020-11-16T22:08:10.164 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:08:10.166 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:08:10.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.060) 0:00:41.055 ******* 2020-11-16T22:08:10.224 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:08:10.226 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:10.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.060) 0:00:41.115 ******* 2020-11-16T22:08:10.285 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:08:10.287 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:08:10.322 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.064) 0:00:41.180 ******* 2020-11-16T22:08:10.359 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:08:10.361 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi185.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:08:10.361 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:10.372 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:10.374 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:10.578 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.084) 0:00:41.264 ******* TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:10 +0000 (0:00:00.060) 0:00:41.324 ******* 2020-11-16T22:08:10.629 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:10.630 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:08:10.670 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.202) 0:00:41.527 ******* 2020-11-16T22:08:10.885 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:10.887 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:08:10.922 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.251) 0:00:41.779 ******* 2020-11-16T22:08:10.949 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:08:10.951 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:08:10.982 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:10 +0000 (0:00:00.060) 0:00:41.840 ******* 2020-11-16T22:08:11.009 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:08:11.011 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:11.042 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.060) 0:00:41.900 ******* 2020-11-16T22:08:11.069 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T22:08:11.070 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:08:11.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.059) 0:00:41.959 ******* 2020-11-16T22:08:11.138 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T22:08:11.140 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:11.150 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:11.161 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:11.162 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:11.162 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T22:08:11.195 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.093) 0:00:42.053 ******* 2020-11-16T22:08:11.223 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T22:08:11.225 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:11.256 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.060) 0:00:42.114 ******* 2020-11-16T22:08:11.366 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:08:11.368 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:11.401 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.144) 0:00:42.258 ******* 2020-11-16T22:08:11.607 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:11.609 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T22:08:11.701 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:08:11.703 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T22:08:11.739 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:11 +0000 (0:00:00.338) 0:00:42.596 ******* 2020-11-16T22:08:12.025 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T22:08:12.027 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T22:08:12.071 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.331) 0:00:42.928 ******* 2020-11-16T22:08:12.079 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:08:12.080 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:08:12.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.045) 0:00:42.973 ******* 2020-11-16T22:08:12.133 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:08:12.135 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:12.169 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.052) 0:00:43.026 ******* 2020-11-16T22:08:12.238 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T22:08:12.240 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:08:12.279 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.109) 0:00:43.136 ******* 2020-11-16T22:08:12.287 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:08:12.289 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:08:12.361 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:08:12.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.117) 0:00:43.254 ******* 2020-11-16T22:08:12.424 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:08:12.426 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:12.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.060) 0:00:43.315 ******* 2020-11-16T22:08:12.474 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:08:12.475 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T22:08:12.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.050) 0:00:43.365 ******* 2020-11-16T22:08:12.514 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:08:12.516 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:08:12.549 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.041) 0:00:43.406 ******* 2020-11-16T22:08:12.577 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:08:12.579 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:12.610 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.060) 0:00:43.467 ******* 2020-11-16T22:08:12.731 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:12.732 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T22:08:12.878 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:08:12.880 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:08:12.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.303) 0:00:43.770 ******* 2020-11-16T22:08:12.931 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:08:12.932 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:08:12.966 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:12 +0000 (0:00:00.053) 0:00:43.823 ******* 2020-11-16T22:08:13.056 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:08:13.057 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:08:13.161 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T22:08:13.162 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T22:08:13.172 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:08:13.174 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:08:13.211 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.245) 0:00:44.069 ******* 2020-11-16T22:08:13.239 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:08:13.241 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:13.272 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.060) 0:00:44.130 ******* 2020-11-16T22:08:13.299 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:08:13.301 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:08:13.332 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.059) 0:00:44.189 ******* 2020-11-16T22:08:13.359 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:08:13.360 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:13.392 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.060) 0:00:44.249 ******* 2020-11-16T22:08:13.596 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:08:13.597 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ActiveEnterTimestampMonotonic": "17515450", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target var-run.mount nss-lookup.target network.target basic.target sysinit.target -.mount systemd-tmpfiles-setup.service system.slice local-fs.target systemd-journald.socket time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:03:31 UTC", "AssertTimestampMonotonic": "17514555", "Before": "xdm.service plymouth-quit.service getty@tty1.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ConditionTimestampMonotonic": "17514555", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-16T22:08:13.598 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1105", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ExecMainStartTimestampMonotonic": "17515422", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:31 UTC", "InactiveExitTimestampMonotonic": "17515450", "InvocationID": "bcd0cedba41b4c4a87833dd307d5f36b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2020-11-16T22:08:13.600 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1105", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2020-11-16T22:08:13.602 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:31 UTC", "StateChangeTimestampMonotonic": "17515450", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:31 UTC", "WatchdogTimestampMonotonic": "17515449", "WatchdogUSec": "0"}} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi062.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T22:08:13.640 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.247) 0:00:44.497 ******* 2020-11-16T22:08:13.672 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.032) 0:00:44.529 ******* 2020-11-16T22:08:13.681 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T22:08:13.682 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:08:13.722 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.048) 0:00:44.578 ******* 2020-11-16T22:08:13.727 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:08:13.729 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:08:13.764 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.043) 0:00:44.622 ******* 2020-11-16T22:08:13.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:13 +0000 (0:00:00.032) 0:00:44.654 ******* 2020-11-16T22:08:13.804 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T22:08:14.007 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:13 +0000 (0:00:00.053) 0:00:44.707 ******* Monday 16 November 2020 22:08:13 +0000 (0:00:00.037) 0:00:44.744 ******* TASK [common : Check for epel] ************************************************* skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Monday 16 November 2020 22:08:13 +0000 (0:00:00.051) 0:00:44.796 ******* TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:13 +0000 (0:00:00.044) 0:00:44.840 ******* TASK [common : Install nrpe without epel] ************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/na 2020-11-16T22:08:14.009 INFO:teuthology.task.ansible.out:gios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ActiveEnterTimestampMonotonic": "17763880", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target systemd-tmpfiles-setup.service basic.target system.slice local-fs.target -.mount var-run.mount remote-fs.target time-sync.target systemd-journald.socket network.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:03:43 UTC", "AssertTimestampMonotonic": "17763241", "Before": "plymouth-quit.service xdm.service getty@tty1.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanI 2020-11-16T22:08:14.211 INFO:teuthology.task.ansible.out:solate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ConditionTimestampMonotonic": "17763241", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1114", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ExecMainStartTimestampMonotonic": "17763853", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", " 2020-11-16T22:08:14.213 INFO:teuthology.task.ansible.out:FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:43 UTC", "InactiveExitTimestampMonotonic": "17763880", "InvocationID": "fc42f565eabf4593a5a3475c9b21ef8d", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1114", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": " 2020-11-16T22:08:14.415 INFO:teuthology.task.ansible.out:infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:43 UTC", "StateChangeTimestampMonotonic": "17763880", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": 2020-11-16T22:08:14.417 INFO:teuthology.task.ansible.out: "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:43 UTC", "WatchdogTimestampMonotonic": "17763879", "WatchdogUSec": "0"}} Monday 16 November 2020 22:08:14 +0000 (0:00:00.064) 0:00:44.905 ******* Monday 16 November 2020 22:08:14 +0000 (0:00:00.031) 0:00:44.937 ******* Monday 16 November 2020 22:08:14 +0000 (0:00:00.030) 0:00:44.967 ******* TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.046) 0:00:45.014 ******* Monday 16 November 2020 22:08:14 +0000 (0:00:00.032) 0:00:45.046 ******* TASK [common : Get SELinux status] ********************************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.042) 0:00:45.089 ******* TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.042) 0:00:45.131 ******* TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 Novemb 2020-11-16T22:08:14.623 INFO:teuthology.task.ansible.out:er 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.173 ******* TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.044) 0:00:45.218 ******* TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.040) 0:00:45.258 ******* TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.040) 0:00:45.298 ******* TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.040) 0:00:45.338 ******* TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.046) 0:00:45.385 ******* TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.426 ******* TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.040) 0:0 2020-11-16T22:08:14.625 INFO:teuthology.task.ansible.out:0:45.467 ******* TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:14.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.509 ******* 2020-11-16T22:08:14.657 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T22:08:14.658 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:08:14.691 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:14 +0000 (0:00:00.039) 0:00:45.548 ******* 2020-11-16T22:08:14.696 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:08:14.698 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:14.731 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:14 +0000 (0:00:00.039) 0:00:45.588 ******* 2020-11-16T22:08:14.736 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T22:08:14.939 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.040) 0:00:45.629 ******* TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.671 ******* TASK [testnode : Including major version specific variables.] ****************** ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", 2020-11-16T22:08:14.941 INFO:teuthology.task.ansible.out:"iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.712 ******* TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.753 ******* TASK [testnode : Including version specific variables.] ************************ ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} Monday 16 November 2020 22:08:14 +0000 (0:00:00.041) 0:00:45.795 ******* 2020-11-16T22:08:14.945 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T22:08:14.947 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:14.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:14 +0000 (0:00:00.043) 0:00:45.838 ******* 2020-11-16T22:08:15.187 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} Monday 16 November 2020 22:08:15 +0000 (0:00:00.061) 0:00:45.899 ******* TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} Monday 16 November 2020 22:08:15 +0000 (0:00:00.061) 0:00:45.961 ******* TASK [testnode : Including distro specific variables.] ************************* ok: [smithi185.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-comm 2020-11-16T22:08:15.189 INFO:teuthology.task.ansible.out:on", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} Monday 16 November 2020 22:08:15 +0000 (0:00:00.044) 0:00:46.005 ******* TASK [testnode : Including major version specific variables.] ****************** ok: [smithi185.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-16T22:08:15.219 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.071) 0:00:46.076 ******* 2020-11-16T22:08:15.236 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T22:08:15.237 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:08:15.438 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.056) 0:00:46.133 ******* Monday 16 November 2020 22:08:15 +0000 (0:00:00.029) 0:00:46.163 ******* TASK [testnode : Including version specific variables.] ************************ ok: [smithi185.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} Monday 16 November 2020 22:08:15 +0000 (0:00:00.042) 0:00:46.205 ******* TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:15 +0000 (0:00:00.041) 0:00:46.247 ******* TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:15.448 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.059) 0:00:46.306 ******* 2020-11-16T22:08:15.464 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:08:15.466 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T22:08:15.501 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.052) 0:00:46.358 ******* 2020-11-16T22:08:15.601 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:08:15.603 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T22:08:15.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.134) 0:00:46.493 ******* 2020-11-16T22:08:15.641 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T22:08:15.643 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:08:15.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.040) 0:00:46.533 ******* 2020-11-16T22:08:15.703 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:15.736 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.059) 0:00:46.593 ******* 2020-11-16T22:08:15.742 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T22:08:15.744 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T22:08:15.946 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:15 +0000 (0:00:00.050) 0:00:46.644 ******* TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:15 +0000 (0:00:00.041) 0:00:46.685 ******* 2020-11-16T22:08:15.980 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T22:08:15.981 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T22:08:16.014 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.185) 0:00:46.871 ******* 2020-11-16T22:08:16.020 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:08:16.022 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:08:16.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.041) 0:00:46.913 ******* 2020-11-16T22:08:16.083 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:08:16.084 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi096"}, "changed": false} 2020-11-16T22:08:16.117 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.060) 0:00:46.974 ******* 2020-11-16T22:08:16.261 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T22:08:16.262 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T22:08:16.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.203) 0:00:47.177 ******* 2020-11-16T22:08:16.379 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T22:08:16.380 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:08:16.413 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.093) 0:00:47.271 ******* 2020-11-16T22:08:16.441 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T22:08:16.443 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi185"}, "changed": false} 2020-11-16T22:08:16.474 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.060) 0:00:47.331 ******* 2020-11-16T22:08:16.513 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T22:08:16.515 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T22:08:16.660 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T22:08:16.701 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.226) 0:00:47.558 ******* 2020-11-16T22:08:16.728 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T22:08:16.730 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:16.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.066) 0:00:47.625 ******* 2020-11-16T22:08:16.795 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-16T22:08:16.797 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T22:08:16.834 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.066) 0:00:47.692 ******* 2020-11-16T22:08:16.872 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T22:08:16.874 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:16.916 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:16 +0000 (0:00:00.081) 0:00:47.773 ******* 2020-11-16T22:08:16.985 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T22:08:16.987 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi096", "ansible_hostname": "smithi096", "ansible_nodename": "smithi096"}, "changed": false, "name": "smithi096"} 2020-11-16T22:08:17.019 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.103) 0:00:47.877 ******* 2020-11-16T22:08:17.048 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi185", "ansible_hostname": "smithi185", "ansible_nodename": "smithi185"}, "changed": false, "name": "smithi185"} 2020-11-16T22:08:17.083 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.063) 0:00:47.940 ******* 2020-11-16T22:08:17.330 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:08:17.332 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T22:08:17.365 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.282) 0:00:48.222 ******* 2020-11-16T22:08:17.392 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T22:08:17.394 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T22:08:17.434 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.069) 0:00:48.291 ******* 2020-11-16T22:08:17.441 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T22:08:17.443 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T22:08:17.477 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.042) 0:00:48.334 ******* 2020-11-16T22:08:17.680 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:08:17.682 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:08:17.715 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.238) 0:00:48.572 ******* 2020-11-16T22:08:17.742 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T22:08:17.744 INFO:teuthology.task.ansible.out:[WARNING]: The service (nagios-nrpe-server) is actually an init script but the system is managed by systemd fatal: [smithi062.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-16T22:08:17.783 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.067) 0:00:48.640 ******* 2020-11-16T22:08:17.800 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T22:08:17.802 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T22:08:17.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.052) 0:00:48.693 ******* 2020-11-16T22:08:17.842 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T22:08:17.938 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:17 +0000 (0:00:00.053) 0:00:48.746 ******* TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:17.955 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:17 +0000 (0:00:00.066) 0:00:48.812 ******* 2020-11-16T22:08:17.982 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.020 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.065) 0:00:48.878 ******* 2020-11-16T22:08:18.048 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-16T22:08:18.049 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.065) 0:00:48.944 ******* 2020-11-16T22:08:18.113 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T22:08:18.115 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.151 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.065) 0:00:49.009 ******* 2020-11-16T22:08:18.178 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T22:08:18.180 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.065) 0:00:49.074 ******* 2020-11-16T22:08:18.223 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:08:18.225 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:08:18.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.041) 0:00:49.116 ******* 2020-11-16T22:08:18.265 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T22:08:18.267 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.304 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.045) 0:00:49.161 ******* 2020-11-16T22:08:18.311 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:08:18.312 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.344 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.040) 0:00:49.202 ******* 2020-11-16T22:08:18.351 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:08:18.353 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.391 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.046) 0:00:49.248 ******* 2020-11-16T22:08:18.422 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.030) 0:00:49.279 ******* 2020-11-16T22:08:18.428 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:08:18.429 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.472 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.050) 0:00:49.329 ******* 2020-11-16T22:08:18.503 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.031) 0:00:49.361 ******* 2020-11-16T22:08:18.510 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T22:08:18.512 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.046) 0:00:49.407 ******* 2020-11-16T22:08:18.582 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.031) 0:00:49.439 ******* 2020-11-16T22:08:18.587 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:08:18.589 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include package type specific vars.] ************************** ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T22:08:18.628 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.046) 0:00:49.485 ******* 2020-11-16T22:08:18.659 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.030) 0:00:49.516 ******* 2020-11-16T22:08:18.666 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:08:18.668 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including distro specific variables.] ************************* ok: [smithi062.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T22:08:18.870 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.048) 0:00:49.565 ******* Monday 16 November 2020 22:08:18 +0000 (0:00:00.030) 0:00:49.596 ******* TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including major version specific variables.] ****************** ok: [smithi062.fr 2020-11-16T22:08:18.872 INFO:teuthology.task.ansible.out:ont.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} Monday 16 November 2020 22:08:18 +0000 (0:00:00.047) 0:00:49.644 ******* Monday 16 November 2020 22:08:18 +0000 (0:00:00.031) 0:00:49.675 ******* TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Including version specific variables.] ************************ ok: [smithi062.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} Monday 16 November 2020 22:08:18 +0000 (0:00:00.047) 0:00:49.722 ******* 2020-11-16T22:08:18.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.038) 0:00:49.761 ******* 2020-11-16T22:08:18.910 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:08:18.912 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:18.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:18 +0000 (0:00:00.041) 0:00:49.802 ******* 2020-11-16T22:08:18.952 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:08:18.953 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.104 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.159) 0:00:49.961 ******* 2020-11-16T22:08:19.110 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T22:08:19.112 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.152 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.047) 0:00:50.009 ******* 2020-11-16T22:08:19.184 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.031) 0:00:50.041 ******* 2020-11-16T22:08:19.387 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:19 +0000 (0:00:00.041) 0:00:50.082 ******* TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:19 +0000 (0:00:00.039) 0:00:50.122 ******* TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:19 +0000 (0:00:00.039) 0:00:50.162 ******* TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:19 +0000 (0:00:00.040) 0:00:50.203 ******* TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T22:08:19.393 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.047) 0:00:50.251 ******* 2020-11-16T22:08:19.424 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.029) 0:00:50.281 ******* 2020-11-16T22:08:19.429 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:08:19.430 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.469 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.045) 0:00:50.326 ******* 2020-11-16T22:08:19.498 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.029) 0:00:50.355 ******* 2020-11-16T22:08:19.503 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T22:08:19.505 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.547 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.049) 0:00:50.405 ******* 2020-11-16T22:08:19.584 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.036) 0:00:50.442 ******* 2020-11-16T22:08:19.591 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:08:19.625 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.040) 0:00:50.482 ******* 2020-11-16T22:08:19.633 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:08:19.667 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.041) 0:00:50.524 ******* 2020-11-16T22:08:19.674 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:08:19.879 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.041) 0:00:50.566 ******* TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** Monday 16 November 2020 22:08:19 +0000 (0:00:00.042) 0:00:50.608 ******* TASK [testnode : Install GPG] ************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:19 +0000 (0:00:00.040) 0:00:50.648 ******* TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi062.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} TASK [testnode : Enable copr repos] ******************************************** Monday 16 November 2020 22:08:19 +0000 (0:00:00.048) 0:00:50.697 ******* Monday 16 November 2020 22:08:19 +0000 (0:00:00.031) 0:00:50.729 ******* TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.041) 0:00:50.770 ******* 2020-11-16T22:08:19.920 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:08:19.921 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.954 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.040) 0:00:50.811 ******* 2020-11-16T22:08:19.959 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:08:19.961 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:19.994 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:19 +0000 (0:00:00.040) 0:00:50.852 ******* 2020-11-16T22:08:20.000 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:08:20.002 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.036 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.041) 0:00:50.893 ******* 2020-11-16T22:08:20.041 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:08:20.043 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.075 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:50.933 ******* 2020-11-16T22:08:20.081 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:08:20.283 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:50.972 ******* TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.042) 0:00:51.015 ******* TASK [testnode : set_fact] ***************************************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.054 ******* TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.094 ******* TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.134 ******* TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.316 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.173 ******* 2020-11-16T22:08:20.323 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T22:08:20.324 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.214 ******* 2020-11-16T22:08:20.362 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T22:08:20.364 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T22:08:20.402 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.045) 0:00:51.260 ******* 2020-11-16T22:08:20.432 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.029) 0:00:51.290 ******* 2020-11-16T22:08:20.438 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:08:20.440 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set hostname var] ********************************************* ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi062"}, "changed": false} 2020-11-16T22:08:20.478 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.045) 0:00:51.335 ******* 2020-11-16T22:08:20.508 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.029) 0:00:51.365 ******* 2020-11-16T22:08:20.513 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:08:20.719 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.406 ******* TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.445 ******* TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.484 ******* TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.524 ******* TASK [testnode : Install packages] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.045) 0:00:51.570 ******* TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.753 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.610 ******* 2020-11-16T22:08:20.761 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:08:20.762 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.794 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.651 ******* 2020-11-16T22:08:20.801 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:20.802 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.041) 0:00:51.693 ******* 2020-11-16T22:08:20.841 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:08:20.842 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.875 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.732 ******* 2020-11-16T22:08:20.880 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:08:20.882 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.914 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.039) 0:00:51.771 ******* 2020-11-16T22:08:20.919 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:20.921 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:20.954 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:20 +0000 (0:00:00.040) 0:00:51.811 ******* 2020-11-16T22:08:20.960 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:08:21.163 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set the system's hostname] ************************************ changed: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi062.front.sepia.ceph.com", "ansible_hostname": "smithi062", "ansible_nodename": "smithi062"}, "changed": true, "name": "smithi062"} Monday 16 November 2020 22:08:20 +0000 (0:00:00.046) 0:00:51.858 ******* Monday 16 November 2020 22:08:21 +0000 (0:00:00.031) 0:00:51.889 ******* TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:51.931 ******* TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:51.972 ******* TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:21 +0000 (0:00:00.039) 0:00:52.011 ******* TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.194 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.039) 0:00:52.051 ******* 2020-11-16T22:08:21.200 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:08:21.202 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.040) 0:00:52.092 ******* 2020-11-16T22:08:21.248 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:08:21.250 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.280 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.045) 0:00:52.138 ******* 2020-11-16T22:08:21.287 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:08:21.288 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.321 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.040) 0:00:52.179 ******* 2020-11-16T22:08:21.328 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:08:21.330 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.362 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.040) 0:00:52.219 ******* 2020-11-16T22:08:21.368 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:08:21.370 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.578 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.040) 0:00:52.260 ******* TASK [testnode : Install abrt] ************************************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:52.302 ******* TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} Monday 16 November 2020 22:08:21 +0000 (0:00:00.048) 0:00:52.350 ******* Monday 16 November 2020 22:08:21 +0000 (0:00:00.029) 0:00:52.380 ******* TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:08:21 +0000 (0:00:00.043) 0:00:52.423 ******* TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:52.465 ******* 2020-11-16T22:08:21.614 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:08:21.616 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.648 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:52.506 ******* 2020-11-16T22:08:21.654 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:08:21.655 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.688 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.039) 0:00:52.545 ******* 2020-11-16T22:08:21.693 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:08:21.695 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.728 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.039) 0:00:52.585 ******* 2020-11-16T22:08:21.755 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:08:21.756 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:21.788 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.060) 0:00:52.646 ******* 2020-11-16T22:08:21.815 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:08:21.817 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.004401", "end": "2020-11-16 22:08:21.814700", "rc": 0, "start": "2020-11-16 22:08:21.810299", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:21.855 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:21 +0000 (0:00:00.066) 0:00:52.712 ******* 2020-11-16T22:08:22.059 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** Monday 16 November 2020 22:08:21 +0000 (0:00:00.060) 0:00:52.773 ******* TASK [testnode : Check for custom repos] *************************************** ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005218", "end": "2020-11-16 22:08:21.915614", "rc": 0, "start": "2020-11-16 22:08:21.910396", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Monday 16 November 2020 22:08:21 +0000 (0:00:00.041) 0:00:52.814 ******* TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} TASK [testnode : Remove custom repos] ****************************************** Monday 16 November 2020 22:08:22 +0000 (0:00:00.057) 0:00:52.872 ******* Monday 16 November 2020 22:08:22 +0000 (0:00:00.032) 0:00:52.904 ******* 2020-11-16T22:08:22.231 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:08:22.233 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:08:22.266 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.218) 0:00:53.123 ******* 2020-11-16T22:08:22.294 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:08:22.295 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.327 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.061) 0:00:53.184 ******* 2020-11-16T22:08:22.354 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:08:22.356 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:08:22.389 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.061) 0:00:53.246 ******* 2020-11-16T22:08:22.416 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:08:22.417 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.449 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.060) 0:00:53.307 ******* 2020-11-16T22:08:22.620 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T22:08:22.622 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T22:08:22.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.210) 0:00:53.518 ******* 2020-11-16T22:08:22.688 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.727 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.066) 0:00:53.584 ******* 2020-11-16T22:08:22.753 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.791 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.064) 0:00:53.649 ******* 2020-11-16T22:08:22.818 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.860 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.068) 0:00:53.717 ******* 2020-11-16T22:08:22.887 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.925 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.065) 0:00:53.783 ******* 2020-11-16T22:08:22.952 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:22.991 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:22 +0000 (0:00:00.065) 0:00:53.848 ******* 2020-11-16T22:08:23.018 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:08:23.019 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:53.914 ******* 2020-11-16T22:08:23.082 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:08:23.084 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.064) 0:00:53.978 ******* 2020-11-16T22:08:23.148 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.187 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.066) 0:00:54.044 ******* 2020-11-16T22:08:23.214 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T22:08:23.252 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:54.110 ******* 2020-11-16T22:08:23.279 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T22:08:23.317 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:54.175 ******* 2020-11-16T22:08:23.345 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T22:08:23.383 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:54.241 ******* 2020-11-16T22:08:23.408 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T22:08:23.409 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.062) 0:00:54.303 ******* 2020-11-16T22:08:23.483 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T22:08:23.484 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.523 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.077) 0:00:54.380 ******* 2020-11-16T22:08:23.550 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T22:08:23.552 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.589 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.066) 0:00:54.447 ******* 2020-11-16T22:08:23.617 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:08:23.618 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.655 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:54.513 ******* 2020-11-16T22:08:23.662 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:08:23.664 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} ok: [smithi185.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T22:08:23.675 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T22:08:23.676 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.713 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.057) 0:00:54.571 ******* 2020-11-16T22:08:23.740 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T22:08:23.742 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.778 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.065) 0:00:54.636 ******* 2020-11-16T22:08:23.805 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T22:08:23.806 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.843 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.064) 0:00:54.700 ******* 2020-11-16T22:08:23.869 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T22:08:23.871 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.907 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.064) 0:00:54.765 ******* 2020-11-16T22:08:23.934 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:08:23.936 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:23.975 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:23 +0000 (0:00:00.068) 0:00:54.833 ******* 2020-11-16T22:08:24.002 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:08:24.004 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.040 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:54.898 ******* 2020-11-16T22:08:24.066 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:24.068 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.105 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:54.962 ******* 2020-11-16T22:08:24.132 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T22:08:24.133 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.170 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.065) 0:00:55.027 ******* 2020-11-16T22:08:24.196 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:08:24.198 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.234 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:55.092 ******* 2020-11-16T22:08:24.260 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:24.262 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.298 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.063) 0:00:55.156 ******* 2020-11-16T22:08:24.324 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T22:08:24.326 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:55.220 ******* 2020-11-16T22:08:24.390 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T22:08:24.391 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.428 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:55.285 ******* 2020-11-16T22:08:24.454 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T22:08:24.456 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.493 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.065) 0:00:55.350 ******* 2020-11-16T22:08:24.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T22:08:24.522 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.563 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.070) 0:00:55.420 ******* 2020-11-16T22:08:24.571 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:08:24.572 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} ok: [smithi185.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T22:08:24.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.043) 0:00:55.464 ******* 2020-11-16T22:08:24.639 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.031) 0:00:55.496 ******* 2020-11-16T22:08:24.645 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T22:08:24.646 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.685 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.046) 0:00:55.543 ******* 2020-11-16T22:08:24.713 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T22:08:24.715 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.752 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.066) 0:00:55.609 ******* 2020-11-16T22:08:24.777 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T22:08:24.779 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.816 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.064) 0:00:55.673 ******* 2020-11-16T22:08:24.843 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-16T22:08:24.844 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:24.881 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:24 +0000 (0:00:00.065) 0:00:55.739 ******* 2020-11-16T22:08:25.084 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T22:08:25.086 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.012882", "end": "2020-11-16 22:08:25.030745", "rc": 0, "start": "2020-11-16 22:08:25.017863", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:25.124 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.242) 0:00:55.981 ******* 2020-11-16T22:08:25.151 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T22:08:25.190 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.066) 0:00:56.048 ******* 2020-11-16T22:08:25.521 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:08:25.523 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:25.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.365) 0:00:56.414 ******* 2020-11-16T22:08:25.563 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:25.602 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.045) 0:00:56.460 ******* 2020-11-16T22:08:25.609 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:08:25.643 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.041) 0:00:56.501 ******* 2020-11-16T22:08:25.685 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.041) 0:00:56.542 ******* 2020-11-16T22:08:25.693 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T22:08:25.694 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T22:08:25.733 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.048) 0:00:56.591 ******* 2020-11-16T22:08:25.762 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T22:08:25.763 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:25.802 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.068) 0:00:56.659 ******* 2020-11-16T22:08:25.810 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:08:25.811 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033484", "end": "2020-11-16 22:08:25.799932", "rc": 0, "start": "2020-11-16 22:08:25.766448", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:25.846 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.043) 0:00:56.703 ******* 2020-11-16T22:08:25.854 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033694", "end": "2020-11-16 22:08:25.842117", "rc": 0, "start": "2020-11-16 22:08:25.808423", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:25.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:25 +0000 (0:00:00.043) 0:00:56.747 ******* 2020-11-16T22:08:27.713 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:08:27.715 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T22:08:28.590 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:08:28.591 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:28.623 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:28 +0000 (0:00:02.733) 0:00:59.480 ******* 2020-11-16T22:08:28.712 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:28.748 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:28 +0000 (0:00:00.124) 0:00:59.605 ******* 2020-11-16T22:08:29.563 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T22:08:29.565 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T22:08:29.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:29 +0000 (0:00:00.855) 0:01:00.460 ******* 2020-11-16T22:08:30.685 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T22:08:30.687 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:08:30.724 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:30 +0000 (0:00:01.120) 0:01:01.581 ******* 2020-11-16T22:08:30.750 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T22:08:30.788 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:30 +0000 (0:00:00.064) 0:01:01.646 ******* 2020-11-16T22:08:31.053 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T22:08:31.055 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.087021", "end": "2020-11-16 22:08:31.010772", "rc": 0, "start": "2020-11-16 22:08:30.923751", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:31.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:31 +0000 (0:00:00.303) 0:01:01.949 ******* 2020-11-16T22:08:35.280 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T22:08:35.281 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:08:35.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:35 +0000 (0:00:04.228) 0:01:06.178 ******* 2020-11-16T22:08:37.700 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T22:08:37.702 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories 2020-11-16T22:08:37.907 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T22:08:37.909 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:08:37 +0000 (0:00:02.414) 0:01:08.593 ******* 2020-11-16T22:08:38.110 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove 2020-11-16T22:08:38.112 INFO:teuthology.task.ansible.out:them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102076 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspee 2020-11-16T22:08:38.114 INFO:teuthology.task.ansible.out:x1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102076 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:08:38 +0000 (0:00:00.376) 0:01:08.969 ******* 2020-11-16T22:08:38.598 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:38.600 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:38.632 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:38 +0000 (0:00:00.520) 0:01:09.490 ******* 2020-11-16T22:08:38.660 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:08:38.661 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:38.693 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:38 +0000 (0:00:00.060) 0:01:09.550 ******* 2020-11-16T22:08:38.987 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:38.989 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:39.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:39 +0000 (0:00:00.328) 0:01:09.878 ******* 2020-11-16T22:08:39.049 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T22:08:39.051 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:39.081 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:39 +0000 (0:00:00.060) 0:01:09.939 ******* 2020-11-16T22:08:47.645 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:47.647 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9795 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T22:08:47.648 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T22:08:47.650 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9795 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T22:08:47.855 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:08:47 +0000 (0:00:08.599) 0:01:18.538 ******* TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi062.front.sepia.ceph.co 2020-11-16T22:08:47.857 INFO:teuthology.task.ansible.out:m] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104569 files and directories currently installed.)\r\nRemoving qemu 2020-11-16T22:08:48.059 INFO:teuthology.task.ansible.out:-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 4 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database 2020-11-16T22:08:48.263 INFO:teuthology.task.ansible.out: ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104569 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:08:47 +0000 (0:00:00.058) 0:01:18.597 ******* TASK [testnode : Install packages] ********************************************* changed: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 2020-11-16T22:08:48.467 INFO:teuthology.task.ansible.out:http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9617 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101934 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ... 2020-11-16T22:08:48.469 INFO:teuthology.task.ansible.out:\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubunt 2020-11-16T22:08:48.671 INFO:teuthology.task.ansible.out:u bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9617 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101934 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacki 2020-11-16T22:08:48.673 INFO:teuthology.task.ansible.out:ng qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} Monday 16 November 2020 22:08:47 +0000 (0:00:00.099) 0:01:18.696 ******* TASK [testnode : Install non aarch64 packages] ********************************* ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} Monday 16 November 2020 22:08:48 +0000 (0:00:00.770) 0:01:19.467 ******* 2020-11-16T22:08:48.874 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} Monday 16 November 2020 22:08:48 +0000 (0:00:00.135) 0:01:19.602 ******* TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} Monday 16 November 2020 22:08:48 +0000 (0:00:00.042) 0:01:19.645 ******* TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [tes 2020-11-16T22:08:48.876 INFO:teuthology.task.ansible.out:tnode : Upgrade packages] ********************************************* ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} Monday 16 November 2020 22:08:48 +0000 (0:00:00.065) 0:01:19.710 ******* 2020-11-16T22:08:48.886 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:48 +0000 (0:00:00.032) 0:01:19.743 ******* 2020-11-16T22:08:48.892 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:08:48.894 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:48.934 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:48 +0000 (0:00:00.048) 0:01:19.792 ******* 2020-11-16T22:08:48.966 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:48 +0000 (0:00:00.032) 0:01:19.824 ******* 2020-11-16T22:08:48.994 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T22:08:48.995 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:49.028 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.061) 0:01:19.885 ******* 2020-11-16T22:08:49.034 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:08:49.035 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:08:49.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.040) 0:01:19.926 ******* 2020-11-16T22:08:49.149 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:08:49.183 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.114) 0:01:20.040 ******* 2020-11-16T22:08:49.379 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:08:49.381 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:08:49.414 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.230) 0:01:20.271 ******* 2020-11-16T22:08:49.493 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:08:49.527 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.113) 0:01:20.385 ******* 2020-11-16T22:08:49.565 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:08:49.567 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:08:49.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.075) 0:01:20.461 ******* 2020-11-16T22:08:49.683 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:08:49.717 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.114) 0:01:20.575 ******* 2020-11-16T22:08:49.734 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:08:49.735 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:08:49.769 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.051) 0:01:20.626 ******* 2020-11-16T22:08:49.849 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:08:49.883 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:49 +0000 (0:00:00.114) 0:01:20.741 ******* 2020-11-16T22:08:50.046 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:08:50.047 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ActiveEnterTimestampMonotonic": "18161529", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount network.target remote-fs.target systemd-journald.socket system.slice systemd-tmpfiles-setup.service nss-lookup.target sysinit.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:03:43 UTC", "AssertTimestampMonotonic": "17763996", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ConditionTimestampMonotonic": "17763995", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T22:08:50.049 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1202", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ExecMainStartTimestampMonotonic": "18161519", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:43 UTC", "InactiveExitTimestampMonotonic": "17764841", "InvocationID": "f9bb3c5c28bb4e7abb24f9134847b818", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T22:08:50.050 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1202", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T22:08:50.052 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:43 UTC", "StateChangeTimestampMonotonic": "18161529", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:43 UTC", "WatchdogTimestampMonotonic": "18161529", "WatchdogUSec": "0"}} 2020-11-16T22:08:50.083 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.199) 0:01:20.940 ******* 2020-11-16T22:08:50.110 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:08:50.111 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.060) 0:01:21.001 ******* 2020-11-16T22:08:50.171 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:08:50.172 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.204 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.060) 0:01:21.061 ******* 2020-11-16T22:08:50.212 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:08:50.213 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ActiveEnterTimestampMonotonic": "17918090", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target systemd-journald.socket systemd-tmpfiles-setup.service network.target sysinit.target -.mount system.slice basic.target remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:03:31 UTC", "AssertTimestampMonotonic": "17517329", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ConditionTimestampMonotonic": "17517328", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T22:08:50.214 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1188", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ExecMainStartTimestampMonotonic": "17918079", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:31 UTC", "InactiveExitTimestampMonotonic": "17518542", "InvocationID": "50445aa879bb4e7e9c449f799ea32df5", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T22:08:50.216 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1188", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T22:08:50.218 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:31 UTC", "StateChangeTimestampMonotonic": "17918090", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:31 UTC", "WatchdogTimestampMonotonic": "17918089", "WatchdogUSec": "0"}} 2020-11-16T22:08:50.247 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.043) 0:01:21.105 ******* 2020-11-16T22:08:50.254 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:08:50.256 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.289 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.041) 0:01:21.147 ******* 2020-11-16T22:08:50.296 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T22:08:50.297 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.329 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.187 ******* 2020-11-16T22:08:50.336 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:08:50.337 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.369 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.039) 0:01:21.226 ******* 2020-11-16T22:08:50.375 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:08:50.376 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.266 ******* 2020-11-16T22:08:50.416 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:08:50.417 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.453 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.043) 0:01:21.310 ******* 2020-11-16T22:08:50.459 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:08:50.460 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.493 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.350 ******* 2020-11-16T22:08:50.499 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:08:50.501 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.534 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.041) 0:01:21.391 ******* 2020-11-16T22:08:50.541 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T22:08:50.542 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.578 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.043) 0:01:21.435 ******* 2020-11-16T22:08:50.585 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:08:50.586 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.619 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.041) 0:01:21.477 ******* 2020-11-16T22:08:50.626 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:08:50.627 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.660 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.518 ******* 2020-11-16T22:08:50.667 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:08:50.669 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.701 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.558 ******* 2020-11-16T22:08:50.707 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:08:50.708 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.599 ******* 2020-11-16T22:08:50.748 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:08:50.750 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.782 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.639 ******* 2020-11-16T22:08:50.789 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:08:50.791 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.826 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.044) 0:01:21.684 ******* 2020-11-16T22:08:50.833 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:08:50.834 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.867 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.724 ******* 2020-11-16T22:08:50.873 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:08:50.874 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.041) 0:01:21.766 ******* 2020-11-16T22:08:50.916 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:50.918 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.042) 0:01:21.808 ******* 2020-11-16T22:08:50.957 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:08:50.959 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:50.991 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:50 +0000 (0:00:00.040) 0:01:21.849 ******* 2020-11-16T22:08:51.001 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:51.003 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.036 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.044) 0:01:21.893 ******* 2020-11-16T22:08:51.042 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:08:51.043 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.080 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.044) 0:01:21.937 ******* 2020-11-16T22:08:51.087 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:08:51.088 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:21.978 ******* 2020-11-16T22:08:51.128 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:51.130 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.162 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.019 ******* 2020-11-16T22:08:51.168 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:51.169 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.203 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.060 ******* 2020-11-16T22:08:51.208 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:08:51.210 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.100 ******* 2020-11-16T22:08:51.249 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:51.251 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.039) 0:01:22.140 ******* 2020-11-16T22:08:51.288 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:08:51.290 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.322 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.039) 0:01:22.180 ******* 2020-11-16T22:08:51.330 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:51.331 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.043) 0:01:22.223 ******* 2020-11-16T22:08:51.375 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:51.376 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.413 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.047) 0:01:22.270 ******* 2020-11-16T22:08:51.423 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:08:51.424 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.043) 0:01:22.314 ******* 2020-11-16T22:08:51.462 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:51.464 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.496 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.039) 0:01:22.354 ******* 2020-11-16T22:08:51.502 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:08:51.504 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.539 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.042) 0:01:22.397 ******* 2020-11-16T22:08:51.545 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:51.547 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.580 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.437 ******* 2020-11-16T22:08:51.587 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:08:51.588 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.622 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.042) 0:01:22.480 ******* 2020-11-16T22:08:51.629 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:08:51.630 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.045) 0:01:22.525 ******* 2020-11-16T22:08:51.675 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:51.676 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.709 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.041) 0:01:22.567 ******* 2020-11-16T22:08:51.716 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:08:51.718 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.752 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.042) 0:01:22.610 ******* 2020-11-16T22:08:51.759 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:51.761 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.795 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.042) 0:01:22.653 ******* 2020-11-16T22:08:51.801 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:08:51.803 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.836 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.693 ******* 2020-11-16T22:08:51.841 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:51.843 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.875 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.039) 0:01:22.732 ******* 2020-11-16T22:08:51.880 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:51.882 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.914 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.039) 0:01:22.772 ******* 2020-11-16T22:08:51.922 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:51.923 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.955 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.040) 0:01:22.812 ******* 2020-11-16T22:08:51.962 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:51.963 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:51.996 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:51 +0000 (0:00:00.041) 0:01:22.853 ******* 2020-11-16T22:08:52.003 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:52.004 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.040) 0:01:22.894 ******* 2020-11-16T22:08:52.044 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:52.045 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.078 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.041) 0:01:22.936 ******* 2020-11-16T22:08:52.085 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:52.087 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.119 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.041) 0:01:22.977 ******* 2020-11-16T22:08:52.126 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:08:52.127 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.160 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.040) 0:01:23.017 ******* 2020-11-16T22:08:52.168 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:08:52.169 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.201 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.041) 0:01:23.058 ******* 2020-11-16T22:08:52.208 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:08:52.209 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.243 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.041) 0:01:23.100 ******* 2020-11-16T22:08:52.269 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:08:52.271 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.307 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.064) 0:01:23.165 ******* 2020-11-16T22:08:52.323 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:08:52.325 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:08:52.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.049) 0:01:23.214 ******* 2020-11-16T22:08:52.363 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:08:52.364 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.397 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.040) 0:01:23.254 ******* 2020-11-16T22:08:52.528 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:08:52.530 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:08:52.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.164) 0:01:23.419 ******* 2020-11-16T22:08:52.660 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:08:52.662 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:08:52.695 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.133) 0:01:23.552 ******* 2020-11-16T22:08:52.723 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.756 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.060) 0:01:23.613 ******* 2020-11-16T22:08:52.868 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:08:52.869 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:08:52.902 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.145) 0:01:23.759 ******* 2020-11-16T22:08:52.929 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:52.962 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:52 +0000 (0:00:00.060) 0:01:23.820 ******* 2020-11-16T22:08:53.072 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:08:53.074 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:08:53.106 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.144) 0:01:23.964 ******* 2020-11-16T22:08:53.290 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:08:53.324 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.217) 0:01:24.181 ******* 2020-11-16T22:08:53.414 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T22:08:53.415 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:08:53.449 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.125) 0:01:24.306 ******* 2020-11-16T22:08:53.631 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:08:53.671 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.222) 0:01:24.529 ******* 2020-11-16T22:08:53.771 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:08:53.773 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:08:53.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.140) 0:01:24.669 ******* 2020-11-16T22:08:53.943 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:08:53.944 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:08:53.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:53 +0000 (0:00:00.169) 0:01:24.838 ******* 2020-11-16T22:08:53.987 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:08:53.988 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:08:54.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.040) 0:01:24.879 ******* 2020-11-16T22:08:54.027 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:08:54.028 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.062 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.040) 0:01:24.920 ******* 2020-11-16T22:08:54.090 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:08:54.092 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.124 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.061) 0:01:24.981 ******* 2020-11-16T22:08:54.151 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:08:54.152 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.185 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.060) 0:01:25.042 ******* 2020-11-16T22:08:54.216 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.031) 0:01:25.073 ******* 2020-11-16T22:08:54.222 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:08:54.224 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.256 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.040) 0:01:25.114 ******* 2020-11-16T22:08:54.263 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T22:08:54.264 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.296 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.039) 0:01:25.154 ******* 2020-11-16T22:08:54.302 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:08:54.303 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.336 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.039) 0:01:25.193 ******* 2020-11-16T22:08:54.344 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T22:08:54.345 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.377 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.040) 0:01:25.234 ******* 2020-11-16T22:08:54.387 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:08:54.388 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.422 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.045) 0:01:25.279 ******* 2020-11-16T22:08:54.429 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:08:54.430 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:08:54.468 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.045) 0:01:25.325 ******* 2020-11-16T22:08:54.475 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T22:08:54.477 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.512 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.044) 0:01:25.369 ******* 2020-11-16T22:08:54.521 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:08:54.523 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.534 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.535 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.568 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.056) 0:01:25.425 ******* 2020-11-16T22:08:54.575 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:08:54.577 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi185.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi185.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.577 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.578 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.589 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.590 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.600 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.602 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi185.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.067) 0:01:25.493 ******* 2020-11-16T22:08:54.673 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.037) 0:01:25.530 ******* 2020-11-16T22:08:54.681 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.682 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.682 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.693 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.704 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.706 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.715 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:08:54.726 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.737 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.738 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.748 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.749 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.759 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.770 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.771 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.809 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.135) 0:01:25.666 ******* 2020-11-16T22:08:54.857 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T22:08:54.859 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003366", "end": "2020-11-16 22:08:54.835683", "rc": 0, "start": "2020-11-16 22:08:54.832317", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:08:54.896 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.086) 0:01:25.753 ******* 2020-11-16T22:08:54.923 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T22:08:54.925 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:54.961 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:54 +0000 (0:00:00.065) 0:01:25.818 ******* 2020-11-16T22:08:54.987 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T22:08:54.989 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:55.022 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.061) 0:01:25.879 ******* 2020-11-16T22:08:55.030 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T22:08:55.032 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003474", "end": "2020-11-16 22:08:54.970464", "rc": 0, "start": "2020-11-16 22:08:54.966990", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:08:55.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.046) 0:01:25.926 ******* 2020-11-16T22:08:55.075 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T22:08:55.077 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:55.115 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.045) 0:01:25.972 ******* 2020-11-16T22:08:55.121 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T22:08:55.123 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:55.162 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.046) 0:01:26.019 ******* 2020-11-16T22:08:55.190 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T22:08:55.191 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:55.223 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.061) 0:01:26.081 ******* 2020-11-16T22:08:55.250 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T22:08:55.252 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:55.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:55 +0000 (0:00:00.060) 0:01:26.141 ******* 2020-11-16T22:08:56.005 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:08:56.007 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:56.040 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.756) 0:01:26.897 ******* 2020-11-16T22:08:56.067 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:08:56.068 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:56.100 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.060) 0:01:26.957 ******* 2020-11-16T22:08:56.137 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:08:56.139 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item={'block_used': 13783676, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311432, 'size_available': 926971625472, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114489, 'block_size': 4096, 'inode_available': 60940487}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311432, "block_size": 4096, "block_total": 240095108, "block_used": 13783676, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940487, "inode_total": 61054976, "inode_used": 114489, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971625472, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:08:56.171 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.071) 0:01:27.029 ******* 2020-11-16T22:08:56.178 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:08:56.180 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:08:56.214 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.042) 0:01:27.071 ******* 2020-11-16T22:08:56.241 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:56.275 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.061) 0:01:27.132 ******* 2020-11-16T22:08:56.302 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:08:56.304 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010798", "end": "2020-11-16 22:08:56.298574", "rc": 0, "start": "2020-11-16 22:08:56.287776", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:56.336 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.061) 0:01:27.194 ******* 2020-11-16T22:08:56.343 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:08:56.345 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13783674, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311434, 'size_available': 926971633664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311434, "block_size": 4096, "block_total": 240095108, "block_used": 13783674, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971633664, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:08:56.382 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.046) 0:01:27.240 ******* 2020-11-16T22:08:56.390 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:08:56.391 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'block_used': 13783676, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311432, 'size_available': 926971625472, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114489, 'block_size': 4096, 'inode_available': 60940487}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311432, "block_size": 4096, "block_total": 240095108, "block_used": 13783676, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940487, "inode_total": 61054976, "inode_used": 114489, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971625472, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:56.427 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.043) 0:01:27.284 ******* 2020-11-16T22:08:56.454 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:08:56.488 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.061) 0:01:27.345 ******* 2020-11-16T22:08:56.515 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:08:56.517 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009729", "end": "2020-11-16 22:08:56.516819", "rc": 0, "start": "2020-11-16 22:08:56.507090", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:08:56.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.062) 0:01:27.408 ******* 2020-11-16T22:08:56.558 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:08:56.560 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:56.591 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:08:56.593 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13783674, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311434, 'size_available': 926971633664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311434, "block_size": 4096, "block_total": 240095108, "block_used": 13783674, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971633664, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:56.625 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.074) 0:01:27.482 ******* 2020-11-16T22:08:56.645 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:08:56.677 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:56 +0000 (0:00:00.051) 0:01:27.534 ******* 2020-11-16T22:08:56.715 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:08:56.716 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:58.661 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzp5A07KGUP16v68T3UGF3X7a0sNxIVqJv'], 'uuids': ['e7494e78-395f-42d4-832e-b855f0f14f49']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.006229", "end": "2020-11-16 22:08:58.642185", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzp5A07KGUP16v68T3UGF3X7a0sNxIVqJv"], "labels": [], "masters": [], "uuids": ["e7494e78-395f-42d4-832e-b855f0f14f49"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:08:56.635956", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:08:58.662 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:58.673 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:58.914 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bc9aAlg8IdPe1rAXSt5Vh2B2YpMfOArWG'], 'uuids': ['985affca-6095-47b9-8db4-c24983452fa9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.072440", "end": "2020-11-16 22:08:58.906925", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bc9aAlg8IdPe1rAXSt5Vh2B2YpMfOArWG"], "labels": [], "masters": [], "uuids": ["985affca-6095-47b9-8db4-c24983452fa9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:08:56.834485", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:08:58.925 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:58.937 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:59.907 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T22:08:59.909 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9382 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T22:08:59.911 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104427 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T22:08:59.912 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 4 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9382 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T22:08:59.914 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104427 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 've 2020-11-16T22:08:59.915 INFO:teuthology.task.ansible.out:ndor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzSwuMah559AC1VOGJowLZK5CXv0rsYeqJ'], 'uuids': ['5b349b2a-424f-4c3c-8649-47ee57919e3d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.111249", "end": "2020-11-16 22:08:59.892921", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzSwuMah559AC1VOGJowLZK5CXv0rsYeqJ"], "labels": [], "masters": [], "uuids": ["5b349b2a-424f-4c3c-8649-47ee57919e3d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:08:58.781672", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'lin 2020-11-16T22:08:59.917 INFO:teuthology.task.ansible.out:ks': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTPG', 'wwn-0x5000c5009293f2ac'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTPG-part1', 'wwn-0x5000c5009293f2ac-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293f2ac', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTPG', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTPG", "wwn-0x5000c5009293f2ac"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTPG-part1", "wwn-0x5000c5009293f2ac-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTPG", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293f2ac"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:08:59.952 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:08:59 +0000 (0:00:03.274) 0:01:30.809 ******* 2020-11-16T22:09:00.189 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bSzDKkzand7pjlchx6i51uisuIFj00hAm'], 'uuids': ['76b305be-5de6-4d90-a389-0a6b4da2ed03']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.129170", "end": "2020-11-16 22:09:00.172550", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bSzDKkzand7pjlchx6i51uisuIFj00hAm"], "labels": [], "masters": [], "uuids": ["76b305be-5de6-4d90-a389-0a6b4da2ed03"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:08:59.043380", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:00.191 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfbebc', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGMP', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, "skip_reason": 2020-11-16T22:09:00.192 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T22:09:01.053 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:09:01.093 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:01 +0000 (0:00:01.140) 0:01:31.950 ******* 2020-11-16T22:09:01.213 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:01.215 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-0cB3dv-iyfD-iMtx-1X5M-9436-MHFy-93d0Cm', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300KY400BGN', 'nvme-nvme.8086-435646543632333330304b5934303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.173332", "end": "2020-11-16 22:09:01.197367", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-0cB3dv-iyfD-iMtx-1X5M-9436-MHFy-93d0Cm", "nvme-INTEL_SSDPEDMD400G4_CVFT623300KY400BGN", "nvme-nvme.8086-435646543632333330304b5934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:00.024035", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-16T22:09:01.217 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:01.226 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:01.226 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:01.269 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T22:09:01.270 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T22:09:01.308 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:01 +0000 (0:00:00.215) 0:01:32.165 ******* 2020-11-16T22:09:01.335 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T22:09:01.337 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:01.373 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:01 +0000 (0:00:00.065) 0:01:32.231 ******* 2020-11-16T22:09:01.545 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:01.547 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-eJP9xu-83Nf-T9od-Ln1a-rFrE-msBn-KMAgXO', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.226852", "end": "2020-11-16 22:09:01.527720", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-eJP9xu-83Nf-T9od-Ln1a-rFrE-msBn-KMAgXO", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:00.300868", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-16T22:09:01.549 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:01.558 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:01.570 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T22:09:01.571 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T22:09:01.611 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:01 +0000 (0:00:00.237) 0:01:32.468 ******* 2020-11-16T22:09:02.088 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T22:09:02.090 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T22:09:02.127 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:02 +0000 (0:00:00.516) 0:01:32.985 ******* 2020-11-16T22:09:02.351 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T22:09:02.353 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T22:09:02.391 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:02 +0000 (0:00:00.263) 0:01:33.248 ******* 2020-11-16T22:09:02.481 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:02.483 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzW8UunQd2qX1cTp5CSgzvUcCF0oZGQ3M2'], 'uuids': ['acad91eb-c483-42e7-b00d-ba74c739ad06']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.130387", "end": "2020-11-16 22:09:02.467119", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzW8UunQd2qX1cTp5CSgzvUcCF0oZGQ3M2"], "labels": [], "masters": [], "uuids": ["acad91eb-c483-42e7-b00d-ba74c739ad06"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:01.336732", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:02.493 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:02.505 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:02.568 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-16T22:09:02.570 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:09:02.608 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:02 +0000 (0:00:00.216) 0:01:33.465 ******* 2020-11-16T22:09:02.832 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:02.834 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bXp641cKBwCBjqYWXzJ2cLZxGkGWfNYT2'], 'uuids': ['97c9a7ab-4cb8-4db6-94ee-4af9ae9cac78']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.165725", "end": "2020-11-16 22:09:02.831229", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bXp641cKBwCBjqYWXzJ2cLZxGkGWfNYT2"], "labels": [], "masters": [], "uuids": ["97c9a7ab-4cb8-4db6-94ee-4af9ae9cac78"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:01.665504", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:02.844 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:02.856 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:02.898 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T22:09:02.900 INFO:teuthology.task.ansible.out:[WARNING]: The service (apache2) is actually an init script but the system is managed by systemd fatal: [smithi062.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-16T22:09:02.938 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:02 +0000 (0:00:00.329) 0:01:33.795 ******* 2020-11-16T22:09:02.965 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.006 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.068) 0:01:33.864 ******* 2020-11-16T22:09:03.035 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T22:09:03.036 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.073 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.067) 0:01:33.931 ******* 2020-11-16T22:09:03.100 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T22:09:03.102 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.068) 0:01:33.999 ******* 2020-11-16T22:09:03.420 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.315) 0:01:34.314 ******* 2020-11-16T22:09:03.484 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T22:09:03.486 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.065) 0:01:34.380 ******* 2020-11-16T22:09:03.549 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T22:09:03.550 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.587 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.064) 0:01:34.444 ******* 2020-11-16T22:09:03.613 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T22:09:03.614 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.064) 0:01:34.509 ******* 2020-11-16T22:09:03.678 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T22:09:03.680 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.717 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.065) 0:01:34.574 ******* 2020-11-16T22:09:03.744 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:03.746 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzBwCxsqK6C0g1BnhFM4B7FEnb7YjQWMeF'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.125131", "end": "2020-11-16 22:09:03.734371", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzBwCxsqK6C0g1BnhFM4B7FEnb7YjQWMeF"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:02.609240", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destro 2020-11-16T22:09:03.747 INFO:teuthology.task.ansible.out:yed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:03.784 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.066) 0:01:34.641 ******* 2020-11-16T22:09:03.791 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.803 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T22:09:03.804 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.842 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.057) 0:01:34.699 ******* 2020-11-16T22:09:03.868 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.907 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.065) 0:01:34.764 ******* 2020-11-16T22:09:03.934 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T22:09:03.936 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:03.976 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:03 +0000 (0:00:00.069) 0:01:34.833 ******* 2020-11-16T22:09:04.003 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:09:04.005 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.041 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:34.899 ******* 2020-11-16T22:09:04.067 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:09:04.069 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.105 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.064) 0:01:34.963 ******* 2020-11-16T22:09:04.112 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:04.113 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bzxxrEmDDPLudK7TmvLSmMhqjkYj8YNUq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.133202", "end": "2020-11-16 22:09:04.096902", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bzxxrEmDDPLudK7TmvLSmMhqjkYj8YNUq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:02.963700", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:04.114 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.126 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:09:04.127 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.164 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.058) 0:01:35.022 ******* 2020-11-16T22:09:04.191 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:09:04.193 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.230 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.087 ******* 2020-11-16T22:09:04.257 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:09:04.258 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.295 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.153 ******* 2020-11-16T22:09:04.322 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T22:09:04.324 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.361 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.218 ******* 2020-11-16T22:09:04.387 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T22:09:04.389 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.425 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.064) 0:01:35.283 ******* 2020-11-16T22:09:04.453 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:09:04.454 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.491 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.349 ******* 2020-11-16T22:09:04.522 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:09:04.523 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.067) 0:01:35.417 ******* 2020-11-16T22:09:04.586 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:09:04.587 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.624 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.482 ******* 2020-11-16T22:09:04.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T22:09:04.652 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.689 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.064) 0:01:35.546 ******* 2020-11-16T22:09:04.715 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T22:09:04.717 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.754 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.064) 0:01:35.611 ******* 2020-11-16T22:09:04.779 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T22:09:04.781 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.818 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.064) 0:01:35.676 ******* 2020-11-16T22:09:04.845 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T22:09:04.846 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:04.883 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:04 +0000 (0:00:00.065) 0:01:35.741 ******* 2020-11-16T22:09:05.004 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:05.006 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzPiUYX22dcuwatROfaSjqcEb1seHi8e22'], 'uuids': ['68af2f52-7eae-47f9-8648-3515b6e4078e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.121709", "end": "2020-11-16 22:09:04.984400", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzPiUYX22dcuwatROfaSjqcEb1seHi8e22"], "labels": [], "masters": [], "uuids": ["68af2f52-7eae-47f9-8648-3515b6e4078e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:03.862691", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:05.049 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:05 +0000 (0:00:00.165) 0:01:35.906 ******* 2020-11-16T22:09:05.067 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T22:09:05.068 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T22:09:05.107 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:05 +0000 (0:00:00.058) 0:01:35.964 ******* 2020-11-16T22:09:05.114 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:05.115 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.381 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzp5A07KGUP16v68T3UGF3X7a0sNxIVqJv'], 'uuids': ['e7494e78-395f-42d4-832e-b855f0f14f49']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.157666", "end": "2020-11-16 22:09:05.367022", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzp5A07KGUP16v68T3UGF3X7a0sNxIVqJv"], "labels": [], "masters": [], "uuids": ["e7494e78-395f-42d4-832e-b855f0f14f49"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.209356", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152982 s, 754 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152982 s, 754 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:05.382 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:05.383 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4b3pp53uQhqkn2aHTtLLWpcgCmeuY4W1iE'], 'uuids': ['d82dbc76-30a6-4ff8-9e51-1730ab7ec2a4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.141855", "end": "2020-11-16 22:09:05.373014", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4b3pp53uQhqkn2aHTtLLWpcgCmeuY4W1iE"], "labels": [], "masters": [], "uuids": ["d82dbc76-30a6-4ff8-9e51-1730ab7ec2a4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:04.231159", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:05.393 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:05.395 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.427 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:05 +0000 (0:00:00.320) 0:01:36.285 ******* 2020-11-16T22:09:05.435 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.468 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.573 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T22:09:05.574 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T22:09:05.613 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:05 +0000 (0:00:00.185) 0:01:36.470 ******* 2020-11-16T22:09:05.640 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:05 +0000 (0:00:00.066) 0:01:36.537 ******* 2020-11-16T22:09:05.687 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:05.687 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzSwuMah559AC1VOGJowLZK5CXv0rsYeqJ'], 'uuids': ['5b349b2a-424f-4c3c-8649-47ee57919e3d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.159306", "end": "2020-11-16 22:09:05.664578", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzSwuMah559AC1VOGJowLZK5CXv0rsYeqJ"], "labels": [], "masters": [], "uuids": ["5b349b2a-424f-4c3c-8649-47ee57919e3d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.505272", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15487 s, 745 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15487 s, 745 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:05.689 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTPG', 'wwn-0x5000c5009293f2ac'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTPG-part1', 'wwn-0x5000c5009293f2ac-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293f2ac', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XTPG', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTPG", "wwn-0x5000c5009293f2ac"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTPG-part1", "wwn-0x5000c5009293f2ac-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTPG", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293f2ac"}}, "skip_reason": 2020-11-16T22:09:05.690 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T22:09:05.752 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bc9aAlg8IdPe1rAXSt5Vh2B2YpMfOArWG'], 'uuids': ['985affca-6095-47b9-8db4-c24983452fa9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.158211", "end": "2020-11-16 22:09:05.742126", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bc9aAlg8IdPe1rAXSt5Vh2B2YpMfOArWG"], "labels": [], "masters": [], "uuids": ["985affca-6095-47b9-8db4-c24983452fa9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.583915", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153414 s, 752 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153414 s, 752 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:05.753 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.764 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.859 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-0cB3dv-iyfD-iMtx-1X5M-9436-MHFy-93d0Cm', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300KY400BGN', 'nvme-nvme.8086-435646543632333330304b5934303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.053984", "end": "2020-11-16 22:09:05.840201", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-0cB3dv-iyfD-iMtx-1X5M-9436-MHFy-93d0Cm", "nvme-INTEL_SSDPEDMD400G4_CVFT623300KY400BGN", "nvme-nvme.8086-435646543632333330304b5934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.786217", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0491537 s, 2.3 GB/s", "std 2020-11-16T22:09:05.860 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0491537 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:05.871 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.079 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bSzDKkzand7pjlchx6i51uisuIFj00hAm'], 'uuids': ['76b305be-5de6-4d90-a389-0a6b4da2ed03']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.160868", "end": "2020-11-16 22:09:06.032327", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bSzDKkzand7pjlchx6i51uisuIFj00hAm"], "labels": [], "masters": [], "uuids": ["76b305be-5de6-4d90-a389-0a6b4da2ed03"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.871459", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156193 s, 738 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156193 s, 738 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': 2020-11-16T22:09:06.081 INFO:teuthology.task.ansible.out: [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091dfbebc', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QGMP', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.137 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzW8UunQd2qX1cTp5CSgzvUcCF0oZGQ3M2'], 'uuids': ['acad91eb-c483-42e7-b00d-ba74c739ad06']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.145962", "end": "2020-11-16 22:09:06.123350", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzW8UunQd2qX1cTp5CSgzvUcCF0oZGQ3M2"], "labels": [], "masters": [], "uuids": ["acad91eb-c483-42e7-b00d-ba74c739ad06"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:05.977388", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.143456 s, 804 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.143456 s, 804 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:06.148 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.149 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.160 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-16T22:09:06.162 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T22:09:06.200 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:06 +0000 (0:00:00.520) 0:01:37.057 ******* 2020-11-16T22:09:06.403 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-eJP9xu-83Nf-T9od-Ln1a-rFrE-msBn-KMAgXO', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.053557", "end": "2020-11-16 22:09:06.208312", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-eJP9xu-83Nf-T9od-Ln1a-rFrE-msBn-KMAgXO", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.154755", "stderr": "110+0 records in\n110+0 2020-11-16T22:09:06.405 INFO:teuthology.task.ansible.out: records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0488296 s, 2.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0488296 s, 2.4 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": nul 2020-11-16T22:09:06.407 INFO:teuthology.task.ansible.out:l, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.439 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzBwCxsqK6C0g1BnhFM4B7FEnb7YjQWMeF'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.164746", "end": "2020-11-16 22:09:06.424887", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzBwCxsqK6C0g1BnhFM4B7FEnb7YjQWMeF"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.260141", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.160013 s, 721 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.160013 s, 721 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:06.450 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:06.493 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bXp641cKBwCBjqYWXzJ2cLZxGkGWfNYT2'], 'uuids': ['97c9a7ab-4cb8-4db6-94ee-4af9ae9cac78']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.148646", "end": "2020-11-16 22:09:06.489350", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bXp641cKBwCBjqYWXzJ2cLZxGkGWfNYT2"], "labels": [], "masters": [], "uuids": ["97c9a7ab-4cb8-4db6-94ee-4af9ae9cac78"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.340704", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144247 s, 800 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144247 s, 800 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:06.695 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d04402 2020-11-16T22:09:06.697 INFO:teuthology.task.ansible.out:19b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T22:09:06.702 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:06 +0000 (0:00:00.502) 0:01:37.559 ******* 2020-11-16T22:09:06.730 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:06.731 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzPiUYX22dcuwatROfaSjqcEb1seHi8e22'], 'uuids': ['68af2f52-7eae-47f9-8648-3515b6e4078e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.159176", "end": "2020-11-16 22:09:06.712887", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AUa4dvFFATeJW5BrJmJxNehSEFxNYgAzPiUYX22dcuwatROfaSjqcEb1seHi8e22"], "labels": [], "masters": [], "uuids": ["68af2f52-7eae-47f9-8648-3515b6e4078e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.553711", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15437 s, 747 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15437 s, 747 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:06.781 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:06 +0000 (0:00:00.079) 0:01:37.639 ******* 2020-11-16T22:09:06.800 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bzxxrEmDDPLudK7TmvLSmMhqjkYj8YNUq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.161406", "end": "2020-11-16 22:09:06.781078", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4bzxxrEmDDPLudK7TmvLSmMhqjkYj8YNUq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.619672", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156718 s, 736 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156718 s, 736 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:06.802 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.073 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T22:09:07.075 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.154077", "end": "2020-11-16 22:09:07.057385", "rc": 0, "start": "2020-11-16 22:09:06.903308", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:07.106 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.324) 0:01:37.964 ******* 2020-11-16T22:09:07.114 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:07.114 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4b3pp53uQhqkn2aHTtLLWpcgCmeuY4W1iE'], 'uuids': ['d82dbc76-30a6-4ff8-9e51-1730ab7ec2a4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.159623", "end": "2020-11-16 22:09:07.070419", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-svUJNsdaduBgzlwCXas8tq7Yc5zT6L4b3pp53uQhqkn2aHTtLLWpcgCmeuY4W1iE"], "labels": [], "masters": [], "uuids": ["d82dbc76-30a6-4ff8-9e51-1730ab7ec2a4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:06.910796", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15502 s, 744 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15502 s, 744 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:07.159 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.053) 0:01:38.017 ******* 2020-11-16T22:09:07.176 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T22:09:07.178 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T22:09:07.216 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.056) 0:01:38.074 ******* 2020-11-16T22:09:07.285 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T22:09:07.287 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049149", "end": "2020-11-16 22:09:07.277799", "rc": 0, "start": "2020-11-16 22:09:07.228650", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:07.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.103) 0:01:38.177 ******* 2020-11-16T22:09:07.346 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:09:07.381 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.061) 0:01:38.239 ******* 2020-11-16T22:09:07.397 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T22:09:07.399 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:09:07.437 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.055) 0:01:38.294 ******* 2020-11-16T22:09:07.443 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:09:07.444 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.119790", "end": "2020-11-16 22:09:07.403651", "rc": 0, "start": "2020-11-16 22:09:07.283861", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:07.478 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.040) 0:01:38.335 ******* 2020-11-16T22:09:07.509 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.031) 0:01:38.367 ******* 2020-11-16T22:09:07.516 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T22:09:07.518 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.561 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.051) 0:01:38.418 ******* 2020-11-16T22:09:07.567 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:09:07.569 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.601 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.039) 0:01:38.458 ******* 2020-11-16T22:09:07.606 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T22:09:07.607 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.647 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.046) 0:01:38.504 ******* 2020-11-16T22:09:07.653 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:09:07.655 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.690 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.042) 0:01:38.547 ******* 2020-11-16T22:09:07.696 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T22:09:07.698 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for physical volumes] *********************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.067439", "end": "2020-11-16 22:09:07.668133", "rc": 0, "start": "2020-11-16 22:09:07.600694", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:07.737 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.047) 0:01:38.595 ******* 2020-11-16T22:09:07.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.030) 0:01:38.625 ******* 2020-11-16T22:09:07.775 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:09:07.776 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.814 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.046) 0:01:38.672 ******* 2020-11-16T22:09:07.845 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.030) 0:01:38.702 ******* 2020-11-16T22:09:07.852 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:09:07.853 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:07.895 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.049) 0:01:38.752 ******* 2020-11-16T22:09:07.933 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.037) 0:01:38.790 ******* 2020-11-16T22:09:07.939 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:07.940 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T22:09:07.979 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:07 +0000 (0:00:00.046) 0:01:38.837 ******* 2020-11-16T22:09:07.985 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:09:07.987 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:09:08.027 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.047) 0:01:38.884 ******* 2020-11-16T22:09:08.059 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.032) 0:01:38.917 ******* 2020-11-16T22:09:08.065 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:09:08.067 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram14) => {"ansible_loop_var": "item", "changed": false, "item": "ram14", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=ram15) => {"ansible_loop_var": "item", "changed": false, "item": "ram15", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.067 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram12) => {"ansible_loop_var": "item", "changed": false, "item": "ram12", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.067 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.068 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T22:09:08.068 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram13) => {"ansible_loop_var": "item", "changed": false, "item": "ram13", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.068 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram10) => {"ansible_loop_var": "item", "changed": false, "item": "ram10", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.068 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram11) => {"ansible_loop_var": "item", "changed": false, "item": "ram11", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.069 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram8) => {"ansible_loop_var": "item", "changed": false, "item": "ram8", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.104 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.044) 0:01:38.961 ******* 2020-11-16T22:09:08.109 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram9) => {"ansible_loop_var": "item", "changed": false, "item": "ram9", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.110 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram4) => {"ansible_loop_var": "item", "changed": false, "item": "ram4", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.111 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram5) => {"ansible_loop_var": "item", "changed": false, "item": "ram5", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.112 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi185.front.sepia.ceph.com] => (item={'block_used': 13783676, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311432, 'size_available': 926971625472, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114489, 'block_size': 4096, 'inode_available': 60940487}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311432, "block_size": 4096, "block_total": 240095108, "block_used": 13783676, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940487, "inode_total": 61054976, "inode_used": 114489, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971625472, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=ram6) => {"ansible_loop_var": "item", "changed": false, "item": "ram6", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.114 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=ram7) => {"ansible_loop_var": "item", "changed": false, "item": "ram7", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=ram0) => {"ansible_loop_var": "item", "changed": false, "item": "ram0", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.114 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram1) => {"ansible_loop_var": "item", "changed": false, "item": "ram1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.147 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.043) 0:01:39.005 ******* 2020-11-16T22:09:08.153 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=ram2) => {"ansible_loop_var": "item", "changed": false, "item": "ram2", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.154 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:09:08.154 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.154 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=ram3) => {"ansible_loop_var": "item", "changed": false, "item": "ram3", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.155 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.156 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.191 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.044) 0:01:39.049 ******* 2020-11-16T22:09:08.198 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.198 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.200 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi062.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.200 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.201 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.201 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.235 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.043) 0:01:39.092 ******* 2020-11-16T22:09:08.241 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.243 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** ok: [smithi062.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T22:09:08.288 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.053) 0:01:39.145 ******* 2020-11-16T22:09:08.319 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.030) 0:01:39.176 ******* 2020-11-16T22:09:08.325 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:08.326 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.361 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.042) 0:01:39.218 ******* 2020-11-16T22:09:08.387 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:08.389 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.421 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.060) 0:01:39.278 ******* 2020-11-16T22:09:08.438 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:09:08.440 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:09:08.472 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.051) 0:01:39.330 ******* 2020-11-16T22:09:08.478 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:09:08.479 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi062.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003762", "end": "2020-11-16 22:09:08.441466", "rc": 0, "start": "2020-11-16 22:09:08.437704", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T22:09:08.527 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.054) 0:01:39.384 ******* 2020-11-16T22:09:08.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.031) 0:01:39.416 ******* 2020-11-16T22:09:08.564 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:09:08.566 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'block_used': 13783674, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226311434, 'size_available': 926971633664, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 114488, 'block_size': 4096, 'inode_available': 60940488}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226311434, "block_size": 4096, "block_total": 240095108, "block_used": 13783674, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60940488, "inode_total": 61054976, "inode_used": 114488, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 926971633664, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.566 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.609 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.050) 0:01:39.467 ******* 2020-11-16T22:09:08.640 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.030) 0:01:39.498 ******* 2020-11-16T22:09:08.646 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:09:08.648 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.686 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.046) 0:01:39.544 ******* 2020-11-16T22:09:08.718 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.031) 0:01:39.575 ******* 2020-11-16T22:09:08.724 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:09:08.726 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.765 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.047) 0:01:39.623 ******* 2020-11-16T22:09:08.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.031) 0:01:39.654 ******* 2020-11-16T22:09:08.803 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:09:08.805 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.041) 0:01:39.696 ******* 2020-11-16T22:09:08.845 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:08.847 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.880 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.041) 0:01:39.738 ******* 2020-11-16T22:09:08.907 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:08.909 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:08.941 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.060) 0:01:39.798 ******* 2020-11-16T22:09:08.968 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:09.002 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:08 +0000 (0:00:00.060) 0:01:39.859 ******* 2020-11-16T22:09:09.029 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:09.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.066) 0:01:39.925 ******* 2020-11-16T22:09:09.096 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:09:09.097 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:09.129 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.061) 0:01:39.987 ******* 2020-11-16T22:09:09.399 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T22:09:09.400 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:09:09.434 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.304) 0:01:40.291 ******* 2020-11-16T22:09:09.441 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:09:09.477 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.043) 0:01:40.334 ******* 2020-11-16T22:09:09.848 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T22:09:09.850 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:09:09.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.411) 0:01:40.746 ******* 2020-11-16T22:09:09.917 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T22:09:09.919 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:09.957 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:09 +0000 (0:00:00.067) 0:01:40.814 ******* 2020-11-16T22:09:09.974 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:09:09.975 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:09.996 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:09.997 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi062.front.sepia.ceph.com] => (item={'block_used': 14062212, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032896, 'size_available': 925830742016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119631, 'block_size': 4096, 'inode_available': 60935345}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032896, "block_size": 4096, "block_total": 240095108, "block_used": 14062212, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935345, "inode_total": 61054976, "inode_used": 119631, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830742016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T22:09:10.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.079) 0:01:40.894 ******* 2020-11-16T22:09:10.209 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:09:10.211 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.221 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.242 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T22:09:10.244 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.028671", "end": "2020-11-16 22:09:10.220470", "rc": 0, "start": "2020-11-16 22:09:10.191799", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:10.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.245) 0:01:41.140 ******* 2020-11-16T22:09:10.320 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T22:09:10.321 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'block_used': 14062212, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032896, 'size_available': 925830742016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119631, 'block_size': 4096, 'inode_available': 60935345}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032896, "block_size": 4096, "block_total": 240095108, "block_used": 14062212, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935345, "inode_total": 61054976, "inode_used": 119631, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830742016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.359 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.076) 0:01:41.217 ******* 2020-11-16T22:09:10.386 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T22:09:10.428 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.069) 0:01:41.286 ******* 2020-11-16T22:09:10.446 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:09:10.448 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.468 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.479 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:10.481 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.491 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.503 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.514 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.515 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.526 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.537 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.549 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.560 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.571 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.572 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.583 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.595 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.606 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.617 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.628 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:10.681 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:09:10.682 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.702 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.920 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.922 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:10.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.529) 0:01:41.815 ******* 2020-11-16T22:09:10.991 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:10 +0000 (0:00:00.032) 0:01:41.848 ******* 2020-11-16T22:09:11.081 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:09:11.083 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:09:11.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.125) 0:01:41.973 ******* 2020-11-16T22:09:11.124 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:09:11.158 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.042) 0:01:42.016 ******* 2020-11-16T22:09:11.259 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:09:11.260 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.301 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.375 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.418 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.482 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.525 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.590 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.601 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:11.638 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.479) 0:01:42.496 ******* 2020-11-16T22:09:11.646 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:11.647 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:11.684 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.046) 0:01:42.542 ******* 2020-11-16T22:09:11.764 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T22:09:11.765 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:11.798 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.113) 0:01:42.656 ******* 2020-11-16T22:09:11.815 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:11.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.050) 0:01:42.706 ******* 2020-11-16T22:09:11.855 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:09:11.857 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:09:11.893 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.043) 0:01:42.750 ******* 2020-11-16T22:09:11.900 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:09:11.940 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:11 +0000 (0:00:00.047) 0:01:42.797 ******* 2020-11-16T22:09:12.361 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:09:12.362 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true} changed: [smithi185.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:09:12.397 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:12 +0000 (0:00:00.457) 0:01:43.254 ******* 2020-11-16T22:09:12.429 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:12 +0000 (0:00:00.032) 0:01:43.286 ******* 2020-11-16T22:09:12.436 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:12.436 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGuNWo50bjk5snPk1ltAl0KOcnQfgJ88UL'], 'uuids': ['3ac7774a-728c-402b-a7b4-e380aaf6cdb1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.618332", "end": "2020-11-16 22:09:12.388984", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGuNWo50bjk5snPk1ltAl0KOcnQfgJ88UL"], "labels": [], "masters": [], "uuids": ["3ac7774a-728c-402b-a7b4-e380aaf6cdb1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:10.770652", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:12.813 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:09:12.814 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:09:12.847 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:12 +0000 (0:00:00.417) 0:01:43.704 ******* 2020-11-16T22:09:12.987 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:09:12.989 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:09:13.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:13 +0000 (0:00:00.174) 0:01:43.879 ******* 2020-11-16T22:09:13.152 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:09:13.153 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:13.185 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:13 +0000 (0:00:00.163) 0:01:44.042 ******* 2020-11-16T22:09:13.502 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:09:13.504 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:09:13.535 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:13 +0000 (0:00:00.350) 0:01:44.393 ******* 2020-11-16T22:09:13.739 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGd8j183bfKCjZy8gWQoThI8YwTAvwGX0q'], 'uuids': ['33a055be-a2f7-41cc-9613-d4dda2be5dad']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.154141", "end": "2020-11-16 22:09:13.711551", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGd8j183bfKCjZy8gWQoThI8YwTAvwGX0q"], "labels": [], "masters": [], "uuids": ["33a055be-a2f7-41cc-9613-d4dda2be5dad"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:12.557410", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-11-16T22:09:13.740 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-11-16T22:09:13.812 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:09:13.814 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:09:13.851 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:13 +0000 (0:00:00.315) 0:01:44.708 ******* 2020-11-16T22:09:13.858 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:09:13.860 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T22:09:13.895 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:13 +0000 (0:00:00.044) 0:01:44.753 ******* 2020-11-16T22:09:13.997 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:09:13.999 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:09:14.032 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:14 +0000 (0:00:00.137) 0:01:44.890 ******* 2020-11-16T22:09:14.164 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:09:14.166 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:14.198 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:14 +0000 (0:00:00.165) 0:01:45.055 ******* 2020-11-16T22:09:14.517 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:09:14.518 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:09:14.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:14 +0000 (0:00:00.352) 0:01:45.408 ******* 2020-11-16T22:09:14.786 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:09:14.788 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:14.820 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:14 +0000 (0:00:00.269) 0:01:45.677 ******* 2020-11-16T22:09:14.869 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:09:14.871 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T22:09:14.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:14 +0000 (0:00:00.083) 0:01:45.761 ******* 2020-11-16T22:09:15.003 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T22:09:15.005 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.046511", "end": "2020-11-16 22:09:14.993101", "rc": 0, "start": "2020-11-16 22:09:14.946590", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T22:09:15.040 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:15 +0000 (0:00:00.135) 0:01:45.897 ******* 2020-11-16T22:09:15.046 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:15.047 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdG0lm4RLQeYi4sdCV9pjF0fz6E65cvY2CS'], 'uuids': ['9ce6d4c9-df03-46b5-bc0e-2a0d43fe5a38']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.129433", "end": "2020-11-16 22:09:15.004910", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdG0lm4RLQeYi4sdCV9pjF0fz6E65cvY2CS"], "labels": [], "masters": [], "uuids": ["9ce6d4c9-df03-46b5-bc0e-2a0d43fe5a38"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:13.875477", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:15.069 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:09:15.071 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:15.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:15 +0000 (0:00:00.063) 0:01:45.961 ******* 2020-11-16T22:09:15.130 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:09:15.132 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:15.163 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:15 +0000 (0:00:00.060) 0:01:46.021 ******* 2020-11-16T22:09:15.806 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:09:15.808 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:15.839 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:15 +0000 (0:00:00.676) 0:01:46.697 ******* 2020-11-16T22:09:16.003 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T22:09:16.004 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.039492", "end": "2020-11-16 22:09:16.001388", "rc": 0, "start": "2020-11-16 22:09:15.961896", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T22:09:16.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.197) 0:01:46.894 ******* 2020-11-16T22:09:16.044 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:09:16.045 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:16.079 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.042) 0:01:46.936 ******* 2020-11-16T22:09:16.085 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:09:16.086 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.119 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.040) 0:01:46.977 ******* 2020-11-16T22:09:16.126 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T22:09:16.128 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.160 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.040) 0:01:47.018 ******* 2020-11-16T22:09:16.166 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:09:16.167 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.201 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.040) 0:01:47.058 ******* 2020-11-16T22:09:16.310 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:16.312 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGih6QN4ahw0D2Vw3rQHENCXFJZl2IuVhf'], 'uuids': ['1f5ed3dd-0b44-4610-af20-407a4e127eb2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.118679", "end": "2020-11-16 22:09:16.288573", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGih6QN4ahw0D2Vw3rQHENCXFJZl2IuVhf"], "labels": [], "masters": [], "uuids": ["1f5ed3dd-0b44-4610-af20-407a4e127eb2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:15.169894", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:16.467 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:09:16.469 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:09:16.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.300) 0:01:47.359 ******* 2020-11-16T22:09:16.528 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:09:16.530 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.562 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.060) 0:01:47.419 ******* 2020-11-16T22:09:16.755 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:09:16.756 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ActiveEnterTimestampMonotonic": "17586465", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket -.mount sysinit.target systemd-tmpfiles-setup.service basic.target system.slice network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 22:03:42 UTC", "AssertTimestampMonotonic": "17292141", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:42 UTC", "ConditionTimestampMonotonic": "17292140", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-16T22:09:16.758 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "978", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:43 UTC", "ExecMainStartTimestampMonotonic": "17586448", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:42 UTC", "InactiveExitTimestampMonotonic": "17292846", "InvocationID": "56896084bcd04b0f95d6dbc37c1f10e9", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limit 2020-11-16T22:09:16.759 INFO:teuthology.task.ansible.out:RTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "978", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:43 UTC", "Sta 2020-11-16T22:09:16.761 INFO:teuthology.task.ansible.out:teChangeTimestampMonotonic": "17586465", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:43 UTC", "WatchdogTimestampMonotonic": "17586464", "WatchdogUSec": "0"}} 2020-11-16T22:09:16.790 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.228) 0:01:47.648 ******* 2020-11-16T22:09:16.816 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:16.818 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:09:16.850 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.059) 0:01:47.707 ******* 2020-11-16T22:09:16.887 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:16.888 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.926 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.075) 0:01:47.783 ******* 2020-11-16T22:09:16.952 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:09:16.954 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:16.986 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:16 +0000 (0:00:00.059) 0:01:47.843 ******* 2020-11-16T22:09:17.054 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:09:17.056 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:17.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:17 +0000 (0:00:00.103) 0:01:47.946 ******* 2020-11-16T22:09:17.117 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T22:09:17.119 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:17.150 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:17 +0000 (0:00:00.061) 0:01:48.007 ******* 2020-11-16T22:09:17.457 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:09:17.459 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:09:17.490 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:17 +0000 (0:00:00.340) 0:01:48.348 ******* 2020-11-16T22:09:17.517 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:09:17.518 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:17.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:17 +0000 (0:00:00.059) 0:01:48.407 ******* 2020-11-16T22:09:17.619 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T22:09:17.620 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGZu3jBMS8ZNddDTXHDM9GnVMzTba4fqrT'], 'uuids': ['c03f347c-2bfb-4fbf-8856-011fa0a6bef5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.134018", "end": "2020-11-16 22:09:17.589270", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGZu3jBMS8ZNddDTXHDM9GnVMzTba4fqrT"], "labels": [], "masters": [], "uuids": ["c03f347c-2bfb-4fbf-8856-011fa0a6bef5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:16.455252", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:17.822 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => 2020-11-16T22:09:18.028 INFO:teuthology.task.ansible.out:(item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W5SV', 'wwn-0x5000c50087b99812'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W5SV-part1', 'wwn-0x5000c50087b99812-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50087b99812', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4W5SV', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W5SV", "wwn-0x5000c50087b99812"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W5SV-part1", "wwn-0x5000c50087b99812-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4W5SV", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50087b99812"}}, "skip_reason": "Conditional result was False" 2020-11-16T22:09:18.231 INFO:teuthology.task.ansible.out:} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': No 2020-11-16T22:09:18.435 INFO:teuthology.task.ansible.out:ne, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 2020-11-16T22:09:18.437 INFO:teuthology.task.ansible.out:'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) 2020-11-16T22:09:18.639 INFO:teuthology.task.ansible.out: => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure ntpd is running.] *********************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ActiveEnterTimestampMonotonic": "17309572", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket systemd-tmpfiles-setup.service system.slice network.target -.mount sysinit.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTim 2020-11-16T22:09:18.843 INFO:teuthology.task.ansible.out:estamp": "Mon 2020-11-16 22:03:30 UTC", "AssertTimestampMonotonic": "17132069", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 22:03:30 UTC", "ConditionTimestampMonotonic": "17132069", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "969", "ExecMainStartTimestamp": "Mon 2020-11-16 22:03:31 UTC", "ExecMainStartTimestampMonotonic": "17309556", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": " 2020-11-16T22:09:19.047 INFO:teuthology.task.ansible.out:0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 22:03:30 UTC", "InactiveExitTimestampMonotonic": "17132811", "InvocationID": "0e7da6886a4b4b39b1f30c6e6ae44f93", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "969", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", 2020-11-16T22:09:19.049 INFO:teuthology.task.ansible.out: "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "in 2020-11-16T22:09:19.050 INFO:teuthology.task.ansible.out:herit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 22:03:31 UTC", "StateChangeTimestampMonotonic": "17309572", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNS 2020-11-16T22:09:19.051 INFO:teuthology.task.ansible.out:ec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 22:03:31 UTC", "WatchdogTimestampMonotonic": "17309571", "WatchdogUSec": "0"}} TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:09:19.052 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:19.053 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lEtvfF-tgy6-i2E3-A43s-xSLL-rqgI-SYRwVl', 'nvme-INTEL_SSDPEDMD400G4_CVFT53440094400BGN', 'nvme-nvme.8086-43564654353334343030393434303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.132839", "end": "2020-11-16 22:09:18.974048", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-lEtvfF-tgy6-i2E3-A43s-xSLL-rqgI-SYRwVl", "nvme-INTEL_SSDPEDMD400G4_CVFT53440094400BGN", "nvme-nvme.8086-43564654353334343030393434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:17.841209", "stderr": "", "stde 2020-11-16T22:09:19.055 INFO:teuthology.task.ansible.out:rr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T22:09:19.107 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:19 +0000 (0:00:01.557) 0:01:49.965 ******* 2020-11-16T22:09:19.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:19 +0000 (0:00:00.031) 0:01:49.996 ******* 2020-11-16T22:09:19.174 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:19 +0000 (0:00:00.035) 0:01:50.031 ******* 2020-11-16T22:09:19.182 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:19.183 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:19.184 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:09:19.185 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:19.185 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:19.187 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:19.188 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:19.390 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:19 +0000 (0:00:00.049) 0:01:50.081 ******* skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [s 2020-11-16T22:09:19.392 INFO:teuthology.task.ansible.out:mithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was Fals 2020-11-16T22:09:19.393 INFO:teuthology.task.ansible.out:e"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:09:19 +0000 (0:00:00.047) 0:01:50.128 ******* skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': 2020-11-16T22:09:19.600 INFO:teuthology.task.ansible.out: '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False 2020-11-16T22:09:19.601 INFO:teuthology.task.ansible.out:"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [ 2020-11-16T22:09:19.603 INFO:teuthology.task.ansible.out:], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:09:19 +0000 (0:00:00.046) 0:01:50.175 ******* skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 2020-11-16T22:09:19.604 INFO:teuthology.task.ansible.out: 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Monday 16 November 2020 22:09:19 +0000 (0:00:00.063) 0:01:50.238 ******* TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} Monday 16 November 2020 22:09:19 +0000 (0:00:00.051) 0:01:50.290 ******* 2020-11-16T22:09:19.806 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} Monday 16 November 2020 22:09:19 +0000 (0:00:00.362) 0:01:50.653 ******* 2020-11-16T22:09:20.226 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:09:20.228 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"cache_update_time": 1605564484, "cache_updated": false, "changed": false} 2020-11-16T22:09:20.260 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:20 +0000 (0:00:00.464) 0:01:51.117 ******* 2020-11-16T22:09:20.286 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:09:20.288 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:20.319 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:20 +0000 (0:00:00.058) 0:01:51.176 ******* 2020-11-16T22:09:20.439 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:09:20.440 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:09:20.472 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:20 +0000 (0:00:00.152) 0:01:51.329 ******* 2020-11-16T22:09:20.767 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:09:20.769 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:09:20.804 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:20 +0000 (0:00:00.332) 0:01:51.661 ******* 2020-11-16T22:09:20.811 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:09:20.813 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:09:20.847 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:20 +0000 (0:00:00.043) 0:01:51.704 ******* 2020-11-16T22:09:21.289 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:21.291 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGuNWo50bjk5snPk1ltAl0KOcnQfgJ88UL'], 'uuids': ['3ac7774a-728c-402b-a7b4-e380aaf6cdb1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:01.815770", "end": "2020-11-16 22:09:21.263323", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGuNWo50bjk5snPk1ltAl0KOcnQfgJ88UL"], "labels": [], "masters": [], "uuids": ["3ac7774a-728c-402b-a7b4-e380aaf6cdb1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:19.447553", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.80322 s, 64.0 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.80322 s, 64.0 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:21.363 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:09:21.365 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:09:21.396 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:21 +0000 (0:00:00.549) 0:01:52.254 ******* 2020-11-16T22:09:21.423 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:09:21.425 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:09:21.456 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:21 +0000 (0:00:00.060) 0:01:52.314 ******* 2020-11-16T22:09:21.535 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:09:21.537 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:09:21.570 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:21 +0000 (0:00:00.113) 0:01:52.427 ******* 2020-11-16T22:09:21.598 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:09:21.600 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564244.3965917, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564244.3925915, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564244.3925915, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:09:21.634 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:21 +0000 (0:00:00.064) 0:01:52.491 ******* 2020-11-16T22:09:21.993 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:09:21.994 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:09:22.026 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.392) 0:01:52.884 ******* 2020-11-16T22:09:22.053 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:09:22.054 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:09:22.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.059) 0:01:52.944 ******* 2020-11-16T22:09:22.117 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.030) 0:01:52.974 ******* 2020-11-16T22:09:22.122 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:09:22.124 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:09:22.156 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.039) 0:01:53.014 ******* 2020-11-16T22:09:22.162 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:09:22.163 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.040) 0:01:53.054 ******* 2020-11-16T22:09:22.285 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:09:22.287 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564231.979818, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564231.975818, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564231.975818, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:09:22.325 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.128) 0:01:53.183 ******* 2020-11-16T22:09:22.331 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:09:22.333 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:09:22.366 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.041) 0:01:53.224 ******* 2020-11-16T22:09:22.447 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:09:22.449 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:09:22.481 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.114) 0:01:53.339 ******* 2020-11-16T22:09:22.499 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:09:22.501 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:09:22.533 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.051) 0:01:53.390 ******* 2020-11-16T22:09:22.540 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:09:22.541 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:09:22.573 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.040) 0:01:53.431 ******* 2020-11-16T22:09:22.581 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:09:22.582 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.615 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.041) 0:01:53.472 ******* 2020-11-16T22:09:22.621 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T22:09:22.623 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.658 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.043) 0:01:53.516 ******* 2020-11-16T22:09:22.666 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:09:22.668 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.668 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.704 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.045) 0:01:53.561 ******* 2020-11-16T22:09:22.742 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:09:22.743 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi185.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.754 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.756 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.789 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.085) 0:01:53.647 ******* 2020-11-16T22:09:22.795 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:09:22.796 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:09:22.829 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.039) 0:01:53.686 ******* 2020-11-16T22:09:22.836 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:22.837 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.872 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.043) 0:01:53.730 ******* 2020-11-16T22:09:22.899 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:09:22.900 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:22.931 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:22 +0000 (0:00:00.058) 0:01:53.789 ******* 2020-11-16T22:09:22.957 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:09:23.011 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} Monday 16 November 2020 22:09:22 +0000 (0:00:00.059) 0:01:53.848 ******* 2020-11-16T22:09:23.027 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.035) 0:01:53.884 ******* 2020-11-16T22:09:23.034 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T22:09:23.035 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.068 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.041) 0:01:53.925 ******* 2020-11-16T22:09:23.075 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:09:23.077 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.111 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.042) 0:01:53.968 ******* 2020-11-16T22:09:23.118 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:09:23.119 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.119 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.131 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:09:23.133 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.164 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.053) 0:01:54.021 ******* 2020-11-16T22:09:23.198 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.034) 0:01:54.056 ******* 2020-11-16T22:09:23.204 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:09:23.206 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.207 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.218 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.220 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:23.220 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.253 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.055) 0:01:54.111 ******* 2020-11-16T22:09:23.286 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.032) 0:01:54.143 ******* 2020-11-16T22:09:23.293 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:23.294 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:23.294 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGd8j183bfKCjZy8gWQoThI8YwTAvwGX0q'], 'uuids': ['33a055be-a2f7-41cc-9613-d4dda2be5dad']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:01.824303", "end": "2020-11-16 22:09:23.255603", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGd8j183bfKCjZy8gWQoThI8YwTAvwGX0q"], "labels": [], "masters": [], "uuids": ["33a055be-a2f7-41cc-9613-d4dda2be5dad"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:21.431300", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.8118 s, 63.7 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.8118 s, 63.7 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.045) 0:01:54.189 ******* 2020-11-16T22:09:23.337 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:09:23.339 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.040) 0:01:54.230 ******* 2020-11-16T22:09:23.378 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:09:23.379 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.411 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.038) 0:01:54.269 ******* 2020-11-16T22:09:23.417 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:23.418 INFO:teuthology.task.ansible.out:skipping: [smithi185.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.450 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.038) 0:01:54.308 ******* 2020-11-16T22:09:23.455 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:09:23.457 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.492 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.041) 0:01:54.349 ******* 2020-11-16T22:09:23.518 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:09:23.520 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.064) 0:01:54.414 ******* 2020-11-16T22:09:23.583 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:09:23.585 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.618 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.061) 0:01:54.475 ******* 2020-11-16T22:09:23.624 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:23.626 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.006723", "end": "2020-11-16 22:09:23.580442", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:09:23.573719", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.636 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:23.638 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.671 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.052) 0:01:54.528 ******* 2020-11-16T22:09:23.698 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:09:23.700 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.734 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.062) 0:01:54.591 ******* 2020-11-16T22:09:23.741 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:23.742 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006049", "end": "2020-11-16 22:09:23.704056", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:09:23.698007", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.753 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:23.754 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:23.786 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.052) 0:01:54.643 ******* 2020-11-16T22:09:23.833 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:23.835 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005575", "end": "2020-11-16 22:09:23.818279", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:09:23.812704", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.927 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011123", "end": "2020-11-16 22:09:23.920268", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:09:23.909145", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.959 INFO:teuthology.task.ansible.out:ok: [smithi185.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.020925", "end": "2020-11-16 22:09:23.944043", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:09:23.923118", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:23.995 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:23 +0000 (0:00:00.209) 0:01:54.852 ******* 2020-11-16T22:09:24.022 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:09:24.058 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:24 +0000 (0:00:00.063) 0:01:54.916 ******* 2020-11-16T22:09:24.064 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:24.066 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006097", "end": "2020-11-16 22:09:24.035423", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:09:24.029326", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:24.159 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005488", "end": "2020-11-16 22:09:24.152394", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:09:24.146906", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:24.180 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:09:24.182 INFO:teuthology.task.ansible.out:changed: [smithi185.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:09:24.296 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:24.298 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.029089", "end": "2020-11-16 22:09:24.285436", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:09:24.256347", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:24.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:24 +0000 (0:00:00.272) 0:01:55.188 ******* 2020-11-16T22:09:24.358 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:09:24.392 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:24 +0000 (0:00:00.061) 0:01:55.249 ******* 2020-11-16T22:09:24.525 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:09:24.527 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:09:25.270 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T22:09:25.272 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdG0lm4RLQeYi4sdCV9pjF0fz6E65cvY2CS'], 'uuids': ['9ce6d4c9-df03-46b5-bc0e-2a0d43fe5a38']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:01.828177", "end": "2020-11-16 22:09:25.247264", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdG0lm4RLQeYi4sdCV9pjF0fz6E65cvY2CS"], "labels": [], "masters": [], "uuids": ["9ce6d4c9-df03-46b5-bc0e-2a0d43fe5a38"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:23.419087", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.81537 s, 63.5 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.81537 s, 63.5 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:27.266 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGih6QN4ahw0D2Vw3rQHENCXFJZl2IuVhf'], 'uuids': ['1f5ed3dd-0b44-4610-af20-407a4e127eb2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:01.836123", "end": "2020-11-16 22:09:27.244529", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGih6QN4ahw0D2Vw3rQHENCXFJZl2IuVhf"], "labels": [], "masters": [], "uuids": ["1f5ed3dd-0b44-4610-af20-407a4e127eb2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:25.408406", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.82289 s, 63.3 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.82289 s, 63.3 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:29.249 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGZu3jBMS8ZNddDTXHDM9GnVMzTba4fqrT'], 'uuids': ['c03f347c-2bfb-4fbf-8856-011fa0a6bef5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:01.814114", "end": "2020-11-16 22:09:29.224261", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-FKUIs3WGyAfL82pOE8EhX1dz3xHb0QdGZu3jBMS8ZNddDTXHDM9GnVMzTba4fqrT"], "labels": [], "masters": [], "uuids": ["c03f347c-2bfb-4fbf-8856-011fa0a6bef5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:27.410147", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.80055 s, 64.1 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.80055 s, 64.1 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:29.250 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W5SV', 'wwn-0x5000c50087b99812'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4W5SV-part1', 'wwn-0x5000c50087b99812-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50087b99812', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4W5SV', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W5SV", "wwn-0x5000c50087b99812"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4W5SV-part1", "wwn-0x5000c50087b99812-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4W5SV", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50087b99812"}}, 2020-11-16T22:09:29.251 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.261 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.272 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.273 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.285 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.286 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.297 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.308 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.309 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:29.899 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-lEtvfF-tgy6-i2E3-A43s-xSLL-rqgI-SYRwVl', 'nvme-INTEL_SSDPEDMD400G4_CVFT53440094400BGN', 'nvme-nvme.8086-43564654353334343030393434303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.425982", "end": "2020-11-16 22:09:29.883999", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-lEtvfF-tgy6-i2E3-A43s-xSLL-rqgI-SYRwVl", "nvme-INTEL_SSDPEDMD400G4_CVFT53440094400BGN", "nvme-nvme.8086-43564654353334343030393434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 22:09:29.458017", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.413649 s, 279 MB/s", "stde 2020-11-16T22:09:29.901 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.413649 s, 279 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:29.964 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:29 +0000 (0:00:05.571) 0:02:00.821 ******* 2020-11-16T22:09:31.013 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T22:09:31.015 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.873088", "end": "2020-11-16 22:09:30.990439", "rc": 0, "start": "2020-11-16 22:09:30.117351", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:31.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:01.088) 0:02:01.909 ******* 2020-11-16T22:09:31.317 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T22:09:31.318 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.084855", "end": "2020-11-16 22:09:31.301634", "rc": 0, "start": "2020-11-16 22:09:31.216779", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:31.357 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.304) 0:02:02.214 ******* 2020-11-16T22:09:31.384 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T22:09:31.425 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.068) 0:02:02.282 ******* 2020-11-16T22:09:31.452 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T22:09:31.454 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:31.493 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.068) 0:02:02.351 ******* 2020-11-16T22:09:31.520 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T22:09:31.522 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:31.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.065) 0:02:02.416 ******* 2020-11-16T22:09:31.586 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:09:31.588 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:31.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.068) 0:02:02.484 ******* 2020-11-16T22:09:31.653 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T22:09:31.655 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:31.692 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.065) 0:02:02.549 ******* 2020-11-16T22:09:31.874 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T22:09:31.876 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T22:09:31.916 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.224) 0:02:02.774 ******* 2020-11-16T22:09:31.953 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T22:09:31.955 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'block_used': 14062212, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032896, 'size_available': 925830742016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119631, 'block_size': 4096, 'inode_available': 60935345}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032896, "block_size": 4096, "block_total": 240095108, "block_used": 14062212, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935345, "inode_total": 61054976, "inode_used": 119631, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925830742016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:31.993 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:31 +0000 (0:00:00.076) 0:02:02.851 ******* 2020-11-16T22:09:32.020 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T22:09:32.022 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.060 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.066) 0:02:02.917 ******* 2020-11-16T22:09:32.087 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:32.088 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.127 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.067) 0:02:02.984 ******* 2020-11-16T22:09:32.153 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:32.155 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.065) 0:02:03.050 ******* 2020-11-16T22:09:32.220 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T22:09:32.221 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.258 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.065) 0:02:03.116 ******* 2020-11-16T22:09:32.285 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T22:09:32.287 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.324 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.065) 0:02:03.181 ******* 2020-11-16T22:09:32.350 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T22:09:32.351 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:32.388 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.064) 0:02:03.246 ******* 2020-11-16T22:09:32.796 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T22:09:32.797 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T22:09:32.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:32 +0000 (0:00:00.446) 0:02:03.692 ******* 2020-11-16T22:09:33.278 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T22:09:33.280 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:33.683 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:34.075 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:34.450 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:34.863 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T22:09:34.903 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:34 +0000 (0:00:02.068) 0:02:05.760 ******* 2020-11-16T22:09:35.085 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T22:09:35.086 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T22:09:35.124 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:35 +0000 (0:00:00.220) 0:02:05.981 ******* 2020-11-16T22:09:35.318 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T22:09:35.320 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:35.486 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:35.641 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:35.797 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T22:09:35.808 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:35.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:35 +0000 (0:00:00.724) 0:02:06.706 ******* 2020-11-16T22:09:36.030 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T22:09:36.031 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:36.071 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:36 +0000 (0:00:00.221) 0:02:06.928 ******* 2020-11-16T22:09:36.108 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T22:09:36.110 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T22:09:36.150 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:36 +0000 (0:00:00.079) 0:02:07.007 ******* 2020-11-16T22:09:36.455 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T22:09:36.457 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true} 2020-11-16T22:09:36.495 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:36 +0000 (0:00:00.345) 0:02:07.352 ******* 2020-11-16T22:09:36.813 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-16T22:09:36.815 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T22:09:36.853 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:36 +0000 (0:00:00.358) 0:02:07.710 ******* 2020-11-16T22:09:37.046 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T22:09:37.048 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:09:37.085 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:37 +0000 (0:00:00.231) 0:02:07.942 ******* 2020-11-16T22:09:37.267 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T22:09:37.268 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T22:09:37.305 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:37 +0000 (0:00:00.220) 0:02:08.163 ******* 2020-11-16T22:09:37.785 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T22:09:37.787 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T22:09:37.824 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:37 +0000 (0:00:00.518) 0:02:08.681 ******* 2020-11-16T22:09:38.384 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-16T22:09:38.386 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T22:09:38.422 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:38 +0000 (0:00:00.598) 0:02:09.279 ******* 2020-11-16T22:09:39.486 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T22:09:39.487 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:09:39.525 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:39 +0000 (0:00:01.102) 0:02:10.382 ******* 2020-11-16T22:09:39.768 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T22:09:39.769 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.060062", "end": "2020-11-16 22:09:39.756862", "rc": 0, "start": "2020-11-16 22:09:39.696800", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T22:09:39.807 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:39 +0000 (0:00:00.282) 0:02:10.664 ******* 2020-11-16T22:09:39.834 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T22:09:39.835 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:39.872 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:39 +0000 (0:00:00.065) 0:02:10.730 ******* 2020-11-16T22:09:39.899 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T22:09:39.901 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:39.938 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:39 +0000 (0:00:00.065) 0:02:10.795 ******* 2020-11-16T22:09:40.997 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T22:09:40.998 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:09:41.036 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:41 +0000 (0:00:01.097) 0:02:11.893 ******* 2020-11-16T22:09:41.062 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T22:09:41.064 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:41.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:41 +0000 (0:00:00.066) 0:02:11.959 ******* 2020-11-16T22:09:41.564 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T22:09:41.566 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T22:09:41.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:41 +0000 (0:00:00.500) 0:02:12.460 ******* 2020-11-16T22:09:41.629 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T22:09:41.631 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:41.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:41 +0000 (0:00:00.065) 0:02:12.525 ******* 2020-11-16T22:09:41.984 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T22:09:41.986 INFO:teuthology.task.ansible.out:[WARNING]: The service (ntp) is actually an init script but the system is managed by systemd fatal: [smithi062.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-16T22:09:42.023 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:42 +0000 (0:00:00.354) 0:02:12.880 ******* 2020-11-16T22:09:42.050 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:42.052 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T22:09:42.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:42 +0000 (0:00:00.065) 0:02:12.946 ******* 2020-11-16T22:09:42.116 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:42.117 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:42.154 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:42 +0000 (0:00:00.065) 0:02:13.012 ******* 2020-11-16T22:09:42.190 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T22:09:42.192 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:42.227 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:42 +0000 (0:00:00.073) 0:02:13.085 ******* 2020-11-16T22:09:43.305 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T22:09:43.307 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"cache_update_time": 1605564485, "cache_updated": false, "changed": false} 2020-11-16T22:09:43.344 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:43 +0000 (0:00:01.116) 0:02:14.201 ******* 2020-11-16T22:09:43.371 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T22:09:43.372 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:43.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:43 +0000 (0:00:00.065) 0:02:14.266 ******* 2020-11-16T22:09:43.582 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T22:09:43.584 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T22:09:43.621 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:43 +0000 (0:00:00.212) 0:02:14.479 ******* 2020-11-16T22:09:44.082 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T22:09:44.084 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T22:09:44.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:44 +0000 (0:00:00.500) 0:02:14.979 ******* 2020-11-16T22:09:45.164 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T22:09:45.166 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T22:09:45.203 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:45 +0000 (0:00:01.081) 0:02:16.060 ******* 2020-11-16T22:09:45.869 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T22:09:45.871 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false} 2020-11-16T22:09:45.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:45 +0000 (0:00:00.705) 0:02:16.766 ******* 2020-11-16T22:09:45.936 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T22:09:45.938 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-16T22:09:45.975 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:45 +0000 (0:00:00.066) 0:02:16.832 ******* 2020-11-16T22:09:46.189 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T22:09:46.191 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605564249.6626954, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605564249.6586955, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605564249.6586955, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T22:09:46.230 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.255) 0:02:17.087 ******* 2020-11-16T22:09:46.411 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T22:09:46.413 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T22:09:46.450 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.220) 0:02:17.308 ******* 2020-11-16T22:09:46.477 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T22:09:46.479 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T22:09:46.517 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.066) 0:02:17.374 ******* 2020-11-16T22:09:46.543 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:46.581 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.064) 0:02:17.439 ******* 2020-11-16T22:09:46.753 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T22:09:46.754 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T22:09:46.792 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.210) 0:02:17.649 ******* 2020-11-16T22:09:46.962 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T22:09:46.964 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T22:09:47.001 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:46 +0000 (0:00:00.208) 0:02:17.858 ******* 2020-11-16T22:09:47.027 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.066 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.065) 0:02:17.923 ******* 2020-11-16T22:09:47.092 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T22:09:47.094 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.103 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.105 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.115 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.116 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.154 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.087) 0:02:18.011 ******* 2020-11-16T22:09:47.190 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T22:09:47.192 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi062.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.202 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.204 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.242 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.088) 0:02:18.099 ******* 2020-11-16T22:09:47.268 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T22:09:47.270 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.308 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.066) 0:02:18.166 ******* 2020-11-16T22:09:47.335 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T22:09:47.336 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.373 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.064) 0:02:18.230 ******* 2020-11-16T22:09:47.399 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T22:09:47.401 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.442 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.069) 0:02:18.300 ******* 2020-11-16T22:09:47.469 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T22:09:47.471 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.507 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.065) 0:02:18.365 ******* 2020-11-16T22:09:47.534 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T22:09:47.536 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.066) 0:02:18.431 ******* 2020-11-16T22:09:47.600 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:47.602 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.641 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.066) 0:02:18.498 ******* 2020-11-16T22:09:47.667 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T22:09:47.669 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.707 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.066) 0:02:18.564 ******* 2020-11-16T22:09:47.733 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T22:09:47.735 INFO:teuthology.task.ansible.out:skipping: [smithi062.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T22:09:47.772 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:47 +0000 (0:00:00.064) 0:02:18.629 ******* 2020-11-16T22:09:47.983 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T22:09:47.985 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.024260", "end": "2020-11-16 22:09:47.971168", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 22:09:47.946908", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:48.170 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.015888", "end": "2020-11-16 22:09:48.155873", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 22:09:48.139985", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:48.336 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.013975", "end": "2020-11-16 22:09:48.331130", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 22:09:48.317155", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:48.543 INFO:teuthology.task.ansible.out:ok: [smithi062.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.040209", "end": "2020-11-16 22:09:48.525845", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 22:09:48.485636", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T22:09:48.586 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:48 +0000 (0:00:00.814) 0:02:19.443 ******* 2020-11-16T22:09:48.613 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T22:09:48.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 22:09:48 +0000 (0:00:00.065) 0:02:19.509 ******* 2020-11-16T22:09:48.812 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T22:09:48.813 INFO:teuthology.task.ansible.out:changed: [smithi062.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T22:09:48.916 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-16T22:09:48.920 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-16T22:09:48.923 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-11-16T22:09:48.924 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi062.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 smithi096.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 smithi185.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 Monday 16 November 2020 22:09:48 +0000 (0:00:00.271) 0:02:19.781 ******* =============================================================================== common : Update apt cache ----------------------------------------------- 5.21s testnode : Update apt cache. -------------------------------------------- 4.23s testnode : Ensure packages are not present. ----------------------------- 2.42s testnode : Create logical volume(s) ------------------------------------- 2.07s Gathering Facts --------------------------------------------------------- 1.73s testnode : Install non aarch64 packages --------------------------------- 1.14s testnode : Install python-apt ------------------------------------------- 1.12s testnode : Install python-pip on apt based systems. --------------------- 1.12s testnode : Ensure perl-doc is installed on apt systems. ----------------- 1.10s testnode : Install ntp package on apt based systems. -------------------- 1.10s testnode : Remove all LVM data ------------------------------------------ 1.09s testnode : Install packages via pip ------------------------------------- 1.08s Gathering Facts --------------------------------------------------------- 0.82s testnode : List any leftover Ceph artifacts from previous jobs ---------- 0.81s testnode : Write /scratch_devs ------------------------------------------ 0.72s testnode : Purge resolvconf --------------------------------------------- 0.71s testnode : Upload CPAN config for root. --------------------------------- 0.60s comm 2020-11-16T22:09:48.926 INFO:teuthology.task.ansible.out:on : Set the hardware clock ----------------------------------------- 0.56s testnode : Zap all non-root disks --------------------------------------- 0.53s testnode : Upload weekly kernel-clean crontab. -------------------------- 0.52s 2020-11-16T22:09:49.243 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi185.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2020-11-16T22:09:49.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:09:49.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi185.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:09:49.327 INFO:teuthology.orchestra.run.smithi185:> true 2020-11-16T22:09:49.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:09:49.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:09:49.597 INFO:teuthology.orchestra.run.smithi096:> true 2020-11-16T22:09:49.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T22:09:49.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T22:09:49.845 INFO:teuthology.orchestra.run.smithi062:> true 2020-11-16T22:09:50.111 INFO:teuthology.run_tasks:Running task clock... 2020-11-16T22:09:50.123 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-16T22:09:50.124 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:09:50.124 INFO:teuthology.orchestra.run.smithi185:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:09:50.183 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:09:50.183 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Command line: ntpd -gq 2020-11-16T22:09:50.184 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: proto: precision = 0.123 usec (-23) 2020-11-16T22:09:50.185 INFO:teuthology.orchestra.run.smithi185.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:09:50.185 INFO:teuthology.orchestra.run.smithi185.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:09:50.185 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:09:50.185 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:09:50.185 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:09:50.186 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:09:50.186 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:09:50.186 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen normally on 3 enp3s0f1 172.21.15.185:123 2020-11-16T22:09:50.186 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen normally on 4 lo [::1]:123 2020-11-16T22:09:50.186 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e49%5]:123 2020-11-16T22:09:50.187 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:50 ntpd[6365]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:09:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:51.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:51.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:51.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:51.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:51 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:52.183 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:52.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:52.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:52.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:52.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:52.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:52.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:52.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:52 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:53.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:53.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:53.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:53.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:53.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:53.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:53.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:53.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:53 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:54.183 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:54.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:54.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:54.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:54.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:54.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:54.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:54.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:54 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:55.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:55.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:55.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:55.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:55.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:55.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:55.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:55.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:55 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:56 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:57.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:57.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:57.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:57.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:57.184 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:09:57.185 INFO:teuthology.orchestra.run.smithi185.stderr:16 Nov 22:09:57 ntpd[6365]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:09:57.185 INFO:teuthology.orchestra.run.smithi185.stdout:16 Nov 22:09:57 ntpd[6365]: ntpd: time slew +0.001695 s 2020-11-16T22:09:57.185 INFO:teuthology.orchestra.run.smithi185.stdout:ntpd: time slew +0.001695s 2020-11-16T22:09:57.295 INFO:teuthology.orchestra.run.smithi185.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:09:57.295 INFO:teuthology.orchestra.run.smithi185.stdout:============================================================================== 2020-11-16T22:09:57.295 INFO:teuthology.orchestra.run.smithi185.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:09:57.295 INFO:teuthology.orchestra.run.smithi185.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:09:57.296 INFO:teuthology.orchestra.run.smithi185.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:09:57.296 INFO:teuthology.orchestra.run.smithi185.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:09:57.297 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:09:57.297 INFO:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:09:57.345 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:09:57.346 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Command line: ntpd -gq 2020-11-16T22:09:57.346 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: proto: precision = 0.123 usec (-23) 2020-11-16T22:09:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:09:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:09:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2020-11-16T22:09:57.349 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen normally on 4 lo [::1]:123 2020-11-16T22:09:57.349 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2020-11-16T22:09:57.349 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:09:57 ntpd[6328]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:09:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:58 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:58.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:58 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:58.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:58 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:58.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:58 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:09:59.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:59 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:09:59.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:59 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:09:59.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:59 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:09:59.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:09:59 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:00.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:00 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:01.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:01 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:01.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:01 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:01.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:01 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:01.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:01 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:02.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:02.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:02.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:02.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:02.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:02.348 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:02 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:03.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:03 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:03.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:03 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:03.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:03 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:03.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:03 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:04.346 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stderr:16 Nov 22:10:04 ntpd[6328]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:10:04.347 INFO:teuthology.orchestra.run.smithi096.stdout:16 Nov 22:10:04 ntpd[6328]: ntpd: time slew -0.007208 s 2020-11-16T22:10:04.348 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.007208s 2020-11-16T22:10:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:10:04.400 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2020-11-16T22:10:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:04.401 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T22:10:04.401 INFO:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T22:10:04.484 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T22:10:04.484 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Command line: ntpd -gq 2020-11-16T22:10:04.486 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: proto: precision = 0.043 usec (-24) 2020-11-16T22:10:04.486 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:10:04.486 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:10:04.487 INFO:teuthology.orchestra.run.smithi062.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T22:10:04.487 INFO:teuthology.orchestra.run.smithi062.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen normally on 3 enp3s0f1 172.21.15.62:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen normally on 4 lo [::1]:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:febb:d1c3%5]:123 2020-11-16T22:10:04.488 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:04 ntpd[5072]: Listening on routing socket on fd #22 for interface updates 2020-11-16T22:10:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:05 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:05 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:05 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:05 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:07 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:07 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:07 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:07 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:08 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:08 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:08 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:08.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:08 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:09.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:09.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:09.486 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:09 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:10.482 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:10 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:10.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:10 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:10.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:10 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:10.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:10 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:11.482 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:11.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:11.484 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:16 Nov 22:10:11 ntpd[5072]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T22:10:11.485 INFO:teuthology.orchestra.run.smithi062.stdout:16 Nov 22:10:11 ntpd[5072]: ntpd: time slew -0.020986 s 2020-11-16T22:10:11.486 INFO:teuthology.orchestra.run.smithi062.stdout:ntpd: time slew -0.020986s 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================== 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T22:10:11.590 INFO:teuthology.run_tasks:Running task install... 2020-11-16T22:10:11.602 DEBUG:teuthology.task.install:project ceph 2020-11-16T22:10:11.603 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}} 2020-11-16T22:10:11.603 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'} 2020-11-16T22:10:11.603 INFO:teuthology.task.install:Using flavor: basic 2020-11-16T22:10:11.609 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-16T22:10:11.610 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-16T22:10:11.610 INFO:teuthology.orchestra.run.smithi185:> sudo lsb_release -is 2020-11-16T22:10:11.665 INFO:teuthology.orchestra.run.smithi185.stdout:Ubuntu 2020-11-16T22:10:11.670 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:11.670 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-16T22:10:11.671 INFO:teuthology.orchestra.run.smithi185:> sudo apt-key list | grep Ceph 2020-11-16T22:10:11.723 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-16T22:10:11.728 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:11.728 INFO:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2020-11-16T22:10:11.729 INFO:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2020-11-16T22:10:11.766 INFO:teuthology.orchestra.run.smithi185.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:10:11.801 INFO:teuthology.orchestra.run.smithi062.stdout:Ubuntu 2020-11-16T22:10:11.809 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:11.810 INFO:teuthology.orchestra.run.smithi062:> sudo apt-key list | grep Ceph 2020-11-16T22:10:11.811 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:10:11.826 INFO:teuthology.orchestra.run.smithi185.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:10:11.826 INFO:teuthology.orchestra.run.smithi185.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:10:11.827 INFO:teuthology.orchestra.run.smithi185:> sudo lsb_release -is 2020-11-16T22:10:11.858 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:10:11.858 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:10:11.859 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-16T22:10:11.875 INFO:teuthology.orchestra.run.smithi185.stdout:Ubuntu 2020-11-16T22:10:11.880 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:11.881 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:10:11.881 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:10:11.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:10:11.918 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-16T22:10:11.919 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:11.919 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:10:11.919 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:10:11.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:10:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T22:10:12.047 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:10:12.048 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:12.090 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:10:12.091 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:12.154 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T22:10:12.154 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T22:10:12.156 INFO:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2020-11-16T22:10:12.177 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:10:12.178 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.182 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.191 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:10:12.191 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:10:12.191 INFO:teuthology.orchestra.run.smithi185.stderr:172 bytes copied, 5.7925e-05 s, 3.0 MB/s 2020-11-16T22:10:12.192 INFO:teuthology.orchestra.run.smithi185:> sudo apt-get update 2020-11-16T22:10:12.211 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:10:12.212 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.216 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.224 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:10:12.224 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:10:12.224 INFO:teuthology.orchestra.run.smithi096.stderr:172 bytes copied, 4.2556e-05 s, 4.0 MB/s 2020-11-16T22:10:12.225 INFO:teuthology.orchestra.run.smithi096:> sudo apt-get update 2020-11-16T22:10:12.228 INFO:teuthology.orchestra.run.smithi062.stdout:Ubuntu 2020-11-16T22:10:12.236 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:10:12.237 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-16T22:10:12.237 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-16T22:10:12.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:10:12.395 INFO:teuthology.orchestra.run.smithi185.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:10:12.401 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default/ 2020-11-16T22:10:12.402 INFO:teuthology.task.install.deb:Package version is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:12.410 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:10:12.462 INFO:teuthology.orchestra.run.smithi185.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:10:12.486 INFO:teuthology.orchestra.run.smithi185.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:10:12.486 INFO:teuthology.orchestra.run.smithi185.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:10:12.505 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:10:12.511 INFO:teuthology.orchestra.run.smithi185.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:10:12.528 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:10:12.529 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.529 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:10:12.530 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:10:12.535 INFO:teuthology.orchestra.run.smithi185.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:10:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T22:10:12.552 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:10:12.558 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:10:12.558 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:10:12.559 INFO:teuthology.orchestra.run.smithi062.stderr:172 bytes copied, 0.00020663 s, 832 kB/s 2020-11-16T22:10:12.560 INFO:teuthology.orchestra.run.smithi062:> sudo apt-get update 2020-11-16T22:10:12.564 INFO:teuthology.orchestra.run.smithi185.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:10:12.571 INFO:teuthology.orchestra.run.smithi185.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:10:12.576 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:10:12.607 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:10:12.610 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:10:12.642 INFO:teuthology.orchestra.run.smithi185.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:10:12.696 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:10:12.912 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T22:10:12.950 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T22:10:12.950 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T22:10:12.950 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T22:10:12.973 INFO:teuthology.orchestra.run.smithi185.stdout:Fetched 26.8 kB in 1s (46.2 kB/s) 2020-11-16T22:10:12.976 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T22:10:13.000 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,951 B] 2020-11-16T22:10:13.028 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T22:10:13.037 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 26.8 kB in 1s (45.2 kB/s) 2020-11-16T22:10:13.052 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T22:10:13.082 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T22:10:14.201 INFO:teuthology.orchestra.run.smithi185.stdout:Reading package lists... 2020-11-16T22:10:14.248 INFO:teuthology.orchestra.run.smithi185:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:14.315 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 26.8 kB in 1s (19.1 kB/s) 2020-11-16T22:10:14.327 INFO:teuthology.orchestra.run.smithi185.stdout:Reading package lists... 2020-11-16T22:10:14.473 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-16T22:10:14.509 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:14.584 INFO:teuthology.orchestra.run.smithi185.stdout:Building dependency tree... 2020-11-16T22:10:14.585 INFO:teuthology.orchestra.run.smithi185.stdout:Reading state information... 2020-11-16T22:10:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-16T22:10:14.749 INFO:teuthology.orchestra.run.smithi185.stdout:The following additional packages will be installed: 2020-11-16T22:10:14.749 INFO:teuthology.orchestra.run.smithi185.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:10:14.750 INFO:teuthology.orchestra.run.smithi185.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:10:14.750 INFO:teuthology.orchestra.run.smithi185.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:10:14.751 INFO:teuthology.orchestra.run.smithi185.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:10:14.751 INFO:teuthology.orchestra.run.smithi185.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:10:14.751 INFO:teuthology.orchestra.run.smithi185.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:14.751 INFO:teuthology.orchestra.run.smithi185.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:10:14.752 INFO:teuthology.orchestra.run.smithi185.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:10:14.753 INFO:teuthology.orchestra.run.smithi185.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:10:14.754 INFO:teuthology.orchestra.run.smithi185.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:10:14.754 INFO:teuthology.orchestra.run.smithi185.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:10:14.754 INFO:teuthology.orchestra.run.smithi185.stdout:Suggested packages: 2020-11-16T22:10:14.754 INFO:teuthology.orchestra.run.smithi185.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:10:14.755 INFO:teuthology.orchestra.run.smithi185.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:10:14.756 INFO:teuthology.orchestra.run.smithi185.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:10:14.833 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-16T22:10:14.833 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-16T22:10:14.856 INFO:teuthology.orchestra.run.smithi185.stdout:The following NEW packages will be installed: 2020-11-16T22:10:14.856 INFO:teuthology.orchestra.run.smithi185.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:10:14.856 INFO:teuthology.orchestra.run.smithi185.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:10:14.856 INFO:teuthology.orchestra.run.smithi185.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:10:14.857 INFO:teuthology.orchestra.run.smithi185.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:10:14.857 INFO:teuthology.orchestra.run.smithi185.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:10:14.858 INFO:teuthology.orchestra.run.smithi185.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:10:14.858 INFO:teuthology.orchestra.run.smithi185.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:10:14.858 INFO:teuthology.orchestra.run.smithi185.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:10:14.859 INFO:teuthology.orchestra.run.smithi185.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:10:14.860 INFO:teuthology.orchestra.run.smithi185.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:10:14.861 INFO:teuthology.orchestra.run.smithi185.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:10:14.861 INFO:teuthology.orchestra.run.smithi185.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:10:14.861 INFO:teuthology.orchestra.run.smithi185.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:10:14.861 INFO:teuthology.orchestra.run.smithi185.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:10:14.861 INFO:teuthology.orchestra.run.smithi185.stdout:The following packages will be upgraded: 2020-11-16T22:10:14.862 INFO:teuthology.orchestra.run.smithi185.stdout: librados2 librbd1 2020-11-16T22:10:15.000 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2020-11-16T22:10:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:10:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:10:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:10:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:10:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:10:15.005 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2020-11-16T22:10:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:10:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:10:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:10:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:10:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:10:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:10:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:10:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:10:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:10:15.023 INFO:teuthology.orchestra.run.smithi185.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:10:15.023 INFO:teuthology.orchestra.run.smithi185.stdout:Need to get 168 MB of archives. 2020-11-16T22:10:15.023 INFO:teuthology.orchestra.run.smithi185.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:10:15.023 INFO:teuthology.orchestra.run.smithi185.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:10:15.047 INFO:teuthology.orchestra.run.smithi185.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:10:15.110 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2020-11-16T22:10:15.110 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:10:15.110 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:10:15.110 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:10:15.112 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:10:15.112 INFO:teuthology.orchestra.run.smithi096.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:10:15.112 INFO:teuthology.orchestra.run.smithi096.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:10:15.112 INFO:teuthology.orchestra.run.smithi096.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:10:15.113 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:10:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:10:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:10:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:10:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:10:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:10:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:10:15.116 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2020-11-16T22:10:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 librbd1 2020-11-16T22:10:15.208 INFO:teuthology.orchestra.run.smithi185.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:10:15.275 INFO:teuthology.orchestra.run.smithi096.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:10:15.275 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 168 MB of archives. 2020-11-16T22:10:15.275 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:10:15.275 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:10:15.297 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:10:15.306 INFO:teuthology.orchestra.run.smithi185.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:10:15.309 INFO:teuthology.orchestra.run.smithi185.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:10:15.313 INFO:teuthology.orchestra.run.smithi185.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:10:15.322 INFO:teuthology.orchestra.run.smithi185.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:10:15.323 INFO:teuthology.orchestra.run.smithi185.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:10:15.326 INFO:teuthology.orchestra.run.smithi185.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:10:15.337 INFO:teuthology.orchestra.run.smithi185.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:10:15.337 INFO:teuthology.orchestra.run.smithi185.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:10:15.343 INFO:teuthology.orchestra.run.smithi185.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:10:15.344 INFO:teuthology.orchestra.run.smithi185.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:10:15.362 INFO:teuthology.orchestra.run.smithi185.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:10:15.469 INFO:teuthology.orchestra.run.smithi185.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:10:15.479 INFO:teuthology.orchestra.run.smithi185.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:10:15.490 INFO:teuthology.orchestra.run.smithi185.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:10:15.493 INFO:teuthology.orchestra.run.smithi185.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:10:15.494 INFO:teuthology.orchestra.run.smithi185.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:10:15.516 INFO:teuthology.orchestra.run.smithi185.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:10:15.546 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:10:15.554 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:10:15.564 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:10:15.564 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:10:15.567 INFO:teuthology.orchestra.run.smithi185.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:10:15.571 INFO:teuthology.orchestra.run.smithi185.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:10:15.596 INFO:teuthology.orchestra.run.smithi185.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:10:15.604 INFO:teuthology.orchestra.run.smithi185.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:10:15.613 INFO:teuthology.orchestra.run.smithi185.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:10:15.623 INFO:teuthology.orchestra.run.smithi185.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:10:15.630 INFO:teuthology.orchestra.run.smithi185.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:10:15.635 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:10:15.648 INFO:teuthology.orchestra.run.smithi185.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:10:15.651 INFO:teuthology.orchestra.run.smithi185.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:10:15.707 INFO:teuthology.orchestra.run.smithi185.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:10:15.708 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:10:15.715 INFO:teuthology.orchestra.run.smithi185.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:10:15.716 INFO:teuthology.orchestra.run.smithi185.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:10:15.717 INFO:teuthology.orchestra.run.smithi185.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:10:15.718 INFO:teuthology.orchestra.run.smithi185.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:10:15.720 INFO:teuthology.orchestra.run.smithi185.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:10:15.746 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:10:15.762 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:10:15.785 INFO:teuthology.orchestra.run.smithi185.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:10:15.786 INFO:teuthology.orchestra.run.smithi185.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:10:15.788 INFO:teuthology.orchestra.run.smithi185.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:10:15.791 INFO:teuthology.orchestra.run.smithi185.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:10:15.863 INFO:teuthology.orchestra.run.smithi185.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:10:15.864 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:10:15.864 INFO:teuthology.orchestra.run.smithi185.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:10:15.865 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:10:15.878 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:10:15.882 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:10:15.884 INFO:teuthology.orchestra.run.smithi185.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:10:15.884 INFO:teuthology.orchestra.run.smithi185.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:10:15.884 INFO:teuthology.orchestra.run.smithi185.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:10:15.885 INFO:teuthology.orchestra.run.smithi185.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:10:15.888 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:10:15.895 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:10:15.901 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:10:15.931 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2020-11-16T22:10:15.942 INFO:teuthology.orchestra.run.smithi185.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:10:15.948 INFO:teuthology.orchestra.run.smithi185.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:10:15.951 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:10:15.953 INFO:teuthology.orchestra.run.smithi185.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:10:15.953 INFO:teuthology.orchestra.run.smithi185.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:10:15.999 INFO:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7226-g6959059e-1bionic cephadm=16.0.0-7226-g6959059e-1bionic ceph-mds=16.0.0-7226-g6959059e-1bionic ceph-mgr=16.0.0-7226-g6959059e-1bionic ceph-common=16.0.0-7226-g6959059e-1bionic ceph-fuse=16.0.0-7226-g6959059e-1bionic ceph-test=16.0.0-7226-g6959059e-1bionic radosgw=16.0.0-7226-g6959059e-1bionic python3-rados=16.0.0-7226-g6959059e-1bionic python3-rgw=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic python3-rbd=16.0.0-7226-g6959059e-1bionic libcephfs2=16.0.0-7226-g6959059e-1bionic libcephfs-dev=16.0.0-7226-g6959059e-1bionic librados2=16.0.0-7226-g6959059e-1bionic librbd1=16.0.0-7226-g6959059e-1bionic rbd-fuse=16.0.0-7226-g6959059e-1bionic python3-cephfs=16.0.0-7226-g6959059e-1bionic cephfs-shell=16.0.0-7226-g6959059e-1bionic 2020-11-16T22:10:16.014 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:10:16.015 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:10:16.019 INFO:teuthology.orchestra.run.smithi185.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:10:16.020 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:10:16.020 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:10:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:10:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:10:16.094 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:10:16.094 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:10:16.095 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:10:16.100 INFO:teuthology.orchestra.run.smithi185.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:10:16.101 INFO:teuthology.orchestra.run.smithi185.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:10:16.118 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:10:16.128 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2020-11-16T22:10:16.191 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:10:16.196 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:10:16.199 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:10:16.202 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:10:16.203 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:10:16.217 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:10:16.263 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:10:16.264 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:10:16.269 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:10:16.270 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:10:16.288 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:10:16.289 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:10:16.290 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:10:16.294 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:10:16.299 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:10:16.345 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:10:16.367 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:10:16.367 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:10:16.368 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:10:16.377 INFO:teuthology.orchestra.run.smithi185.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:10:16.403 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2020-11-16T22:10:16.404 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2020-11-16T22:10:16.421 INFO:teuthology.orchestra.run.smithi185.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:10:16.422 INFO:teuthology.orchestra.run.smithi185.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:10:16.453 INFO:teuthology.orchestra.run.smithi185.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:10:16.454 INFO:teuthology.orchestra.run.smithi185.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:10:16.454 INFO:teuthology.orchestra.run.smithi185.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:10:16.455 INFO:teuthology.orchestra.run.smithi185.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:10:16.455 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:10:16.456 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:10:16.456 INFO:teuthology.orchestra.run.smithi185.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:10:16.456 INFO:teuthology.orchestra.run.smithi185.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:10:16.457 INFO:teuthology.orchestra.run.smithi185.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:10:16.470 INFO:teuthology.orchestra.run.smithi185.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:10:16.536 INFO:teuthology.orchestra.run.smithi185.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:10:16.537 INFO:teuthology.orchestra.run.smithi185.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:10:16.544 INFO:teuthology.orchestra.run.smithi185.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:10:16.545 INFO:teuthology.orchestra.run.smithi185.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:10:16.549 INFO:teuthology.orchestra.run.smithi185.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:10:16.549 INFO:teuthology.orchestra.run.smithi185.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:10:16.549 INFO:teuthology.orchestra.run.smithi185.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:10:16.583 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2020-11-16T22:10:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T22:10:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T22:10:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T22:10:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T22:10:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-16T22:10:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T22:10:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T22:10:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T22:10:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T22:10:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T22:10:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T22:10:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T22:10:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T22:10:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T22:10:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T22:10:16.601 INFO:teuthology.orchestra.run.smithi185.stdout:Get:71 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:10:16.614 INFO:teuthology.orchestra.run.smithi185.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:10:16.617 INFO:teuthology.orchestra.run.smithi185.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:10:16.618 INFO:teuthology.orchestra.run.smithi185.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:10:16.619 INFO:teuthology.orchestra.run.smithi185.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:10:16.664 INFO:teuthology.orchestra.run.smithi185.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:10:16.692 INFO:teuthology.orchestra.run.smithi185.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:10:16.698 INFO:teuthology.orchestra.run.smithi185.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:10:16.706 INFO:teuthology.orchestra.run.smithi185.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:10:16.706 INFO:teuthology.orchestra.run.smithi185.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:10:16.708 INFO:teuthology.orchestra.run.smithi185.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:10:16.709 INFO:teuthology.orchestra.run.smithi185.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:10:16.709 INFO:teuthology.orchestra.run.smithi185.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:10:16.726 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:10:16.757 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2020-11-16T22:10:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T22:10:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T22:10:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-16T22:10:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-16T22:10:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-16T22:10:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-16T22:10:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-16T22:10:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T22:10:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T22:10:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T22:10:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-16T22:10:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-16T22:10:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-16T22:10:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T22:10:16.763 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages will be upgraded: 2020-11-16T22:10:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 librbd1 2020-11-16T22:10:16.770 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:10:16.770 INFO:teuthology.orchestra.run.smithi185.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:10:16.772 INFO:teuthology.orchestra.run.smithi185.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:16.772 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:10:16.775 INFO:teuthology.orchestra.run.smithi185.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:10:16.794 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:10:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:10:16.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:10:16.803 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:10:16.804 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:10:16.804 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:10:16.805 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:10:16.806 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:10:16.807 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:10:16.849 INFO:teuthology.orchestra.run.smithi185.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:10:16.849 INFO:teuthology.orchestra.run.smithi185.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:10:16.850 INFO:teuthology.orchestra.run.smithi185.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:10:16.853 INFO:teuthology.orchestra.run.smithi185.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:10:16.858 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:10:16.872 INFO:teuthology.orchestra.run.smithi185.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:10:16.872 INFO:teuthology.orchestra.run.smithi185.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:10:16.886 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:10:16.888 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:10:16.894 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:10:16.895 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:10:16.899 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:10:16.899 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:10:16.899 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:10:16.949 INFO:teuthology.orchestra.run.smithi185.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:10:16.965 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:10:16.968 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:10:16.970 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:10:16.977 INFO:teuthology.orchestra.run.smithi062.stdout:2 upgraded, 114 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:10:16.977 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 168 MB of archives. 2020-11-16T22:10:16.977 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T22:10:16.977 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T22:10:17.008 INFO:teuthology.orchestra.run.smithi185.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:10:17.009 INFO:teuthology.orchestra.run.smithi185.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:10:17.010 INFO:teuthology.orchestra.run.smithi185.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:10:17.017 INFO:teuthology.orchestra.run.smithi185.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:10:17.018 INFO:teuthology.orchestra.run.smithi185.stdout:Get:98 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:10:17.018 INFO:teuthology.orchestra.run.smithi185.stdout:Get:99 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:10:17.019 INFO:teuthology.orchestra.run.smithi185.stdout:Get:100 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:17.020 INFO:teuthology.orchestra.run.smithi185.stdout:Get:101 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:10:17.021 INFO:teuthology.orchestra.run.smithi185.stdout:Get:102 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:10:17.042 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:10:17.045 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:10:17.049 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:10:17.049 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:10:17.051 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:10:17.052 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:10:17.052 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:10:17.121 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:10:17.123 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:17.127 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:10:17.160 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7226-g6959059e-1bionic [1,924 kB] 2020-11-16T22:10:17.199 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:10:17.199 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:10:17.200 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:10:17.203 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:10:17.223 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:10:17.223 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:10:17.241 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-16T22:10:17.246 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-16T22:10:17.264 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T22:10:17.265 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T22:10:17.359 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:10:17.360 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:10:17.361 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:10:17.370 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:10:17.370 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:10:17.371 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:10:17.371 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:17.373 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:10:17.373 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:10:17.412 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T22:10:17.420 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T22:10:17.430 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T22:10:17.514 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T22:10:17.514 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T22:10:17.535 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T22:10:17.541 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T22:10:17.548 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T22:10:17.562 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T22:10:17.571 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T22:10:17.593 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T22:10:17.597 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T22:10:17.653 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T22:10:17.662 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T22:10:17.663 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T22:10:17.663 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T22:10:17.664 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T22:10:17.666 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T22:10:17.666 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T22:10:17.729 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7226-g6959059e-1bionic [3,159 kB] 2020-11-16T22:10:17.733 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T22:10:17.737 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T22:10:17.740 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T22:10:17.813 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T22:10:17.814 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T22:10:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T22:10:17.834 INFO:teuthology.orchestra.run.smithi062.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T22:10:17.835 INFO:teuthology.orchestra.run.smithi062.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T22:10:17.836 INFO:teuthology.orchestra.run.smithi062.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T22:10:17.839 INFO:teuthology.orchestra.run.smithi062.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T22:10:17.892 INFO:teuthology.orchestra.run.smithi062.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T22:10:17.897 INFO:teuthology.orchestra.run.smithi062.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T22:10:17.899 INFO:teuthology.orchestra.run.smithi062.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T22:10:17.972 INFO:teuthology.orchestra.run.smithi062.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T22:10:18.008 INFO:teuthology.orchestra.run.smithi062.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T22:10:18.008 INFO:teuthology.orchestra.run.smithi062.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T22:10:18.170 INFO:teuthology.orchestra.run.smithi185.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:10:18.170 INFO:teuthology.orchestra.run.smithi185.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:10:18.214 INFO:teuthology.orchestra.run.smithi185.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:10:18.261 INFO:teuthology.orchestra.run.smithi062.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7226-g6959059e-1bionic [478 kB] 2020-11-16T22:10:18.306 INFO:teuthology.orchestra.run.smithi185.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:10:18.307 INFO:teuthology.orchestra.run.smithi185.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:10:18.309 INFO:teuthology.orchestra.run.smithi185.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:10:18.313 INFO:teuthology.orchestra.run.smithi062.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T22:10:18.331 INFO:teuthology.orchestra.run.smithi062.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T22:10:18.332 INFO:teuthology.orchestra.run.smithi062.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T22:10:18.385 INFO:teuthology.orchestra.run.smithi062.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7226-g6959059e-1bionic [332 kB] 2020-11-16T22:10:18.395 INFO:teuthology.orchestra.run.smithi062.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T22:10:18.396 INFO:teuthology.orchestra.run.smithi062.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T22:10:18.396 INFO:teuthology.orchestra.run.smithi062.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T22:10:18.397 INFO:teuthology.orchestra.run.smithi062.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T22:10:18.398 INFO:teuthology.orchestra.run.smithi062.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T22:10:18.399 INFO:teuthology.orchestra.run.smithi062.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T22:10:18.399 INFO:teuthology.orchestra.run.smithi062.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T22:10:18.452 INFO:teuthology.orchestra.run.smithi062.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7226-g6959059e-1bionic [20.6 kB] 2020-11-16T22:10:18.453 INFO:teuthology.orchestra.run.smithi062.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7226-g6959059e-1bionic [168 kB] 2020-11-16T22:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7226-g6959059e-1bionic [27.7 kB] 2020-11-16T22:10:18.463 INFO:teuthology.orchestra.run.smithi062.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7226-g6959059e-1bionic [331 kB] 2020-11-16T22:10:18.473 INFO:teuthology.orchestra.run.smithi062.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T22:10:18.475 INFO:teuthology.orchestra.run.smithi062.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T22:10:18.490 INFO:teuthology.orchestra.run.smithi185.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:10:18.526 INFO:teuthology.orchestra.run.smithi062.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7226-g6959059e-1bionic [3,176 kB] 2020-11-16T22:10:18.552 INFO:teuthology.orchestra.run.smithi062.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T22:10:18.554 INFO:teuthology.orchestra.run.smithi062.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T22:10:18.561 INFO:teuthology.orchestra.run.smithi062.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T22:10:18.562 INFO:teuthology.orchestra.run.smithi062.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T22:10:18.563 INFO:teuthology.orchestra.run.smithi062.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T22:10:18.566 INFO:teuthology.orchestra.run.smithi062.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T22:10:18.568 INFO:teuthology.orchestra.run.smithi062.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T22:10:18.568 INFO:teuthology.orchestra.run.smithi062.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T22:10:18.632 INFO:teuthology.orchestra.run.smithi062.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T22:10:18.639 INFO:teuthology.orchestra.run.smithi062.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T22:10:18.711 INFO:teuthology.orchestra.run.smithi062.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-16T22:10:18.713 INFO:teuthology.orchestra.run.smithi062.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T22:10:18.721 INFO:teuthology.orchestra.run.smithi062.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T22:10:18.722 INFO:teuthology.orchestra.run.smithi062.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T22:10:18.722 INFO:teuthology.orchestra.run.smithi062.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T22:10:18.723 INFO:teuthology.orchestra.run.smithi062.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T22:10:18.724 INFO:teuthology.orchestra.run.smithi062.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:18.726 INFO:teuthology.orchestra.run.smithi062.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T22:10:18.791 INFO:teuthology.orchestra.run.smithi062.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T22:10:18.793 INFO:teuthology.orchestra.run.smithi062.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T22:10:18.871 INFO:teuthology.orchestra.run.smithi062.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T22:10:18.874 INFO:teuthology.orchestra.run.smithi062.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T22:10:18.896 INFO:teuthology.orchestra.run.smithi062.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T22:10:18.897 INFO:teuthology.orchestra.run.smithi062.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T22:10:18.926 INFO:teuthology.orchestra.run.smithi185.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:10:18.926 INFO:teuthology.orchestra.run.smithi185.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:10:18.926 INFO:teuthology.orchestra.run.smithi185.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:10:19.037 INFO:teuthology.orchestra.run.smithi062.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T22:10:19.038 INFO:teuthology.orchestra.run.smithi062.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T22:10:19.040 INFO:teuthology.orchestra.run.smithi062.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T22:10:19.046 INFO:teuthology.orchestra.run.smithi062.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T22:10:19.046 INFO:teuthology.orchestra.run.smithi062.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T22:10:19.047 INFO:teuthology.orchestra.run.smithi062.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T22:10:19.048 INFO:teuthology.orchestra.run.smithi062.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T22:10:19.050 INFO:teuthology.orchestra.run.smithi062.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T22:10:19.051 INFO:teuthology.orchestra.run.smithi062.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T22:10:19.080 INFO:teuthology.orchestra.run.smithi062.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7226-g6959059e-1bionic [99.2 kB] 2020-11-16T22:10:19.083 INFO:teuthology.orchestra.run.smithi062.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7226-g6959059e-1bionic [414 kB] 2020-11-16T22:10:19.157 INFO:teuthology.orchestra.run.smithi062.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7226-g6959059e-1bionic [20.7 MB] 2020-11-16T22:10:19.822 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:10:20.571 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:10:20.576 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:10:20.794 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:10:21.723 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:10:21.827 INFO:teuthology.orchestra.run.smithi185.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:10:21.827 INFO:teuthology.orchestra.run.smithi185.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:10:21.828 INFO:teuthology.orchestra.run.smithi185.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:10:22.387 INFO:teuthology.orchestra.run.smithi185.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:10:22.473 INFO:teuthology.orchestra.run.smithi062.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7226-g6959059e-1bionic [5,567 kB] 2020-11-16T22:10:22.872 INFO:teuthology.orchestra.run.smithi062.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7226-g6959059e-1bionic [165 kB] 2020-11-16T22:10:22.875 INFO:teuthology.orchestra.run.smithi062.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7226-g6959059e-1bionic [1,361 kB] 2020-11-16T22:10:22.973 INFO:teuthology.orchestra.run.smithi062.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7226-g6959059e-1bionic [6,658 kB] 2020-11-16T22:10:23.096 INFO:teuthology.orchestra.run.smithi185.stdout:Fetched 168 MB in 8s (22.4 MB/s) 2020-11-16T22:10:23.216 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:10:23.256 INFO:teuthology.orchestra.run.smithi185.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:10:23.260 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:10:23.289 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:10:23.449 INFO:teuthology.orchestra.run.smithi062.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7226-g6959059e-1bionic [24.6 MB] 2020-11-16T22:10:23.609 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:10:23.610 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:10:23.611 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:10:23.621 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:10:23.642 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:10:23.659 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:10:23.766 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:10:23.768 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:10:23.770 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:10:23.943 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:10:23.962 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:10:23.974 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:10:24.012 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:10:24.315 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:10:24.327 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:10:24.356 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:10:24.438 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:10:24.438 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:10:24.439 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:10:24.693 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:10:24.705 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:10:24.810 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:10:25.143 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:25.186 INFO:teuthology.orchestra.run.smithi062.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7226-g6959059e-1bionic [3,812 B] 2020-11-16T22:10:25.187 INFO:teuthology.orchestra.run.smithi062.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7226-g6959059e-1bionic [563 kB] 2020-11-16T22:10:25.217 INFO:teuthology.orchestra.run.smithi062.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7226-g6959059e-1bionic [1,948 kB] 2020-11-16T22:10:25.264 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:25.354 INFO:teuthology.orchestra.run.smithi062.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7226-g6959059e-1bionic [51.7 kB] 2020-11-16T22:10:25.355 INFO:teuthology.orchestra.run.smithi062.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7226-g6959059e-1bionic [56.2 kB] 2020-11-16T22:10:25.365 INFO:teuthology.orchestra.run.smithi062.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7226-g6959059e-1bionic [2,877 kB] 2020-11-16T22:10:25.569 INFO:teuthology.orchestra.run.smithi062.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7226-g6959059e-1bionic [7,875 kB] 2020-11-16T22:10:25.805 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:25.927 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:26.125 INFO:teuthology.orchestra.run.smithi062.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7226-g6959059e-1bionic [3,928 B] 2020-11-16T22:10:26.125 INFO:teuthology.orchestra.run.smithi062.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7226-g6959059e-1bionic [22.3 kB] 2020-11-16T22:10:26.128 INFO:teuthology.orchestra.run.smithi062.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7226-g6959059e-1bionic [49.6 MB] 2020-11-16T22:10:26.573 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:10:26.585 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:26.623 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:26.893 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:10:26.905 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:26.934 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:27.270 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:10:27.282 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:27.320 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:27.437 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:10:27.438 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:10:27.438 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:10:27.606 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:10:27.619 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:27.656 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:27.957 INFO:teuthology.orchestra.run.smithi096.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:10:27.968 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:10:27.979 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:28.009 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:28.279 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:10:28.292 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:10:28.329 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:10:28.591 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:10:28.600 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:28.640 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:28.652 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 168 MB in 13s (13.1 MB/s) 2020-11-16T22:10:28.767 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:10:28.805 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T22:10:28.808 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:10:28.841 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:10:29.002 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:10:29.012 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:10:29.052 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:10:29.161 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:10:29.173 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:10:29.211 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:10:29.355 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:10:29.368 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:10:29.405 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:10:29.405 INFO:teuthology.orchestra.run.smithi062.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7226-g6959059e-1bionic [17.0 kB] 2020-11-16T22:10:29.406 INFO:teuthology.orchestra.run.smithi062.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7226-g6959059e-1bionic [19.5 kB] 2020-11-16T22:10:29.406 INFO:teuthology.orchestra.run.smithi062.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7226-g6959059e-1bionic [9,854 kB] 2020-11-16T22:10:29.522 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:10:29.535 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:10:29.572 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:10:29.749 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:10:29.761 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:29.799 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:29.833 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:10:29.845 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:10:29.875 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:10:29.979 INFO:teuthology.orchestra.run.smithi062.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201116.174825/6959059e6aae2fb5241382a4eeeca62785867604/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7226-g6959059e-1bionic [77.0 kB] 2020-11-16T22:10:30.211 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:10:30.224 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:10:30.303 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:10:30.344 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:10:30.355 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:30.394 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:30.616 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:30.680 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:10:30.691 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:10:30.722 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:10:30.740 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:31.025 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:10:31.036 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 168 MB in 13s (12.8 MB/s) 2020-11-16T22:10:31.037 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:10:31.083 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:10:31.227 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T22:10:31.297 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104570 files and directories currently installed.) 2020-11-16T22:10:31.300 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T22:10:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:31.349 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:10:31.428 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:31.461 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:10:31.473 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:10:31.512 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:10:31.718 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-16T22:10:31.733 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-16T22:10:31.781 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:10:31.793 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-16T22:10:31.794 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:31.831 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:32.090 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:10:32.103 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:32.138 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cmd2. 2020-11-16T22:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:32.153 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-16T22:10:32.196 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:10:32.251 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:10:32.264 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:32.301 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:32.417 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:32.540 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T22:10:32.554 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T22:10:32.599 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:10:32.820 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:10:32.832 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:32.870 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:33.018 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T22:10:33.032 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T22:10:33.118 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:10:33.173 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:10:33.184 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:10:33.529 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../005-librbd1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:33.538 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:33.575 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:33.664 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librbd1 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:33.879 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:10:33.892 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:10:34.181 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:10:34.191 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:10:34.191 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:34.202 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:34.231 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:34.250 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:34.287 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../006-librados2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librados2 (16.0.0-7226-g6959059e-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T22:10:34.626 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:10:34.637 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:10:34.676 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:10:34.937 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:10:34.948 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:10:34.979 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:10:35.122 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T22:10:35.138 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:35.163 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:10:35.174 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:10:35.197 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:35.213 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:10:35.315 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:10:35.326 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:35.390 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:35.508 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:10:35.516 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rados. 2020-11-16T22:10:35.519 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:10:35.531 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:35.549 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:10:35.574 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:35.802 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:10:35.814 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:10:35.852 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:10:35.951 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:10:35.962 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:35.977 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T22:10:35.993 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:36.000 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:36.035 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:36.163 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:10:36.176 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:10:36.213 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:10:36.312 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:10:36.325 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:10:36.353 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:10:36.438 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T22:10:36.452 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:36.496 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:36.584 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:10:36.596 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:10:36.634 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:10:36.648 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:10:36.660 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:10:36.698 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:10:36.841 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T22:10:36.856 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:36.891 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:36.903 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:10:36.915 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:36.953 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:37.105 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:10:37.117 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:10:37.160 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:10:37.227 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T22:10:37.242 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T22:10:37.285 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T22:10:37.316 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:10:37.329 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:10:37.366 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:10:37.429 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:10:37.441 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:37.471 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:37.571 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T22:10:37.585 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:37.645 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:10:37.654 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:37.655 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:10:37.695 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:10:38.024 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:10:38.036 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:38.048 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:10:38.059 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:10:38.065 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T22:10:38.074 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:38.080 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T22:10:38.098 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:10:38.124 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:10:38.395 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:10:38.408 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:10:38.445 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:10:38.518 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T22:10:38.532 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T22:10:38.610 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:10:38.748 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:10:38.761 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:10:38.798 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:10:38.979 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librgw2. 2020-11-16T22:10:38.991 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../016-librgw2_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:39.029 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:39.059 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:10:39.071 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:10:39.101 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:10:39.371 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:10:39.384 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:10:39.421 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:10:39.640 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T22:10:39.654 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:39.698 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:39.724 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:10:39.737 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:10:39.774 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:10:39.897 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:10:39.907 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:39.947 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:40.077 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:10:40.089 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:10:40.118 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T22:10:40.127 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:10:40.132 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T22:10:40.176 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:10:40.447 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:10:40.458 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:10:40.495 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T22:10:40.497 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:10:40.510 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T22:10:40.554 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:10:40.791 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:10:40.804 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:10:40.833 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:10:40.885 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:10:40.897 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:10:40.935 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:10:41.023 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liboath0. 2020-11-16T22:10:41.039 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T22:10:41.082 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T22:10:41.128 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:10:41.141 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:10:41.178 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:10:41.221 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:10:41.233 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:10:41.288 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:10:41.451 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T22:10:41.467 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:41.509 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:41.557 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:10:41.569 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:10:41.599 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:10:41.656 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:10:41.667 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:10:41.706 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:10:41.894 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:10:41.905 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:10:41.935 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:10:41.984 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:10:41.988 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-common. 2020-11-16T22:10:41.997 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:10:42.002 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:42.026 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:10:42.037 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:42.256 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:10:42.268 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:10:42.306 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:10:42.337 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:10:42.351 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:10:42.387 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:10:42.583 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:10:42.595 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:42.674 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:10:42.687 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:10:42.724 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:10:42.767 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:43.035 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:10:43.048 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:10:43.085 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:10:43.105 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:10:43.116 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:10:43.154 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:10:43.399 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:10:43.412 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:10:43.440 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:10:43.482 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:10:43.494 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:10:43.532 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:10:43.769 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:10:43.780 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:43.819 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:43.852 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:10:43.865 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:10:43.903 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:10:44.202 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-base. 2020-11-16T22:10:44.214 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:10:44.218 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:44.226 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:10:44.264 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:10:44.281 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:10:44.285 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:44.293 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:44.330 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:44.542 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:10:44.553 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:10:44.583 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:10:44.887 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:10:44.899 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:10:44.936 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:10:45.268 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:10:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:10:45.274 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:45.284 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:10:45.318 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:45.323 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:10:45.332 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T22:10:45.347 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T22:10:45.382 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T22:10:45.609 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:10:45.619 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:10:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:10:45.776 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T22:10:45.791 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T22:10:45.834 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:10:45.937 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:10:45.947 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:10:45.978 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:10:46.162 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T22:10:46.178 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T22:10:46.212 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:10:46.264 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:10:46.276 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:10:46.314 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:10:46.607 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T22:10:46.622 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T22:10:46.634 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:10:46.645 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:10:46.691 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:10:46.693 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:10:47.029 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:10:47.041 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:10:47.079 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:10:47.110 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T22:10:47.125 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T22:10:47.160 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:10:47.382 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:10:47.393 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:10:47.432 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:10:47.512 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T22:10:47.528 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:47.604 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:48.007 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph. 2020-11-16T22:10:48.020 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:48.057 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:48.067 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T22:10:48.082 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T22:10:48.125 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T22:10:48.377 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:10:48.389 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:48.427 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:48.494 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:10:48.505 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:10:48.536 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T22:10:48.544 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:10:48.555 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T22:10:48.595 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:10:48.780 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:10:48.793 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:48.821 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:10:48.830 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:48.833 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:10:48.863 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:10:48.998 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T22:10:49.015 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T22:10:49.056 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:10:49.132 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:10:49.145 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:10:49.183 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:10:49.317 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:10:49.329 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:49.367 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:49.460 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T22:10:49.476 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T22:10:49.518 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T22:10:49.552 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:10:49.563 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:10:49.602 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:10:49.670 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:10:49.683 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:10:49.728 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:10:49.837 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T22:10:49.853 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T22:10:49.895 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T22:10:49.940 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:10:49.952 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:10:49.990 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:10:50.048 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:10:50.059 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:50.098 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:50.386 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:10:50.397 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:50.426 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:10:50.435 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:50.438 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:10:50.476 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:10:50.755 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:10:50.766 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:10:50.796 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:10:50.864 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:10:50.876 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:50.914 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:51.107 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:10:51.119 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:51.157 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:51.224 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-mako. 2020-11-16T22:10:51.238 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T22:10:51.282 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:10:51.727 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T22:10:51.743 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T22:10:51.785 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:10:51.867 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:10:51.878 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:51.951 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:52.009 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:10:52.023 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:10:52.075 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:10:52.087 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T22:10:52.103 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T22:10:52.137 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:10:52.499 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webob. 2020-11-16T22:10:52.513 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T22:10:52.557 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:10:52.615 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:10:52.628 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:10:52.665 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:10:52.951 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:10:52.964 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:10:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T22:10:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T22:10:53.001 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:10:53.026 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T22:10:53.404 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T22:10:53.419 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T22:10:53.454 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T22:10:53.832 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T22:10:53.847 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T22:10:53.924 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T22:10:53.957 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2020-11-16T22:10:53.969 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:53.998 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:54.126 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:10:54.138 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:10:54.176 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:10:54.277 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-paste. 2020-11-16T22:10:54.293 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T22:10:54.326 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:10:54.335 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:10:54.337 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:54.376 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:54.639 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:10:54.650 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:10:54.688 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:10:54.704 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T22:10:54.719 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T22:10:54.763 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:10:54.788 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:10:54.798 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:54.838 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:55.042 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:10:55.054 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:10:55.092 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:10:55.133 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T22:10:55.148 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T22:10:55.216 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:10:55.266 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:10:55.277 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:55.316 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:55.560 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T22:10:55.576 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T22:10:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:10:55.632 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:10:55.647 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:55.652 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:10:55.662 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:10:55.691 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:55.702 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:10:56.005 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T22:10:56.005 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:10:56.017 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:56.021 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T22:10:56.055 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:56.079 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T22:10:56.400 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:10:56.412 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:10:56.450 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:10:56.451 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T22:10:56.466 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T22:10:56.509 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:10:56.720 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:10:56.730 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:10:56.761 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:10:56.896 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T22:10:56.912 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:56.953 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:10:56.954 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:56.964 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:10:57.003 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:10:57.056 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:10:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:10:57.106 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:57.264 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:10:57.278 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:10:57.314 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:10:57.483 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T22:10:57.498 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:57.541 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:57.617 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:10:57.630 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:10:57.667 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:10:57.953 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:10:57.967 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:10:58.002 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:10:58.012 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:10:58.016 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:10:58.066 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:10:58.349 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:10:58.361 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:10:58.399 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:10:58.536 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T22:10:58.550 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:10:58.611 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:10:58.655 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:10:58.668 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:10:58.693 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:10:58.705 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:10:58.707 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:10:58.752 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:10:59.022 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:10:59.036 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:10:59.049 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:10:59.061 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:10:59.080 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:10:59.099 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:10:59.527 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:10:59.540 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:10:59.601 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:10:59.947 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:10:59.958 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:10:59.996 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:11:00.249 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:11:00.259 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:11:00.298 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:11:00.384 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:11:00.395 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:11:00.433 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:11:00.745 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:11:00.758 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:11:00.793 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:11:00.795 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:11:00.925 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph. 2020-11-16T22:11:00.940 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../051-ceph_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:00.983 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:01.147 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:11:01.159 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:11:01.175 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:11:01.188 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:01.197 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:11:01.224 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:01.444 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T22:11:01.453 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:01.494 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:01.528 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:11:01.541 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:11:01.586 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:11:01.703 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:11:01.717 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:01.761 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:01.931 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:11:01.944 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:11:01.964 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T22:11:01.979 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:02.014 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:02.061 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:02.353 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:11:02.366 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:02.403 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:02.584 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephadm. 2020-11-16T22:11:02.602 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../054-cephadm_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:02.642 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:02.732 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:11:02.744 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:11:02.781 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:02.856 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:11:02.868 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:11:02.906 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:11:03.003 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T22:11:03.019 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T22:11:03.062 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:11:03.110 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:11:03.122 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:03.159 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:11:03.296 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:11:03.333 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:11:03.412 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package jq. 2020-11-16T22:11:03.426 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:03.471 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:11:03.507 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T22:11:03.521 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:03.565 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:03.595 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:11:03.607 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:11:03.645 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:11:03.724 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package socat. 2020-11-16T22:11:03.737 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:11:03.774 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:03.947 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:11:03.960 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T22:11:03.960 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:11:03.975 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T22:11:03.997 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:11:04.018 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T22:11:04.186 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:11:04.197 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:11:04.236 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:04.359 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:11:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:11:04.379 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-routes. 2020-11-16T22:11:04.394 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-16T22:11:04.409 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:11:04.462 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T22:11:04.497 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:11:04.510 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:11:04.539 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:11:04.728 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:11:04.741 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:11:04.778 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:11:04.815 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T22:11:04.835 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:04.843 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:11:04.850 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:04.873 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:04.893 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:05.115 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:11:05.127 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:11:05.164 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:11:05.594 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:11:05.607 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:11:05.643 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:11:05.980 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:11:05.993 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:11:06.030 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:11:06.090 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T22:11:06.109 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T22:11:06.148 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:11:06.416 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:11:06.430 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:11:06.474 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:06.803 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:11:06.814 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:11:06.852 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:11:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T22:11:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T22:11:06.970 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T22:11:07.248 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:11:07.259 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:07.298 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:07.340 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T22:11:07.355 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T22:11:07.398 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:11:07.634 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:11:07.647 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:11:07.692 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:11:08.037 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:11:08.049 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:11:08.176 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:08.498 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:11:08.510 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:11:08.548 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:11:08.742 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:11:08.755 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:08.801 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:08.801 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:11:08.813 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:08.815 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T22:11:08.831 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T22:11:08.850 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:11:09.095 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:11:09.108 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:09.128 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:11:09.137 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:09.141 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:11:09.178 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:09.398 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:11:09.401 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T22:11:09.410 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:11:09.417 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-16T22:11:09.448 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:09.459 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T22:11:09.531 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:11:09.544 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:09.573 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:09.769 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:11:09.783 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:09.827 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:11:09.834 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2020-11-16T22:11:09.847 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:09.892 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:11:09.929 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T22:11:09.945 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T22:11:09.988 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T22:11:10.114 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:11:10.126 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:11:10.136 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2020-11-16T22:11:10.147 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:11:10.164 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:10.425 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:11:10.439 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:10.484 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:11:10.573 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:11:10.586 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:11:10.631 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:10.727 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T22:11:10.743 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:10.786 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:10.811 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-py. 2020-11-16T22:11:10.823 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:11:10.861 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:11:10.901 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:11:10.913 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:11:10.942 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:11:11.190 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:11:11.202 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:11.240 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:11:11.296 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:11:11.309 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:11.346 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:11.585 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:11:11.599 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:11.643 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:11:11.946 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:11:11.959 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:11:11.996 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:11:12.130 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T22:11:12.145 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-16T22:11:12.188 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T22:11:12.467 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:11:12.479 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:11:12.516 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:11:12.558 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T22:11:12.573 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T22:11:12.616 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T22:11:12.828 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:11:12.839 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:11:12.877 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:13.028 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T22:11:13.043 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T22:11:13.078 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:11:13.223 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:11:13.236 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:11:13.281 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:11:13.439 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T22:11:13.455 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T22:11:13.497 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:11:13.718 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:11:13.732 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:11:13.776 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:11:13.936 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T22:11:13.952 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T22:11:13.994 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:11:14.079 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:11:14.093 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:11:14.137 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:11:14.355 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T22:11:14.370 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T22:11:14.405 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T22:11:14.767 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T22:11:14.782 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T22:11:14.810 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:11:14.822 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:11:14.858 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:11:14.860 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:11:15.050 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:11:15.063 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:11:15.108 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:11:15.163 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:11:15.175 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:11:15.213 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:11:15.369 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:11:15.383 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:15.444 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:15.454 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T22:11:15.469 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T22:11:15.511 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:11:15.512 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T22:11:15.522 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:11:15.552 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:15.688 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:11:15.702 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:15.747 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:15.897 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:11:15.907 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T22:11:15.910 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:15.922 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T22:11:15.946 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:15.957 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T22:11:15.982 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:11:15.995 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:11:16.032 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:16.233 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:11:16.244 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:11:16.274 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:11:16.412 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:11:16.424 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:16.461 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T22:11:16.462 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:11:16.476 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T22:11:16.519 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:16.536 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:11:16.550 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:11:16.586 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:11:16.781 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:11:16.793 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:11:16.831 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:16.889 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:11:16.897 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T22:11:16.900 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:16.914 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T22:11:16.938 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:11:16.955 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:11:17.121 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:17.183 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:11:17.284 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:11:17.297 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:11:17.334 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:17.470 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-py. 2020-11-16T22:11:17.483 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:11:17.520 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:11:17.526 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T22:11:17.541 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:17.608 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:17.653 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:11:17.667 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:11:17.712 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:11:17.851 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:17.889 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:11:17.970 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T22:11:17.986 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T22:11:18.028 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T22:11:18.166 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:11:18.178 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:18.215 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:18.293 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:11:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:18.342 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:11:18.448 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T22:11:18.464 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T22:11:18.627 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:18.645 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:11:18.658 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:11:18.695 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:11:18.970 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T22:11:18.986 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:19.023 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:11:19.028 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:19.036 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:11:19.073 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:11:19.236 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:11:19.247 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:19.285 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:19.334 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:11:19.348 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:11:19.384 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:19.390 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T22:11:19.406 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T22:11:19.459 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:19.579 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:11:19.729 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:11:19.741 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:11:19.795 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:11:19.874 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:11:19.885 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T22:11:19.900 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:19.943 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:20.224 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:11:20.234 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:11:20.238 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:11:20.262 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package jq. 2020-11-16T22:11:20.279 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T22:11:20.282 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:11:20.320 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T22:11:20.360 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:11:20.585 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:11:20.597 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:11:20.623 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package socat. 2020-11-16T22:11:20.638 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T22:11:20.659 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:11:20.681 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:20.971 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:21.127 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T22:11:21.143 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T22:11:21.185 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:21.298 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:11:21.310 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:11:21.348 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:11:21.364 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:11:21.529 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T22:11:21.545 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T22:11:21.632 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:11:21.659 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:11:21.673 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:11:21.717 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:11:21.829 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T22:11:21.941 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:11:22.079 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:11:22.094 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:22.151 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:11:22.277 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:11:22.284 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-test. 2020-11-16T22:11:22.300 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:22.350 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:22.524 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:11:22.538 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:22.582 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:22.629 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:11:22.885 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:11:22.897 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:11:22.934 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:11:23.012 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:11:23.288 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:11:23.301 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:11:23.306 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:11:23.338 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:11:23.600 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:23.657 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:11:23.670 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:23.707 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:11:23.901 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:11:24.036 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:11:24.050 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:11:24.110 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:24.202 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:24.328 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:11:24.430 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:11:24.442 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:11:24.480 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:11:24.859 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:11:24.870 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:24.909 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:25.273 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:11:25.574 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:11:25.883 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:11:26.029 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:11:26.040 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:26.078 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:26.101 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:26.367 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:11:26.436 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:11:26.526 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:11:26.638 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-colorama. 2020-11-16T22:11:26.657 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-16T22:11:26.687 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-16T22:11:26.692 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:11:26.762 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:26.880 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:11:26.998 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephfs-shell. 2020-11-16T22:11:27.015 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7226-g6959059e-1bionic_all.deb ... 2020-11-16T22:11:27.048 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:11:27.051 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:11:27.081 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:27.161 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:11:27.349 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:11:27.401 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T22:11:27.417 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:27.459 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:27.651 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:11:27.755 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:27.770 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T22:11:27.786 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T22:11:27.828 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:27.977 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:11:28.140 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:11:28.241 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-attr. 2020-11-16T22:11:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:28.278 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:11:28.324 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T22:11:28.432 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:11:28.629 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:11:28.669 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T22:11:28.685 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T22:11:28.719 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:28.733 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:11:28.910 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:11:28.955 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:29.019 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:11:29.080 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T22:11:29.095 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:29.138 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T22:11:29.206 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:29.346 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:11:29.499 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-py. 2020-11-16T22:11:29.516 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-16T22:11:29.582 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T22:11:29.682 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:11:29.867 INFO:teuthology.orchestra.run.smithi185.stdout:Adding system user cephadm....done 2020-11-16T22:11:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:11:29.994 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-pytest. 2020-11-16T22:11:30.011 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:30.047 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:11:30.052 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T22:11:30.274 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:30.348 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:30.457 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:30.464 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-attr. 2020-11-16T22:11:30.480 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-16T22:11:30.522 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T22:11:30.584 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:11:30.592 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:11:30.852 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:30.867 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T22:11:30.886 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T22:11:30.925 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T22:11:30.935 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:11:30.961 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:11:31.236 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:31.278 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T22:11:31.295 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-16T22:11:31.336 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T22:11:31.362 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:11:31.647 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T22:11:31.664 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T22:11:31.697 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:31.714 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:31.914 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:11:31.998 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:11:32.109 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T22:11:32.125 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T22:11:32.167 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:11:32.223 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:11:32.274 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:11:32.392 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:11:32.508 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:11:32.679 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tz. 2020-11-16T22:11:32.696 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-16T22:11:32.737 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T22:11:32.742 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:32.751 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:33.060 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:11:33.115 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T22:11:33.120 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:33.131 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T22:11:33.132 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:11:33.173 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:11:33.209 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:11:33.241 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:11:33.362 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:33.480 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:11:33.487 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:11:33.714 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:11:33.781 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:11:33.937 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T22:11:33.955 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T22:11:33.995 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T22:11:33.998 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:11:34.065 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:11:34.299 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:11:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T22:11:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T22:11:34.417 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:11:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T22:11:34.608 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:11:34.726 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:34.869 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T22:11:34.884 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:11:34.885 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T22:11:34.919 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:11:34.926 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:35.228 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:11:35.338 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:35.347 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T22:11:35.363 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T22:11:35.426 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:11:35.429 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:35.579 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:35.672 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:11:35.808 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-py. 2020-11-16T22:11:35.823 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-16T22:11:35.849 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:11:35.865 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T22:11:35.871 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:35.975 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:11:36.235 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:11:36.260 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T22:11:36.278 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T22:11:36.335 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T22:11:36.361 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:11:36.538 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2020-11-16T22:11:36.669 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:11:36.696 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T22:11:36.712 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T22:11:36.755 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T22:11:36.762 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:11:36.888 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:11:36.962 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:37.113 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:37.165 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:11:37.200 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T22:11:37.216 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T22:11:37.239 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:11:37.258 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:37.457 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:11:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:11:37.653 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T22:11:37.668 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T22:11:37.703 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:11:37.750 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:11:37.916 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:38.042 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:11:38.051 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:11:38.257 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package radosgw. 2020-11-16T22:11:38.272 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../114-radosgw_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:38.315 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:38.360 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:38.411 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:11:38.662 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:11:38.739 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:11:39.038 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:11:39.149 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:11:39.365 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:11:39.372 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:11:39.569 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T22:11:39.584 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7226-g6959059e-1bionic_amd64.deb ... 2020-11-16T22:11:39.641 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:11:39.651 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:39.732 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:40.009 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:11:40.038 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T22:11:40.083 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:40.172 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:11:40.212 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:11:40.319 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:11:40.407 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T22:11:40.445 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:11:40.453 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:11:40.788 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:11:40.858 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T22:11:41.009 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T22:11:41.097 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:11:41.448 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:11:41.608 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:11:41.758 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:41.762 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T22:11:41.935 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:11:41.950 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:11:42.247 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T22:11:42.253 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:11:42.369 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:42.379 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:11:42.442 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:11:42.598 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T22:11:42.695 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:11:42.755 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:11:42.871 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:11:42.965 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T22:11:42.997 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:11:43.181 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:11:43.259 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T22:11:43.282 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:11:43.408 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:11:43.418 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T22:11:43.424 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:11:43.551 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:11:43.677 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:43.801 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:11:43.802 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:11:43.820 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T22:11:43.902 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:11:44.112 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:44.212 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:11:44.214 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T22:11:44.238 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:44.364 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:44.482 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:11:44.521 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:11:44.639 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T22:11:44.852 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:11:44.855 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:11:45.007 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-argparse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:45.144 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:11:45.156 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:11:45.366 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T22:11:45.479 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:11:45.541 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:11:45.734 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T22:11:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:11:45.860 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T22:11:46.126 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:11:46.332 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:11:46.445 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:11:46.787 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:11:47.084 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T22:11:47.147 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:11:47.153 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:11:47.457 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:11:47.481 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T22:11:47.488 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:47.582 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:11:47.614 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:47.841 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T22:11:47.907 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:48.008 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:48.117 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T22:11:48.218 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:11:48.218 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:11:48.493 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:48.544 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T22:11:48.619 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:48.667 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T22:11:48.745 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:11:48.754 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:11:48.835 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:11:48.875 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:11:48.970 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T22:11:49.072 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:11:49.096 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T22:11:49.173 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:11:49.331 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-16T22:11:49.415 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:11:49.541 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:11:49.698 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T22:11:49.834 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:49.900 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:11:49.976 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:11:50.100 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T22:11:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:11:50.303 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:50.429 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:50.436 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:11:50.467 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T22:11:50.555 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:50.562 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:11:50.681 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:50.688 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:50.814 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:11:50.827 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T22:11:51.140 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:51.203 INFO:teuthology.orchestra.run.smithi185.stdout:Adding group ceph....done 2020-11-16T22:11:51.266 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:51.278 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T22:11:51.384 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:51.485 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:11:51.593 INFO:teuthology.orchestra.run.smithi185.stdout:Adding system user ceph....done 2020-11-16T22:11:51.713 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T22:11:51.811 INFO:teuthology.orchestra.run.smithi185.stdout:Setting system user ceph properties....done 2020-11-16T22:11:51.871 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:11:51.959 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:11:52.072 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:52.168 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:11:52.172 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:11:52.426 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:52.507 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:11:52.691 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:11:52.692 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:11:53.070 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user cephadm....done 2020-11-16T22:11:53.077 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:53.203 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:53.254 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T22:11:53.368 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:11:53.614 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T22:11:53.739 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T22:11:53.760 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:11:53.890 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T22:11:54.031 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:54.206 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:11:54.253 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:11:54.253 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:11:54.300 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T22:11:54.507 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:54.532 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:54.625 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:54.658 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:54.668 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T22:11:54.827 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T22:11:54.934 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.035 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.061 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:11:55.061 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:11:55.228 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.243 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:11:55.243 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:11:55.336 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.478 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.528 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.621 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T22:11:55.654 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:55.780 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:11:55.842 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:11:55.843 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:11:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:11:55.902 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:11:56.039 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T22:11:56.098 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:56.165 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T22:11:56.200 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:11:56.325 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:11:56.325 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:11:56.575 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T22:11:56.599 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:56.725 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:56.827 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:56.868 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:56.969 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:11:56.994 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:11:57.026 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T22:11:57.124 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T22:11:57.163 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T22:11:57.195 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:11:57.296 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:57.422 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:57.453 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T22:11:57.564 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:57.691 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:57.867 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T22:11:58.191 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2020-11-16T22:11:58.239 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T22:11:58.602 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2020-11-16T22:11:58.665 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T22:11:58.820 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2020-11-16T22:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:11:59.033 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T22:11:59.193 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:11:59.256 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:11:59.284 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T22:11:59.291 INFO:teuthology.orchestra.run.smithi185.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:11:59.340 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:11:59.668 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:11:59.770 INFO:teuthology.orchestra.run.smithi185.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:11:59.773 INFO:teuthology.orchestra.run.smithi185:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:11:59.785 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T22:11:59.858 INFO:teuthology.orchestra.run.smithi185.stdout:Reading package lists... 2020-11-16T22:11:59.883 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T22:11:59.918 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:11:59.918 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:12:00.119 INFO:teuthology.orchestra.run.smithi185.stdout:Building dependency tree... 2020-11-16T22:12:00.119 INFO:teuthology.orchestra.run.smithi185.stdout:Reading state information... 2020-11-16T22:12:00.196 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T22:12:00.294 INFO:teuthology.orchestra.run.smithi185.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:12:00.295 INFO:teuthology.orchestra.run.smithi185.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:12:00.295 INFO:teuthology.orchestra.run.smithi185.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:12:00.295 INFO:teuthology.orchestra.run.smithi185.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:00.295 INFO:teuthology.orchestra.run.smithi185.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:00.295 INFO:teuthology.orchestra.run.smithi185.stdout:The following additional packages will be installed: 2020-11-16T22:12:00.296 INFO:teuthology.orchestra.run.smithi185.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:12:00.296 INFO:teuthology.orchestra.run.smithi185.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:12:00.296 INFO:teuthology.orchestra.run.smithi185.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:12:00.296 INFO:teuthology.orchestra.run.smithi185.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:00.296 INFO:teuthology.orchestra.run.smithi185.stdout: zlib1g-dev 2020-11-16T22:12:00.298 INFO:teuthology.orchestra.run.smithi185.stdout:Suggested packages: 2020-11-16T22:12:00.298 INFO:teuthology.orchestra.run.smithi185.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:12:00.298 INFO:teuthology.orchestra.run.smithi185.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:12:00.320 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:00.333 INFO:teuthology.orchestra.run.smithi185.stdout:The following NEW packages will be installed: 2020-11-16T22:12:00.334 INFO:teuthology.orchestra.run.smithi185.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:12:00.334 INFO:teuthology.orchestra.run.smithi185.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:12:00.335 INFO:teuthology.orchestra.run.smithi185.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:12:00.335 INFO:teuthology.orchestra.run.smithi185.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:12:00.335 INFO:teuthology.orchestra.run.smithi185.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:00.335 INFO:teuthology.orchestra.run.smithi185.stdout: zlib1g-dev 2020-11-16T22:12:00.396 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T22:12:00.471 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:00.523 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T22:12:00.533 INFO:teuthology.orchestra.run.smithi185.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:12:00.534 INFO:teuthology.orchestra.run.smithi185.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:12:00.534 INFO:teuthology.orchestra.run.smithi185.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:12:00.534 INFO:teuthology.orchestra.run.smithi185.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:12:00.807 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T22:12:00.808 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:12:00.942 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T22:12:00.990 INFO:teuthology.orchestra.run.smithi185.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:12:01.070 INFO:teuthology.orchestra.run.smithi185.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:12:01.097 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:01.119 INFO:teuthology.orchestra.run.smithi185.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:12:01.138 INFO:teuthology.orchestra.run.smithi185.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:12:01.148 INFO:teuthology.orchestra.run.smithi185.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:12:01.152 INFO:teuthology.orchestra.run.smithi185.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:12:01.152 INFO:teuthology.orchestra.run.smithi185.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:12:01.170 INFO:teuthology.orchestra.run.smithi185.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:12:01.174 INFO:teuthology.orchestra.run.smithi185.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:12:01.175 INFO:teuthology.orchestra.run.smithi185.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:12:01.176 INFO:teuthology.orchestra.run.smithi185.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:12:01.229 INFO:teuthology.orchestra.run.smithi185.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:12:01.230 INFO:teuthology.orchestra.run.smithi185.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:12:01.239 INFO:teuthology.orchestra.run.smithi185.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:12:01.307 INFO:teuthology.orchestra.run.smithi185.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:12:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:12:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:12:01.319 INFO:teuthology.orchestra.run.smithi185.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:12:01.320 INFO:teuthology.orchestra.run.smithi185.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:12:01.322 INFO:teuthology.orchestra.run.smithi185.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:12:01.342 INFO:teuthology.orchestra.run.smithi185.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:12:01.342 INFO:teuthology.orchestra.run.smithi185.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:12:01.418 INFO:teuthology.orchestra.run.smithi185.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:12:01.419 INFO:teuthology.orchestra.run.smithi185.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:12:01.444 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T22:12:01.448 INFO:teuthology.orchestra.run.smithi185.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:12:01.484 INFO:teuthology.orchestra.run.smithi185.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:12:01.485 INFO:teuthology.orchestra.run.smithi185.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:12:01.512 INFO:teuthology.orchestra.run.smithi185.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:12:01.526 INFO:teuthology.orchestra.run.smithi185.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:12:01.527 INFO:teuthology.orchestra.run.smithi185.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:12:01.528 INFO:teuthology.orchestra.run.smithi185.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:12:01.541 INFO:teuthology.orchestra.run.smithi185.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:12:01.594 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T22:12:01.598 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:01.741 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:01.954 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T22:12:02.041 INFO:teuthology.orchestra.run.smithi185.stdout:Fetched 7,861 kB in 1s (6,605 kB/s) 2020-11-16T22:12:02.168 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:12:02.168 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:12:02.189 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package flex. 2020-11-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi185.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 110001 files and directories currently installed.) 2020-11-16T22:12:02.234 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:12:02.279 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:12:02.338 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T22:12:02.444 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:02.570 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:02.666 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:12:02.679 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:02.708 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:02.712 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:02.731 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T22:12:02.933 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:12:02.933 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:12:03.036 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package bison. 2020-11-16T22:12:03.048 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:03.089 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:03.107 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T22:12:03.222 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:03.426 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:12:03.426 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:12:03.480 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package dump. 2020-11-16T22:12:03.493 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:12:03.531 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:12:03.567 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T22:12:03.706 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:03.857 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:03.955 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T22:12:03.968 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package indent. 2020-11-16T22:12:03.981 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:12:04.000 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:04.010 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:12:04.102 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:12:04.279 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:12:04.291 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:12:04.306 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T22:12:04.311 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:12:04.329 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:04.581 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:12:04.595 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:12:04.640 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:04.704 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-16T22:12:04.893 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:12:04.905 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:12:04.943 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:05.088 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T22:12:05.187 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:12:05.200 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:12:05.237 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:05.574 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T22:12:05.590 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:12:05.604 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:12:05.691 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:05.934 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T22:12:05.935 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:12:05.948 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:12:05.985 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:12:06.034 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T22:12:06.255 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:12:06.266 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:06.297 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:06.407 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:12:06.456 INFO:teuthology.orchestra.run.smithi096.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:12:06.514 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:12:06.557 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:12:06.569 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:06.607 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:06.927 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:12:06.939 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:06.966 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:12:06.968 INFO:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:12:06.977 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:07.049 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2020-11-16T22:12:07.263 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:12:07.274 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:12:07.304 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:07.313 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2020-11-16T22:12:07.314 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2020-11-16T22:12:07.484 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:12:07.484 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:12:07.484 INFO:teuthology.orchestra.run.smithi096.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:12:07.485 INFO:teuthology.orchestra.run.smithi096.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:07.485 INFO:teuthology.orchestra.run.smithi096.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:07.485 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2020-11-16T22:12:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:12:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:12:07.486 INFO:teuthology.orchestra.run.smithi096.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:12:07.486 INFO:teuthology.orchestra.run.smithi096.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:07.486 INFO:teuthology.orchestra.run.smithi096.stdout: zlib1g-dev 2020-11-16T22:12:07.487 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2020-11-16T22:12:07.487 INFO:teuthology.orchestra.run.smithi096.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:12:07.487 INFO:teuthology.orchestra.run.smithi096.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:12:07.522 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2020-11-16T22:12:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T22:12:07.523 INFO:teuthology.orchestra.run.smithi096.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:12:07.523 INFO:teuthology.orchestra.run.smithi096.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:12:07.524 INFO:teuthology.orchestra.run.smithi096.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:12:07.524 INFO:teuthology.orchestra.run.smithi096.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:12:07.524 INFO:teuthology.orchestra.run.smithi096.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:07.524 INFO:teuthology.orchestra.run.smithi096.stdout: zlib1g-dev 2020-11-16T22:12:07.616 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:12:07.628 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:07.668 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:07.706 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:12:07.706 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:12:07.707 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:12:07.707 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:12:07.949 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T22:12:08.004 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:12:08.015 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:08.054 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:08.149 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:12:08.231 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:12:08.286 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:12:08.305 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:12:08.312 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:12:08.314 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:12:08.316 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:12:08.323 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:12:08.365 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T22:12:08.365 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:12:08.365 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:12:08.365 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:12:08.365 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:12:08.366 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:08.373 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:08.392 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:12:08.392 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:12:08.451 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T22:12:08.472 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:12:08.472 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:12:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:12:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:12:08.487 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:12:08.506 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:12:08.507 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:12:08.585 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:12:08.585 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:12:08.611 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:12:08.648 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:12:08.649 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:12:08.671 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:12:08.692 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:12:08.692 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:12:08.693 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:12:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:12:08.752 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:12:08.764 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:08.802 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:08.944 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-16T22:12:09.121 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:12:09.135 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 7,861 kB in 1s (6,724 kB/s) 2020-11-16T22:12:09.202 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:09.312 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T22:12:09.346 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package flex. 2020-11-16T22:12:09.385 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 110001 files and directories currently installed.) 2020-11-16T22:12:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T22:12:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librados2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs-dev (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rados (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:12:13.622 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:13.623 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:12:13.624 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package ppp. 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:13.625 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package bison. 2020-11-16T22:12:13.626 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package dump. 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package indent. 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:12:13.627 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:12:13.628 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:12:13.629 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:13.630 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:12:13.630 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:12:13.630 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:12:13.716 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:12:13.729 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:12:13.774 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:13.819 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:12:13.833 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:13.869 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:14.114 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:12:14.126 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:14.184 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:14.312 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T22:12:14.491 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:12:14.504 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:12:14.505 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:14.515 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:12:14.549 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:14.555 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:12:14.705 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libradosstriper1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:14.773 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:12:14.784 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:12:14.814 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:14.831 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cephfs (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:14.902 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:12:14.916 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:12:14.960 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:15.193 INFO:teuthology.orchestra.run.smithi185.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:12:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librgw2 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:15.204 INFO:teuthology.orchestra.run.smithi185.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:12:15.252 INFO:teuthology.orchestra.run.smithi185.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:15.339 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:12:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:15.352 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:15.389 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:15.523 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:15.596 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:12:15.596 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T22:12:15.640 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:15.750 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:12:15.763 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:15.800 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:16.152 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librbd1 (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:16.161 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:12:16.174 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:16.211 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:16.303 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephfs-shell (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:16.429 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T22:12:16.448 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:12:16.506 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:12:16.520 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:16.552 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T22:12:16.564 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:16.579 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:12:16.585 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T22:12:16.705 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:16.831 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:16.931 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:16.966 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T22:12:16.983 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:17.092 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:17.186 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:12:17.199 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:12:17.218 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:17.387 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:17.739 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:12:17.751 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:12:17.773 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:17.774 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:17.789 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:17.827 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-modules-core (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:18.040 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:18.150 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:12:18.154 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T22:12:18.157 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:18.164 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:18.218 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:18.267 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:18.435 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:18.555 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:18.569 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:18.665 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up rbd-fuse (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:18.695 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rbd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:18.821 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:18.856 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:12:18.868 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:12:18.917 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-common (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:18.947 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:19.041 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:19.073 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:19.163 INFO:teuthology.orchestra.run.smithi185.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:12:19.368 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:19.495 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:19.553 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:12:19.555 INFO:teuthology.orchestra.run.smithi062.stdout:Adding group ceph....done 2020-11-16T22:12:19.565 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:12:19.621 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:19.713 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:19.746 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:19.873 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:19.998 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:20.045 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user ceph....done 2020-11-16T22:12:20.141 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:12:20.154 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:20.239 INFO:teuthology.orchestra.run.smithi062.stdout:Setting system user ceph properties....done 2020-11-16T22:12:20.304 INFO:teuthology.orchestra.run.smithi185.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:12:20.345 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:20.485 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T22:12:20.540 INFO:teuthology.orchestra.run.smithi185.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:12:20.660 INFO:teuthology.orchestra.run.smithi185.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:12:20.701 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:20.826 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T22:12:20.828 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:20.954 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:20.973 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ppp. 2020-11-16T22:12:20.986 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:12:21.080 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:12:21.206 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:21.293 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:21.410 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up radosgw (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:21.628 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:12:21.642 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:12:21.678 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:21.761 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:12:21.762 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T22:12:21.788 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:12:21.788 INFO:teuthology.orchestra.run.smithi185.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:12:21.789 INFO:teuthology.orchestra.run.smithi185.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:12:21.789 INFO:teuthology.orchestra.run.smithi185.stdout:- enp3s0f1 2020-11-16T22:12:21.789 INFO:teuthology.orchestra.run.smithi185.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:12:21.789 INFO:teuthology.orchestra.run.smithi185.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:12:21.789 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:12:21.919 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:12:22.106 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:12:22.118 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:12:22.136 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:12:22.164 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:12:22.362 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:12:22.400 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:12:22.412 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:12:22.459 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:22.579 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-test (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:22.730 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-base (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:22.837 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:12:22.851 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:12:22.860 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:12:22.922 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:23.037 INFO:teuthology.orchestra.run.smithi185.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:23.176 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T22:12:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:23.300 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:12:23.300 INFO:teuthology.orchestra.run.smithi185.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:12:23.342 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:23.631 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:12:23.766 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:23.840 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:12:24.036 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:12:24.036 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T22:12:24.228 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:12:24.365 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:12:24.482 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:24.600 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:24.633 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:12:24.742 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:12:24.760 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-osd (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:24.810 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:24.936 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:25.062 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:25.230 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:25.461 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:12:25.461 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T22:12:25.823 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:25.824 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:26.048 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-cephadm (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:26.110 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:26.205 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-rook (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:26.236 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:26.381 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mds (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:26.386 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:26.518 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:12:26.563 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:26.627 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:12:26.656 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:12:26.656 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T22:12:26.689 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:12:26.908 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:27.012 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:12:27.042 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:27.113 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:12:27.168 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:27.224 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mon (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:27.291 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:12:27.361 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:27.389 INFO:teuthology.orchestra.run.smithi185.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:12:27.480 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:12:27.480 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T22:12:27.487 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:27.613 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:27.739 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:27.840 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:27.854 INFO:teuthology.orchestra.run.smithi185.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:12:27.856 DEBUG:teuthology.parallel:result is None 2020-11-16T22:12:27.966 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:28.034 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:28.244 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-dashboard (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:28.329 INFO:teuthology.orchestra.run.smithi096.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:12:28.423 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph (16.0.0-7226-g6959059e-1bionic) ... 2020-11-16T22:12:28.555 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:12:28.763 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:28.873 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:12:28.889 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:29.015 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:29.133 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:12:29.259 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:29.892 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:12:29.893 INFO:teuthology.orchestra.run.smithi096.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:12:29.893 INFO:teuthology.orchestra.run.smithi096.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:12:29.893 INFO:teuthology.orchestra.run.smithi096.stdout:- enp3s0f1 2020-11-16T22:12:29.893 INFO:teuthology.orchestra.run.smithi096.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:12:29.893 INFO:teuthology.orchestra.run.smithi096.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:12:29.894 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:12:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:12:30.238 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:12:30.477 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:12:31.013 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:12:31.173 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:31.418 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:12:31.419 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:12:31.760 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:12:31.943 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:12:32.417 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:12:32.454 INFO:teuthology.orchestra.run.smithi062.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T22:12:32.517 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:12:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:12:33.272 INFO:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-16T22:12:33.406 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2020-11-16T22:12:33.671 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2020-11-16T22:12:33.672 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2020-11-16T22:12:33.854 INFO:teuthology.orchestra.run.smithi062.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2020-11-16T22:12:33.855 INFO:teuthology.orchestra.run.smithi062.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-16T22:12:33.856 INFO:teuthology.orchestra.run.smithi062.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-16T22:12:33.856 INFO:teuthology.orchestra.run.smithi062.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-16T22:12:33.856 INFO:teuthology.orchestra.run.smithi062.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:33.856 INFO:teuthology.orchestra.run.smithi062.stdout: zlib1g-dev 2020-11-16T22:12:33.857 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2020-11-16T22:12:33.857 INFO:teuthology.orchestra.run.smithi062.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-16T22:12:33.858 INFO:teuthology.orchestra.run.smithi062.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-16T22:12:33.913 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2020-11-16T22:12:33.913 INFO:teuthology.orchestra.run.smithi062.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-16T22:12:33.914 INFO:teuthology.orchestra.run.smithi062.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-16T22:12:33.914 INFO:teuthology.orchestra.run.smithi062.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-16T22:12:33.914 INFO:teuthology.orchestra.run.smithi062.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-16T22:12:33.915 INFO:teuthology.orchestra.run.smithi062.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-16T22:12:33.915 INFO:teuthology.orchestra.run.smithi062.stdout: zlib1g-dev 2020-11-16T22:12:34.111 INFO:teuthology.orchestra.run.smithi062.stdout:0 upgraded, 31 newly installed, 0 to remove and 4 not upgraded. 2020-11-16T22:12:34.111 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 7,861 kB of archives. 2020-11-16T22:12:34.111 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-16T22:12:34.111 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-16T22:12:34.304 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:12:34.396 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:12:34.558 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-16T22:12:34.638 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-16T22:12:34.691 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-16T22:12:34.709 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-16T22:12:34.719 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-16T22:12:34.721 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-16T22:12:34.722 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-16T22:12:34.738 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-16T22:12:34.742 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-16T22:12:34.742 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-16T22:12:34.743 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-16T22:12:34.764 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:12:34.820 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-16T22:12:34.822 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-16T22:12:34.840 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:12:34.900 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-16T22:12:34.901 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-16T22:12:34.915 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-16T22:12:34.916 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-16T22:12:34.917 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-16T22:12:34.934 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-16T22:12:34.935 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-16T22:12:35.014 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-16T22:12:35.014 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-16T22:12:35.036 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-16T22:12:35.063 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-16T22:12:35.064 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-16T22:12:35.098 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-16T22:12:35.116 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:12:35.119 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-16T22:12:35.121 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-16T22:12:35.122 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-16T22:12:35.140 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-16T22:12:35.577 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:12:35.579 DEBUG:teuthology.parallel:result is None 2020-11-16T22:12:35.844 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 7,861 kB in 1s (6,575 kB/s) 2020-11-16T22:12:36.009 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package flex. 2020-11-16T22:12:36.083 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112494 files and directories currently installed.) 2020-11-16T22:12:36.086 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-16T22:12:36.123 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking flex (2.6.4-6) ... 2020-11-16T22:12:36.652 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-16T22:12:36.669 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:36.711 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:37.114 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package bison. 2020-11-16T22:12:37.129 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-16T22:12:37.209 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:37.642 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package dump. 2020-11-16T22:12:37.659 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-16T22:12:37.700 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking dump (0.4b46-3) ... 2020-11-16T22:12:38.054 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package indent. 2020-11-16T22:12:38.070 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-16T22:12:38.129 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking indent (2.2.11-5) ... 2020-11-16T22:12:38.583 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package iputils-arping. 2020-11-16T22:12:38.599 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-16T22:12:38.641 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:38.969 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-16T22:12:38.986 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-16T22:12:39.046 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:39.331 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-16T22:12:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-16T22:12:39.406 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:39.767 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-16T22:12:39.783 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-16T22:12:39.825 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:40.220 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-16T22:12:40.237 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-16T22:12:40.337 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:40.640 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-16T22:12:40.656 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-16T22:12:40.698 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:12:41.093 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-16T22:12:41.108 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:41.151 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:41.462 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-16T22:12:41.477 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:41.537 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:41.923 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-16T22:12:41.941 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:41.981 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-16T22:12:42.424 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-16T22:12:42.485 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:42.846 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-16T22:12:42.861 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:42.896 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:43.315 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-16T22:12:43.331 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:43.397 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:43.928 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-16T22:12:43.945 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:44.003 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:44.347 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-16T22:12:44.364 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:44.397 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:44.800 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-16T22:12:44.817 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-16T22:12:44.892 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:45.211 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-16T22:12:45.229 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-16T22:12:45.269 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:45.891 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-16T22:12:45.907 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-16T22:12:45.940 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:46.384 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package modemmanager. 2020-11-16T22:12:46.401 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-16T22:12:46.491 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package wpasupplicant. 2020-11-16T22:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-16T22:12:47.648 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:48.288 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package policykit-1. 2020-11-16T22:12:48.303 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-16T22:12:48.368 INFO:teuthology.orchestra.run.smithi062.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-16T22:12:48.369 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-16T22:12:48.613 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:49.261 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package network-manager. 2020-11-16T22:12:49.276 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-16T22:12:49.377 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:50.093 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ppp. 2020-11-16T22:12:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-16T22:12:50.730 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:51.098 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package pptp-linux. 2020-11-16T22:12:51.114 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-16T22:12:51.156 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:51.634 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-16T22:12:51.652 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-16T22:12:51.716 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-16T22:12:52.037 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-16T22:12:52.047 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-16T22:12:52.087 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:52.507 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-16T22:12:52.524 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-16T22:12:52.613 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:53.121 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-16T22:12:54.147 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-16T22:12:54.444 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up flex (2.6.4-6) ... 2020-11-16T22:12:54.578 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:54.738 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up dump (0.4b46-3) ... 2020-11-16T22:12:54.859 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-16T22:12:54.931 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-16T22:12:55.065 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-16T22:12:55.183 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-16T22:12:55.376 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-16T22:12:55.832 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:55.832 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-16T22:12:56.438 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-16T22:12:56.580 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-16T22:12:56.706 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-16T22:12:56.899 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:57.026 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:12:57.151 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up indent (2.2.11-5) ... 2020-11-16T22:12:57.311 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:57.437 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-16T22:12:57.571 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-16T22:12:57.652 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-16T22:12:57.723 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-16T22:12:57.874 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-16T22:12:58.000 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-16T22:12:58.168 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-16T22:12:58.302 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-16T22:12:58.462 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-16T22:12:58.812 INFO:teuthology.orchestra.run.smithi062.stdout:Removed /run/systemd/system/polkit.service. 2020-11-16T22:12:59.304 INFO:teuthology.orchestra.run.smithi062.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-16T22:12:59.581 INFO:teuthology.orchestra.run.smithi062.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-16T22:12:59.640 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:12:59.791 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-16T22:12:59.901 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-16T22:13:00.052 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-16T22:13:00.186 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-16T22:13:01.109 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-16T22:13:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:which means they are currently configured by ifupdown: 2020-11-16T22:13:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:- enp3s0f1 2020-11-16T22:13:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-16T22:13:01.110 INFO:teuthology.orchestra.run.smithi062.stdout:remove their configuration from /etc/network/interfaces. 2020-11-16T22:13:01.111 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:01.286 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-16T22:13:01.649 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-16T22:13:02.055 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-16T22:13:03.132 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-16T22:13:03.292 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-16T22:13:03.613 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:13:03.613 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-16T22:13:04.395 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T22:13:04.754 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T22:13:08.681 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-16T22:13:08.781 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-16T22:13:09.225 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T22:13:09.342 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-16T22:13:09.752 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T22:13:10.473 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T22:13:10.480 DEBUG:teuthology.parallel:result is None 2020-11-16T22:13:10.481 INFO:teuthology.orchestra.run.smithi185:> sudo lsb_release -is 2020-11-16T22:13:10.537 INFO:teuthology.orchestra.run.smithi185.stdout:Ubuntu 2020-11-16T22:13:10.542 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:13:10.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:13:10.713 INFO:teuthology.orchestra.run.smithi185:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:13:10.736 INFO:teuthology.orchestra.run.smithi185.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:10.736 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:10.737 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:13:10.737 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-11-16T22:13:10.795 INFO:teuthology.orchestra.run.smithi096.stdout:Ubuntu 2020-11-16T22:13:10.800 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:13:10.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:13:10.972 INFO:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:13:10.996 INFO:teuthology.orchestra.run.smithi096.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:10.997 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:10.997 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:13:10.997 INFO:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2020-11-16T22:13:11.076 INFO:teuthology.orchestra.run.smithi062.stdout:Ubuntu 2020-11-16T22:13:11.085 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T22:13:11.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=6959059e6aae2fb5241382a4eeeca62785867604 2020-11-16T22:13:11.249 INFO:teuthology.orchestra.run.smithi062:> dpkg-query -W -f '${Version}' ceph 2020-11-16T22:13:11.285 INFO:teuthology.orchestra.run.smithi062.stdout:16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:11.286 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7226-g6959059e-1bionic 2020-11-16T22:13:11.286 INFO:teuthology.task.install:The correct ceph version 16.0.0-7226-g6959059e-1bionic is installed. 2020-11-16T22:13:11.286 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-16T22:13:11.287 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:11.287 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.292 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.301 INFO:teuthology.orchestra.run.smithi185.stderr:24+1 records in 2020-11-16T22:13:11.302 INFO:teuthology.orchestra.run.smithi185.stderr:24+1 records out 2020-11-16T22:13:11.302 INFO:teuthology.orchestra.run.smithi185.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.00022441 s, 56.4 MB/s 2020-11-16T22:13:11.303 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:11.303 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.307 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:24+1 records in 2020-11-16T22:13:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:24+1 records out 2020-11-16T22:13:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000219547 s, 57.7 MB/s 2020-11-16T22:13:11.317 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:11.318 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.340 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T22:13:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:24+1 records in 2020-11-16T22:13:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:24+1 records out 2020-11-16T22:13:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000679238 s, 18.6 MB/s 2020-11-16T22:13:11.357 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-16T22:13:11.357 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:11.358 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.362 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.371 INFO:teuthology.orchestra.run.smithi185.stderr:5+1 records in 2020-11-16T22:13:11.371 INFO:teuthology.orchestra.run.smithi185.stderr:5+1 records out 2020-11-16T22:13:11.371 INFO:teuthology.orchestra.run.smithi185.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.9366e-05 s, 30.8 MB/s 2020-11-16T22:13:11.372 INFO:teuthology.orchestra.run.smithi185:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:13:11.430 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:11.430 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.435 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.443 INFO:teuthology.orchestra.run.smithi096.stderr:5+1 records in 2020-11-16T22:13:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:5+1 records out 2020-11-16T22:13:11.444 INFO:teuthology.orchestra.run.smithi096.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.8087e-05 s, 31.2 MB/s 2020-11-16T22:13:11.445 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:13:11.503 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:11.504 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.518 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T22:13:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:5+1 records in 2020-11-16T22:13:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:5+1 records out 2020-11-16T22:13:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.000296036 s, 10.3 MB/s 2020-11-16T22:13:11.534 INFO:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T22:13:11.562 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-16T22:13:11.562 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:11.563 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.567 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.576 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:11.576 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:11.576 INFO:teuthology.orchestra.run.smithi185.stderr:299 bytes copied, 7.0403e-05 s, 4.2 MB/s 2020-11-16T22:13:11.577 INFO:teuthology.orchestra.run.smithi185:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:13:11.634 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:11.634 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.639 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.647 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:11.648 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:11.648 INFO:teuthology.orchestra.run.smithi096.stderr:299 bytes copied, 6.7916e-05 s, 4.4 MB/s 2020-11-16T22:13:11.648 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:13:11.708 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:11.708 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T22:13:11.736 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:11.736 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:11.736 INFO:teuthology.orchestra.run.smithi062.stderr:299 bytes copied, 0.000223137 s, 1.3 MB/s 2020-11-16T22:13:11.738 INFO:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T22:13:11.764 INFO:teuthology.run_tasks:Running task ceph... 2020-11-16T22:13:11.831 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-16T22:13:11.832 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2020-11-16T22:13:11.838 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2020-11-16T22:13:11.840 INFO:teuthology.orchestra.run.smithi185:> sudo chmod 777 /var/log/ceph 2020-11-16T22:13:11.861 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-16T22:13:11.861 INFO:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:13:11.866 INFO:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:13:11.895 INFO:teuthology.orchestra.run.smithi185:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T22:13:11.909 INFO:tasks.ceph:Creating extra log directories... 2020-11-16T22:13:11.910 INFO:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:13:11.936 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:13:11.955 INFO:teuthology.orchestra.run.smithi185:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T22:13:11.976 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-16T22:13:11.977 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-16T22:13:11.977 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-16T22:13:11.977 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:11.978 INFO:teuthology.orchestra.run.smithi185:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.015 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.017 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:12.017 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:12.017 INFO:teuthology.orchestra.run.smithi185.stderr:408 bytes copied, 6.5543e-05 s, 6.2 MB/s 2020-11-16T22:13:12.018 INFO:teuthology.orchestra.run.smithi185:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:12.098 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:12.099 INFO:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.103 INFO:teuthology.orchestra.run.smithi096.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.105 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:12.105 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:12.105 INFO:teuthology.orchestra.run.smithi096.stderr:408 bytes copied, 7.5524e-05 s, 5.4 MB/s 2020-11-16T22:13:12.106 INFO:teuthology.orchestra.run.smithi096:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:12.181 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:12.181 INFO:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.195 INFO:teuthology.orchestra.run.smithi062.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-16T22:13:12.199 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:12.199 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:12.199 INFO:teuthology.orchestra.run.smithi062.stderr:408 bytes copied, 0.000246947 s, 1.7 MB/s 2020-11-16T22:13:12.200 INFO:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:12.329 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-16T22:13:12.329 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-16T22:13:12.329 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi/5631936', 'branch': 'wip-pdonnell-testing-20201116.174825', 'description': 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5631936', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2020-11-16_20:59:43-multimds-wip-pdonnell-testing-20201116.174825-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20201116.174825'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Scrub error on inode'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20201116.174825', 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mgr.x', 'mds.b', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0', 'client.1']], 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604', 'sleep_before_teardown': 0, 'suite': 'multimds', 'suite_branch': 'wip-pdonnell-testing-20201116.174825', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201116.174825/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '6959059e6aae2fb5241382a4eeeca62785867604', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (3, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '6959059e6aae2fb5241382a4eeeca62785867604'}}, {'ceph': None}, {'kclient': None}, {'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_20:59:43', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.24835', 'targets': {'smithi185.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCf2u2VpfI2OiGTlXbLsu5ah2Tn+IfD7QHUuqb8apbpZ/qrIxfqGvWWkCbsXA5sp2l0Nd120DAz5CRiY78zSD9zO8nUnYQYaTE/pYHsnpL7FAZ0dpF6r3Wx9kFYj5HwwUubBjZYlaaYHoX8oj8DQ2I8WpC1nzseAeeO0ENup3WCrrlbdWvjSQDBCSsL1um+2uQDHOFFNs9c7Dq+ol0Y6BxvOhJVJOlNuQQc3Hjw9fq0PrzLyCmTz+27MxtQtm8v1Z+HECsaLi3MmRAXaNPTR83FDiH5Fn97OR1KrJqg9AE9WtIZ7OKkRYo7L1s+jeMH9N8ISTFerXVljuzipatSbyih', 'smithi096.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzjvIZwUJ9GPhyZcPgODoG47eKuYPW5cY/P45M3uons0W9I8NkMymG1KWSSFO26xCnAT1VU7mXSS7BRrFKBno+3dzDJnc/ReYU45HEgrA/08JenVvvUvngKYkzIzYtvIdYIZc38kNaI6WBMp8QehgyfVLnqMZOvsqss4mykZU7N+FoyAjBSH9we51opWLo1chcffaMNS43Iaz3hChrh6F01uUg5JYy+EJu6DvDIjtdH5R3UpfDPXxmls1XY0yvd4lWnZKTvxoZR8IUKIxWgk2wVobiDTsvllHMQXnpt0WoLcbvOYxcPkWDx0+VG4k8+V0cANE53w56La877vmgYcsN', 'smithi062.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuzY3IcuYwznKE0zVDwYqPet1njx9kdFtD64h4NQk/zw9+xuDbhlL2OkwyhWwM/sccB0Pd2eAA3T9DJ/6a/aa+P9/Th1NpgiCMmObrXOV2bpjJNeMP32YQxpF6VznvHurtM0KD/4143QuOPmKEvCyQ9y25LlksWu896WmUtcD/o71i+2/qk3g0dQu1SCWXAAqu8CynJbSUrWvK2o5L/z+M7qZER9r3TUXwtvu/FnKlgm6iJCUSxFuFRfvLQT2mdV5vCL+iwaFRC/OFCwPzXEYpYmg14ePfno2udmsWIg30FzW+bH0YTqFuTPLbXi7Y6PljKcdNHWhzmUYU0vFwz3mb'}} 2020-11-16T22:13:12.330 INFO:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:13:12.336 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:13:12.338 INFO:teuthology.orchestra.run.smithi185:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T22:13:12.346 INFO:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:13:12.352 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:13:12.387 INFO:teuthology.orchestra.run.smithi185:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T22:13:12.402 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:12.402 INFO:teuthology.orchestra.run.smithi185:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:13:12.447 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:13:12.449 INFO:teuthology.orchestra.run.smithi185.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:13:12.449 INFO:teuthology.orchestra.run.smithi185.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:13:12.449 INFO:teuthology.orchestra.run.smithi185.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:13:12.449 INFO:teuthology.orchestra.run.smithi185.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:13:12.449 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:12.450 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:12.450 INFO:teuthology.orchestra.run.smithi185.stderr:72 bytes copied, 0.000149063 s, 483 kB/s 2020-11-16T22:13:12.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:13:12.450 INFO:teuthology.orchestra.run.smithi185:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:13:12.498 INFO:teuthology.orchestra.run.smithi185.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:13:12.498 INFO:teuthology.orchestra.run.smithi185.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.498 INFO:teuthology.orchestra.run.smithi185.stdout:Device: 6h/6d Inode: 339 Links: 1 2020-11-16T22:13:12.499 INFO:teuthology.orchestra.run.smithi185.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.499 INFO:teuthology.orchestra.run.smithi185.stdout:Access: 2020-11-16 22:09:10.103893633 +0000 2020-11-16T22:13:12.499 INFO:teuthology.orchestra.run.smithi185.stdout:Modify: 2020-11-16 22:09:10.027893635 +0000 2020-11-16T22:13:12.499 INFO:teuthology.orchestra.run.smithi185.stdout:Change: 2020-11-16 22:09:10.027893635 +0000 2020-11-16T22:13:12.499 INFO:teuthology.orchestra.run.smithi185.stdout: Birth: - 2020-11-16T22:13:12.507 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records in 2020-11-16T22:13:12.507 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records out 2020-11-16T22:13:12.507 INFO:teuthology.orchestra.run.smithi185.stderr:512 bytes copied, 0.000395793 s, 1.3 MB/s 2020-11-16T22:13:12.511 INFO:teuthology.orchestra.run.smithi185:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:13:12.558 INFO:teuthology.orchestra.run.smithi185.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:13:12.558 INFO:teuthology.orchestra.run.smithi185.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.558 INFO:teuthology.orchestra.run.smithi185.stdout:Device: 6h/6d Inode: 346 Links: 1 2020-11-16T22:13:12.558 INFO:teuthology.orchestra.run.smithi185.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.558 INFO:teuthology.orchestra.run.smithi185.stdout:Access: 2020-11-16 22:09:10.335893628 +0000 2020-11-16T22:13:12.559 INFO:teuthology.orchestra.run.smithi185.stdout:Modify: 2020-11-16 22:09:10.227893631 +0000 2020-11-16T22:13:12.559 INFO:teuthology.orchestra.run.smithi185.stdout:Change: 2020-11-16 22:09:10.227893631 +0000 2020-11-16T22:13:12.559 INFO:teuthology.orchestra.run.smithi185.stdout: Birth: - 2020-11-16T22:13:12.567 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records in 2020-11-16T22:13:12.567 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records out 2020-11-16T22:13:12.568 INFO:teuthology.orchestra.run.smithi185.stderr:512 bytes copied, 0.000320344 s, 1.6 MB/s 2020-11-16T22:13:12.571 INFO:teuthology.orchestra.run.smithi185:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:13:12.618 INFO:teuthology.orchestra.run.smithi185.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:13:12.618 INFO:teuthology.orchestra.run.smithi185.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.618 INFO:teuthology.orchestra.run.smithi185.stdout:Device: 6h/6d Inode: 352 Links: 1 2020-11-16T22:13:12.618 INFO:teuthology.orchestra.run.smithi185.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.619 INFO:teuthology.orchestra.run.smithi185.stdout:Access: 2020-11-16 22:09:10.583893623 +0000 2020-11-16T22:13:12.619 INFO:teuthology.orchestra.run.smithi185.stdout:Modify: 2020-11-16 22:09:10.483893625 +0000 2020-11-16T22:13:12.619 INFO:teuthology.orchestra.run.smithi185.stdout:Change: 2020-11-16 22:09:10.483893625 +0000 2020-11-16T22:13:12.619 INFO:teuthology.orchestra.run.smithi185.stdout: Birth: - 2020-11-16T22:13:12.627 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records in 2020-11-16T22:13:12.627 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records out 2020-11-16T22:13:12.628 INFO:teuthology.orchestra.run.smithi185.stderr:512 bytes copied, 0.000377081 s, 1.4 MB/s 2020-11-16T22:13:12.631 INFO:teuthology.orchestra.run.smithi185:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:13:12.678 INFO:teuthology.orchestra.run.smithi185.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:13:12.678 INFO:teuthology.orchestra.run.smithi185.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.678 INFO:teuthology.orchestra.run.smithi185.stdout:Device: 6h/6d Inode: 358 Links: 1 2020-11-16T22:13:12.678 INFO:teuthology.orchestra.run.smithi185.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.679 INFO:teuthology.orchestra.run.smithi185.stdout:Access: 2020-11-16 22:09:10.815893618 +0000 2020-11-16T22:13:12.679 INFO:teuthology.orchestra.run.smithi185.stdout:Modify: 2020-11-16 22:09:10.739893619 +0000 2020-11-16T22:13:12.679 INFO:teuthology.orchestra.run.smithi185.stdout:Change: 2020-11-16 22:09:10.739893619 +0000 2020-11-16T22:13:12.679 INFO:teuthology.orchestra.run.smithi185.stdout: Birth: - 2020-11-16T22:13:12.687 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records in 2020-11-16T22:13:12.687 INFO:teuthology.orchestra.run.smithi185.stderr:1+0 records out 2020-11-16T22:13:12.688 INFO:teuthology.orchestra.run.smithi185.stderr:512 bytes copied, 0.00039513 s, 1.3 MB/s 2020-11-16T22:13:12.691 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:12.692 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:12.692 INFO:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:13:12.696 INFO:teuthology.orchestra.run.smithi096.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T22:13:12.698 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/vg_nvme/lv_1 2020-11-16T22:13:12.698 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/vg_nvme/lv_2 2020-11-16T22:13:12.698 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/vg_nvme/lv_3 2020-11-16T22:13:12.698 INFO:teuthology.orchestra.run.smithi096.stdout:/dev/vg_nvme/lv_4 2020-11-16T22:13:12.699 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:12.699 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:12.699 INFO:teuthology.orchestra.run.smithi096.stderr:72 bytes copied, 0.000165274 s, 436 kB/s 2020-11-16T22:13:12.699 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T22:13:12.699 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T22:13:12.748 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T22:13:12.748 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.748 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-16T22:13:12.749 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.749 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-11-16 22:09:10.089137136 +0000 2020-11-16T22:13:12.749 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-11-16 22:09:10.005137135 +0000 2020-11-16T22:13:12.749 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-11-16 22:09:10.005137135 +0000 2020-11-16T22:13:12.749 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-11-16T22:13:12.758 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-11-16T22:13:12.758 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-11-16T22:13:12.758 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000394841 s, 1.3 MB/s 2020-11-16T22:13:12.762 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T22:13:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T22:13:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-11-16 22:09:10.325137138 +0000 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-11-16 22:09:10.233137137 +0000 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-11-16 22:09:10.233137137 +0000 2020-11-16T22:13:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-11-16T22:13:12.817 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-11-16T22:13:12.818 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-11-16T22:13:12.818 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000358147 s, 1.4 MB/s 2020-11-16T22:13:12.821 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T22:13:12.868 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T22:13:12.868 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.868 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-11-16T22:13:12.869 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.869 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-11-16 22:09:10.561137140 +0000 2020-11-16T22:13:12.869 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-11-16 22:09:10.457137139 +0000 2020-11-16T22:13:12.869 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-11-16 22:09:10.457137139 +0000 2020-11-16T22:13:12.869 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-11-16T22:13:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-11-16T22:13:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-11-16T22:13:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000386481 s, 1.3 MB/s 2020-11-16T22:13:12.883 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T22:13:12.930 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-11-16 22:09:10.797137142 +0000 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-11-16 22:09:10.689137141 +0000 2020-11-16T22:13:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-11-16 22:09:10.689137141 +0000 2020-11-16T22:13:12.932 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-11-16T22:13:12.939 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-11-16T22:13:12.939 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-11-16T22:13:12.939 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000281728 s, 1.8 MB/s 2020-11-16T22:13:12.943 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:12.944 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi185.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T22:13:12.944 INFO:tasks.ceph:Generating config... 2020-11-16T22:13:12.945 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-16T22:13:12.946 INFO:tasks.ceph:[client] debug client = 20 2020-11-16T22:13:12.946 INFO:tasks.ceph:[client] debug ms = 1 2020-11-16T22:13:12.946 INFO:tasks.ceph:[client] fuse default permissions = False 2020-11-16T22:13:12.946 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-16T22:13:12.946 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-16T22:13:12.947 INFO:tasks.ceph:[global] ms die on skipped message = False 2020-11-16T22:13:12.947 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-16T22:13:12.947 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-16T22:13:12.947 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-16T22:13:12.947 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-16T22:13:12.948 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-16T22:13:12.948 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-16T22:13:12.948 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-16T22:13:12.948 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-16T22:13:12.948 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-16T22:13:12.949 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-16T22:13:12.949 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-16T22:13:12.949 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-16T22:13:12.949 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-16T22:13:12.949 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-16T22:13:12.950 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-16T22:13:12.950 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-16T22:13:12.950 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-16T22:13:12.950 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-16T22:13:12.950 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-16T22:13:12.951 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-16T22:13:12.951 INFO:tasks.ceph:[osd] bdev async discard = True 2020-11-16T22:13:12.951 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-11-16T22:13:12.951 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2020-11-16T22:13:12.951 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-16T22:13:12.952 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-16T22:13:12.952 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-16T22:13:12.952 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-16T22:13:12.952 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-16T22:13:12.952 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-16T22:13:12.953 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-16T22:13:12.953 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-16T22:13:12.953 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-16T22:13:12.953 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-16T22:13:12.953 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-16T22:13:12.954 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-16T22:13:12.954 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-16T22:13:12.954 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-16T22:13:12.954 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-16T22:13:12.954 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2020-11-16T22:13:12.955 INFO:tasks.ceph:Setting up mon.a... 2020-11-16T22:13:12.955 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-16T22:13:12.987 INFO:teuthology.orchestra.run.smithi185.stdout:creating /etc/ceph/ceph.keyring 2020-11-16T22:13:12.989 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-16T22:13:13.066 INFO:teuthology.orchestra.run.smithi185:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.125 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.185'), ('mon.c', '[v2:172.21.15.185:3301,v1:172.21.15.185:6790]'), ('mon.b', '172.21.15.96')] 2020-11-16T22:13:13.126 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.185,[v2:172.21.15.185:3301,v1:172.21.15.185:6790],172.21.15.96', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}} 2020-11-16T22:13:13.127 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:13.127 INFO:teuthology.orchestra.run.smithi185:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:13:13.171 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:13:13.173 INFO:teuthology.orchestra.run.smithi185.stderr:7+1 records in 2020-11-16T22:13:13.173 INFO:teuthology.orchestra.run.smithi185.stderr:7+1 records out 2020-11-16T22:13:13.173 INFO:teuthology.orchestra.run.smithi185.stderr:3613 bytes (3.6 kB, 3.5 KiB) copied, 0.000112214 s, 32.2 MB/s 2020-11-16T22:13:13.174 INFO:teuthology.orchestra.run.smithi185:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.185 --addv c '[v2:172.21.15.185:3301,v1:172.21.15.185:6790]' --add b 172.21.15.96 --print /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.253 INFO:teuthology.orchestra.run.smithi185.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:monmaptool: generated fsid 7d23284a-d49b-4d10-9d1b-194105919b4f 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:epoch 0 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:fsid 7d23284a-d49b-4d10-9d1b-194105919b4f 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:last_changed 2020-11-16T22:13:13.274217+0000 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:created 2020-11-16T22:13:13.274217+0000 2020-11-16T22:13:13.254 INFO:teuthology.orchestra.run.smithi185.stdout:min_mon_release 0 (unknown) 2020-11-16T22:13:13.255 INFO:teuthology.orchestra.run.smithi185.stdout:election_strategy: 1 2020-11-16T22:13:13.255 INFO:teuthology.orchestra.run.smithi185.stdout:0: [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] mon.a 2020-11-16T22:13:13.255 INFO:teuthology.orchestra.run.smithi185.stdout:1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2020-11-16T22:13:13.255 INFO:teuthology.orchestra.run.smithi185.stdout:2: [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] mon.c 2020-11-16T22:13:13.255 INFO:teuthology.orchestra.run.smithi185.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-16T22:13:13.256 INFO:teuthology.orchestra.run.smithi185:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T22:13:13.304 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7d23284a-d49b-4d10-9d1b-194105919b4f... 2020-11-16T22:13:13.305 INFO:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:13:13.312 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:13:13.314 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T22:13:13.350 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2020-11-16T22:13:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2020-11-16T22:13:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:13:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2020-11-16T22:13:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:13:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:13:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2020-11-16T22:13:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2020-11-16T22:13:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.185,[v2:172.21.15.185:3301,v1:172.21.15.185:6790],172.21.15.96" 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on skipped message = False 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 7d23284a-d49b-4d10-9d1b-194105919b4f 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2020-11-16T22:13:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:13:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bdev async discard = True 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bdev enable discard = True 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore allocator = bitmap 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2020-11-16T22:13:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug filestore = 20 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug journal = 20 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 20 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 25 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2020-11-16T22:13:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: osd shutdown pgref assert = True 2020-11-16T22:13:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-11-16T22:13:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: mon op complaint time = 120 2020-11-16T22:13:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: client mount timeout = 600 2020-11-16T22:13:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: fuse default permissions = False 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2020-11-16T22:13:13.367 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout:[mds] 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds = 20 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal frag = True 2020-11-16T22:13:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal fragment size max = 10000 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal merge size = 5 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split bits = 3 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds bal split size = 100 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug frag = True 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug scatterstat = True 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds op complaint time = 180 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: mds verify scatter = True 2020-11-16T22:13:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2020-11-16T22:13:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.379 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2020-11-16T22:13:13.380 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2020-11-16T22:13:13.381 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:13:13.382 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2020-11-16T22:13:13.383 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:13:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2020-11-16T22:13:13.385 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.185,[v2:172.21.15.185:3301,v1:172.21.15.185:6790],172.21.15.96" 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 7d23284a-d49b-4d10-9d1b-194105919b4f 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.386 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2020-11-16T22:13:13.387 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.388 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.389 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 20 2020-11-16T22:13:13.390 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 20 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: debug filestore = 20 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: debug journal = 20 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 20 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 25 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 10 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:13:13.391 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: osd shutdown pgref assert = True 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2020-11-16T22:13:13.392 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2020-11-16T22:13:13.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2020-11-16T22:13:13.394 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2020-11-16T22:13:13.395 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2020-11-16T22:13:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: fuse default permissions = False 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2020-11-16T22:13:13.397 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal frag = True 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal fragment size max = 10000 2020-11-16T22:13:13.398 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal merge size = 5 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal split bits = 3 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal split size = 100 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2020-11-16T22:13:13.399 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.400 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.400 INFO:teuthology.orchestra.run.smithi185.stdout:[global] 2020-11-16T22:13:13.400 INFO:teuthology.orchestra.run.smithi185.stdout: chdir = "" 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: auth supported = cephx 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: filestore xattr use omap = true 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: mon clock drift allowed = 1.000 2020-11-16T22:13:13.401 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: osd crush chooseleaf type = 0 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: auth debug = true 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: ms die on old message = true 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: ms die on bug = true 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.402 INFO:teuthology.orchestra.run.smithi185.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: mon pg warn max object skew = 0 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: osd pool default size = 2 2020-11-16T22:13:13.403 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd allow primary affinity = true 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd allow pg remap = true 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon warn on legacy crush tunables = false 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon warn on crush straw calc version zero = false 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon warn on no sortbitwise = false 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon warn on osd down out interval zero = false 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon warn on too few osds = false 2020-11-16T22:13:13.404 INFO:teuthology.orchestra.run.smithi185.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: mon_allow_pool_size_one = true 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: osd default data pool replay window = 5 2020-11-16T22:13:13.405 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: mon allow pool delete = true 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: mon cluster log file level = debug 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: debug asserts on shutdown = true 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: mon health detail to clog = false 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: mon host = "172.21.15.185,[v2:172.21.15.185:3301,v1:172.21.15.185:6790],172.21.15.96" 2020-11-16T22:13:13.406 INFO:teuthology.orchestra.run.smithi185.stdout: ms die on skipped message = False 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: fsid = 7d23284a-d49b-4d10-9d1b-194105919b4f 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout:[osd] 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: osd journal size = 100 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: osd scrub load threshold = 5.0 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: osd scrub max interval = 600 2020-11-16T22:13:13.407 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: osd recover clone overlap = true 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: osd recovery max chunk = 1048576 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: osd debug shutdown = true 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: osd debug op order = true 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: osd debug verify stray on activate = true 2020-11-16T22:13:13.408 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: osd open classes on start = true 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: osd debug pg log writeout = true 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: osd deep scrub update digest min age = 30 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: osd map max advance = 10 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.409 INFO:teuthology.orchestra.run.smithi185.stdout: journal zero on create = true 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: filestore ondisk finisher threads = 3 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: filestore apply finisher threads = 3 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: bdev debug aio = true 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: osd debug misdirected ops = true 2020-11-16T22:13:13.410 INFO:teuthology.orchestra.run.smithi185.stdout: bdev async discard = True 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: bdev enable discard = True 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: bluestore allocator = bitmap 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: bluestore block size = 96636764160 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: bluestore fsck on mount = True 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: debug bluefs = 20 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: debug bluestore = 20 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: debug filestore = 20 2020-11-16T22:13:13.411 INFO:teuthology.orchestra.run.smithi185.stdout: debug journal = 20 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: debug ms = 20 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: debug osd = 25 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: debug rocksdb = 10 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd full ratio = 0.9 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd nearfull ratio = 0.8 2020-11-16T22:13:13.412 INFO:teuthology.orchestra.run.smithi185.stdout: osd failsafe full ratio = 0.95 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: osd objectstore = bluestore 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: osd op complaint time = 180 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: osd shutdown pgref assert = True 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout:[mgr] 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: debug ms = 1 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: debug mgr = 20 2020-11-16T22:13:13.413 INFO:teuthology.orchestra.run.smithi185.stdout: debug mon = 20 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: debug auth = 20 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: mgr/telemetry/nag = false 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout:[mon] 2020-11-16T22:13:13.414 INFO:teuthology.orchestra.run.smithi185.stdout: debug ms = 1 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: debug mon = 20 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: debug paxos = 20 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: debug auth = 20 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: mon data avail warn = 5 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: mon mgr mkfs grace = 240 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: mon reweight min pgs per osd = 4 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd reporter subtree level = osd 2020-11-16T22:13:13.415 INFO:teuthology.orchestra.run.smithi185.stdout: mon osd prime pg temp = true 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: mon reweight min bytes per osd = 10 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: mon op complaint time = 120 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout:[client] 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: rgw cache enabled = true 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: rgw enable ops log = true 2020-11-16T22:13:13.416 INFO:teuthology.orchestra.run.smithi185.stdout: rgw enable usage log = true 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: client mount timeout = 600 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: debug client = 20 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: debug ms = 1 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: fuse default permissions = False 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.417 INFO:teuthology.orchestra.run.smithi185.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mon.a] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mon.c] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mon.b] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.a] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.b] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.c] 2020-11-16T22:13:13.418 INFO:teuthology.orchestra.run.smithi185.stdout:[mds] 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: debug mds = 20 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: debug ms = 1 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds bal frag = True 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds bal fragment size max = 10000 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds bal merge size = 5 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds bal split bits = 3 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds bal split size = 100 2020-11-16T22:13:13.419 INFO:teuthology.orchestra.run.smithi185.stdout: mds debug frag = True 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: mds debug scatterstat = True 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: mds op complaint time = 180 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: mds verify scatter = True 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: osd op complaint time = 180 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: rados mon op timeout = 15m 2020-11-16T22:13:13.420 INFO:teuthology.orchestra.run.smithi185.stdout: rados osd op timeout = 15m 2020-11-16T22:13:13.421 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-16T22:13:13.421 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-16T22:13:13.488 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-16T22:13:13.515 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:13.516 INFO:teuthology.orchestra.run.smithi185:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:13:13.520 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T22:13:13.522 INFO:teuthology.orchestra.run.smithi185.stdout:[mon.] 2020-11-16T22:13:13.522 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB5+bJfTqv/BBAANNvHcF1cO5mfKVXUphTVaw== 2020-11-16T22:13:13.522 INFO:teuthology.orchestra.run.smithi185.stdout:[client.admin] 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ== 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stdout: caps mds = "allow *" 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stdout: caps mgr = "allow *" 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stdout: caps mon = "allow *" 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stdout: caps osd = "allow *" 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:13.523 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:13.524 INFO:teuthology.orchestra.run.smithi185.stderr:206 bytes copied, 0.000176964 s, 1.2 MB/s 2020-11-16T22:13:13.524 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:13.524 INFO:teuthology.orchestra.run.smithi185:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:13:13.567 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T22:13:13.569 INFO:teuthology.orchestra.run.smithi185.stdout: �}#(JԛM�A��Oy��_h9Xy��_h9X�aXa �����bXb �`��`cXc �����abc 2020-11-16T22:13:13.569 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:13.569 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:13.569 INFO:teuthology.orchestra.run.smithi185.stderr:409 bytes copied, 0.000162324 s, 2.5 MB/s 2020-11-16T22:13:13.570 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:13:13.570 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:13.570 INFO:teuthology.orchestra.run.smithi185:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.570 INFO:teuthology.orchestra.run.smithi185:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.616 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.625 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:13.625 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:13.625 INFO:teuthology.orchestra.run.smithi185.stderr:206 bytes copied, 0.000121997 s, 1.7 MB/s 2020-11-16T22:13:13.626 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.636 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:13.636 INFO:teuthology.orchestra.run.smithi185:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.683 INFO:teuthology.orchestra.run.smithi185.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.685 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:13.685 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:13.685 INFO:teuthology.orchestra.run.smithi185.stderr:409 bytes copied, 0.000132257 s, 3.1 MB/s 2020-11-16T22:13:13.686 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:13:13.686 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:13.686 INFO:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.686 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.691 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.700 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:13.700 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:13.700 INFO:teuthology.orchestra.run.smithi096.stderr:206 bytes copied, 6.5076e-05 s, 3.2 MB/s 2020-11-16T22:13:13.701 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.710 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:13.710 INFO:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.758 INFO:teuthology.orchestra.run.smithi096.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.759 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:13.760 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:13.760 INFO:teuthology.orchestra.run.smithi096.stderr:409 bytes copied, 7.8394e-05 s, 5.2 MB/s 2020-11-16T22:13:13.760 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:13:13.760 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:13.760 INFO:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.761 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T22:13:13.798 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:13.798 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:13.798 INFO:teuthology.orchestra.run.smithi062.stderr:206 bytes copied, 0.00022106 s, 932 kB/s 2020-11-16T22:13:13.800 INFO:teuthology.orchestra.run.smithi062.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T22:13:13.820 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:13.820 INFO:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.876 INFO:teuthology.orchestra.run.smithi062.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:13.880 INFO:teuthology.orchestra.run.smithi062.stderr:409 bytes copied, 9.3957e-05 s, 4.4 MB/s 2020-11-16T22:13:13.880 INFO:tasks.ceph:Setting up mon nodes... 2020-11-16T22:13:13.880 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-16T22:13:13.881 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:13:13.920 INFO:teuthology.orchestra.run.smithi185.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T22:13:13.922 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:13:13.963 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T22:13:13.965 INFO:tasks.ceph:Setting up mds nodes... 2020-11-16T22:13:13.965 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:13:14.010 INFO:teuthology.orchestra.run.smithi185.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-16T22:13:14.012 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-16T22:13:14.070 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-16T22:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-16T22:13:14.115 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-16T22:13:14.172 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2020-11-16T22:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2020-11-16T22:13:14.260 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2020-11-16T22:13:14.316 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-16T22:13:14.317 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-16T22:13:14.381 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-16T22:13:14.407 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-16T22:13:14.461 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-16T22:13:14.489 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-16T22:13:14.490 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi185.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T22:13:14.490 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-16T22:13:14.504 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:14.504 INFO:tasks.ceph:role: osd.0 2020-11-16T22:13:14.504 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:13:14.504 INFO:teuthology.orchestra.run.smithi185:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:13:15.019 INFO:teuthology.orchestra.run.smithi185.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:15.019 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:15.019 INFO:teuthology.orchestra.run.smithi185.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:15.019 INFO:teuthology.orchestra.run.smithi185.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:15.020 INFO:teuthology.orchestra.run.smithi185.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:15.020 INFO:teuthology.orchestra.run.smithi185.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:15.020 INFO:teuthology.orchestra.run.smithi185.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:15.020 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:15.020 INFO:teuthology.orchestra.run.smithi185.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:15.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi185.front.sepia.ceph.com -o noatime 2020-11-16T22:13:15.021 INFO:teuthology.orchestra.run.smithi185:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-16T22:13:15.102 INFO:teuthology.orchestra.run.smithi185:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-16T22:13:15.112 INFO:teuthology.orchestra.run.smithi185.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:15.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:15.113 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-16T22:13:15.169 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:15.170 INFO:tasks.ceph:role: osd.1 2020-11-16T22:13:15.170 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:13:15.170 INFO:teuthology.orchestra.run.smithi185:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T22:13:15.691 INFO:teuthology.orchestra.run.smithi185.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:15.691 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:15.691 INFO:teuthology.orchestra.run.smithi185.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:15.692 INFO:teuthology.orchestra.run.smithi185.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:15.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi185.front.sepia.ceph.com -o noatime 2020-11-16T22:13:15.694 INFO:teuthology.orchestra.run.smithi185:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-16T22:13:15.762 INFO:teuthology.orchestra.run.smithi185:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-16T22:13:15.768 INFO:teuthology.orchestra.run.smithi185.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:15.769 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-16T22:13:15.825 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:15.825 INFO:tasks.ceph:role: osd.2 2020-11-16T22:13:15.825 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:13:15.826 INFO:teuthology.orchestra.run.smithi185:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:16.321 INFO:teuthology.orchestra.run.smithi185.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:16.322 INFO:teuthology.orchestra.run.smithi185.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:16.322 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:16.322 INFO:teuthology.orchestra.run.smithi185.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:16.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi185.front.sepia.ceph.com -o noatime 2020-11-16T22:13:16.323 INFO:teuthology.orchestra.run.smithi185:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-16T22:13:16.362 INFO:teuthology.orchestra.run.smithi185:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-16T22:13:16.409 INFO:teuthology.orchestra.run.smithi185.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:16.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:16.411 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-16T22:13:16.466 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:16.466 INFO:tasks.ceph:role: osd.3 2020-11-16T22:13:16.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi185.front.sepia.ceph.com 2020-11-16T22:13:16.466 INFO:teuthology.orchestra.run.smithi185:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T22:13:16.996 INFO:teuthology.orchestra.run.smithi185.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:16.997 INFO:teuthology.orchestra.run.smithi185.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:16.998 INFO:teuthology.orchestra.run.smithi185.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:16.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi185.front.sepia.ceph.com -o noatime 2020-11-16T22:13:16.999 INFO:teuthology.orchestra.run.smithi185:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-16T22:13:17.050 INFO:teuthology.orchestra.run.smithi185:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-16T22:13:17.061 INFO:teuthology.orchestra.run.smithi185.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:17.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:17.062 INFO:teuthology.orchestra.run.smithi185:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:17.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.180+0000 7fe70af86d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-16T22:13:17.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.184+0000 7fe70af86d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-16T22:13:17.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.184+0000 7fe70af86d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:13:17.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.184+0000 7fe70af86d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:13:17.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.184+0000 7fe70af86d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T22:13:17.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:17.192+0000 7fe70af86d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-16T22:13:20.119 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-16T22:13:20.128 INFO:teuthology.orchestra.run.smithi185:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:20.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-16T22:13:20.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-16T22:13:20.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:13:20.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:13:20.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T22:13:20.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:20.210+0000 7fb1ff0c2d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-16T22:13:23.146 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-16T22:13:23.197 INFO:teuthology.orchestra.run.smithi185:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:23.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-16T22:13:23.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-16T22:13:23.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:13:23.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:13:23.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T22:13:23.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:23.285+0000 7fb05ec76d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-16T22:13:26.214 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-16T22:13:26.270 INFO:teuthology.orchestra.run.smithi185:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:26.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.355+0000 7f1d12739d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-16T22:13:26.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.359+0000 7f1d12739d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-16T22:13:26.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.359+0000 7f1d12739d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:13:26.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.359+0000 7f1d12739d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:13:26.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.359+0000 7f1d12739d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T22:13:26.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:26.359+0000 7f1d12739d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-16T22:13:29.305 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-16T22:13:29.360 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-16T22:13:29.375 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:29.376 INFO:tasks.ceph:role: osd.4 2020-11-16T22:13:29.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:13:29.376 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T22:13:29.881 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:29.881 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:29.881 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:29.882 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:29.882 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:29.882 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:29.882 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:29.883 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-11-16T22:13:29.883 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-16T22:13:29.963 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-16T22:13:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:29.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:29.976 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-16T22:13:30.032 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:30.032 INFO:tasks.ceph:role: osd.5 2020-11-16T22:13:30.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:13:30.033 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T22:13:30.540 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:30.540 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:30.540 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:30.540 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:30.541 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:30.541 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:30.541 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:30.541 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:30.541 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:30.542 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-11-16T22:13:30.543 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-16T22:13:30.612 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-16T22:13:30.623 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:30.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:30.625 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-16T22:13:30.678 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:30.678 INFO:tasks.ceph:role: osd.6 2020-11-16T22:13:30.678 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:13:30.678 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T22:13:31.162 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:31.163 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:31.164 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-11-16T22:13:31.164 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-16T22:13:31.216 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-16T22:13:31.227 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:31.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:31.227 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-16T22:13:31.287 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T22:13:31.287 INFO:tasks.ceph:role: osd.7 2020-11-16T22:13:31.287 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com 2020-11-16T22:13:31.288 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T22:13:31.820 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T22:13:31.821 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T22:13:31.821 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T22:13:31.821 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T22:13:31.821 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-11-16T22:13:31.821 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T22:13:31.822 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T22:13:31.822 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T22:13:31.822 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T22:13:31.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-11-16T22:13:31.823 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-16T22:13:31.875 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-16T22:13:31.887 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T22:13:31.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:13:31.888 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:31.993 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.006+0000 7f44a3068d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-16T22:13:31.995 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.010+0000 7f44a3068d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-16T22:13:31.995 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.010+0000 7f44a3068d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:13:31.996 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.010+0000 7f44a3068d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:13:31.996 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.010+0000 7f44a3068d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T22:13:32.004 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:32.018+0000 7f44a3068d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-16T22:13:34.974 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-16T22:13:34.990 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-16T22:13:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-16T22:13:35.071 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:13:35.072 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:13:35.072 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T22:13:35.072 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:35.086+0000 7f92b4c19d80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-16T22:13:38.045 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-16T22:13:38.101 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-16T22:13:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-16T22:13:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:13:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:13:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T22:13:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:38.194+0000 7f1a6cd07d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-16T22:13:41.112 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-16T22:13:41.168 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:41.250 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.262+0000 7f980376dd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-16T22:13:41.250 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.266+0000 7f980376dd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-16T22:13:41.250 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.266+0000 7f980376dd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:13:41.250 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.266+0000 7f980376dd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:13:41.251 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.266+0000 7f980376dd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T22:13:41.251 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:41.266+0000 7f980376dd80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-16T22:13:42.331 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:42.338 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:42.340 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:13:44.201 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-16T22:13:44.217 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-16T22:13:44.217 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.217 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:13:44.222 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T22:13:44.230 INFO:teuthology.orchestra.run.smithi185.stdout:[mgr.y] 2020-11-16T22:13:44.231 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A== 2020-11-16T22:13:44.231 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.231 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.231 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.000157103 s, 356 kB/s 2020-11-16T22:13:44.232 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.232 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:13:44.277 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-16T22:13:44.286 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.a] 2020-11-16T22:13:44.286 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA== 2020-11-16T22:13:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.00016977 s, 330 kB/s 2020-11-16T22:13:44.287 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.288 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:13:44.333 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T22:13:44.341 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.0] 2020-11-16T22:13:44.341 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw== 2020-11-16T22:13:44.341 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.341 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.341 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.000157314 s, 356 kB/s 2020-11-16T22:13:44.342 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.342 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:13:44.389 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T22:13:44.396 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.1] 2020-11-16T22:13:44.397 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA== 2020-11-16T22:13:44.397 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.397 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.397 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.000158226 s, 354 kB/s 2020-11-16T22:13:44.398 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.398 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:13:44.445 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T22:13:44.454 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.2] 2020-11-16T22:13:44.454 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww== 2020-11-16T22:13:44.454 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.454 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.454 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.00016233 s, 345 kB/s 2020-11-16T22:13:44.455 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:44.455 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T22:13:44.501 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T22:13:44.510 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.3] 2020-11-16T22:13:44.510 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ== 2020-11-16T22:13:44.510 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:44.510 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:44.510 INFO:teuthology.orchestra.run.smithi185.stderr:56 bytes copied, 0.000160683 s, 349 kB/s 2020-11-16T22:13:44.511 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.511 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:13:44.516 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T22:13:44.525 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2020-11-16T22:13:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug== 2020-11-16T22:13:44.525 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.525 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.525 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000174573 s, 321 kB/s 2020-11-16T22:13:44.526 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.526 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-16T22:13:44.573 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-16T22:13:44.582 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-16T22:13:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA== 2020-11-16T22:13:44.582 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.583 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.583 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000179591 s, 312 kB/s 2020-11-16T22:13:44.584 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.584 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-16T22:13:44.629 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-16T22:13:44.638 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-16T22:13:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew== 2020-11-16T22:13:44.638 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.638 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.638 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000178426 s, 314 kB/s 2020-11-16T22:13:44.639 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.640 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T22:13:44.685 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T22:13:44.692 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2020-11-16T22:13:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ== 2020-11-16T22:13:44.693 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.693 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.693 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000196683 s, 285 kB/s 2020-11-16T22:13:44.694 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.694 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T22:13:44.741 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T22:13:44.749 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2020-11-16T22:13:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag== 2020-11-16T22:13:44.749 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.750 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.750 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 7.4461e-05 s, 752 kB/s 2020-11-16T22:13:44.751 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.751 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T22:13:44.796 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T22:13:44.800 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2020-11-16T22:13:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g== 2020-11-16T22:13:44.801 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.801 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.801 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000156673 s, 357 kB/s 2020-11-16T22:13:44.802 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:44.802 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T22:13:44.847 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T22:13:44.852 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2020-11-16T22:13:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw== 2020-11-16T22:13:44.852 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:44.852 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:44.853 INFO:teuthology.orchestra.run.smithi096.stderr:56 bytes copied, 0.000160654 s, 349 kB/s 2020-11-16T22:13:44.853 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:44.854 INFO:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:13:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T22:13:44.870 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2020-11-16T22:13:44.870 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg== 2020-11-16T22:13:44.871 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:44.871 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:44.871 INFO:teuthology.orchestra.run.smithi062.stderr:59 bytes copied, 0.000188644 s, 313 kB/s 2020-11-16T22:13:44.871 INFO:teuthology.orchestra.run.smithi062:> set -ex 2020-11-16T22:13:44.872 INFO:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T22:13:44.927 INFO:teuthology.orchestra.run.smithi062.stderr:+ dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T22:13:44.930 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2020-11-16T22:13:44.930 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q== 2020-11-16T22:13:44.931 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2020-11-16T22:13:44.931 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2020-11-16T22:13:44.931 INFO:teuthology.orchestra.run.smithi062.stderr:59 bytes copied, 0.000438794 s, 134 kB/s 2020-11-16T22:13:44.931 INFO:tasks.ceph:Adding keys to all mons... 2020-11-16T22:13:44.932 INFO:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T22:13:44.934 INFO:teuthology.orchestra.run.smithi185:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T22:13:44.944 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.y] 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A== 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA== 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw== 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2020-11-16T22:13:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA== 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww== 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ== 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2020-11-16T22:13:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug== 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA== 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c] 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew== 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ== 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2020-11-16T22:13:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag== 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g== 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw== 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg== 2020-11-16T22:13:44.948 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2020-11-16T22:13:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q== 2020-11-16T22:13:44.949 INFO:teuthology.orchestra.run.smithi185.stdout:[mgr.y] 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A== 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.a] 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA== 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.0] 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw== 2020-11-16T22:13:44.950 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.1] 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA== 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.2] 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww== 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.3] 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ== 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout:[mgr.x] 2020-11-16T22:13:44.951 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug== 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.b] 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA== 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout:[mds.c] 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew== 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.4] 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ== 2020-11-16T22:13:44.952 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.5] 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag== 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.6] 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g== 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout:[osd.7] 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw== 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout:[client.0] 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg== 2020-11-16T22:13:44.953 INFO:teuthology.orchestra.run.smithi185.stdout:[client.1] 2020-11-16T22:13:44.954 INFO:teuthology.orchestra.run.smithi185.stdout: key = AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q== 2020-11-16T22:13:44.954 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:13:44.995 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:13:45.038 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.074 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.126 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.162 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.211 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.246 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.294 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.334 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.381 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.418 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.462 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:13:45.502 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T22:13:45.546 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.586 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.631 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.670 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-16T22:13:45.716 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.759 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.799 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.842 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.885 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.926 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:45.970 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:46.010 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T22:13:46.055 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:13:46.094 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:13:46.142 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:13:46.143 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T22:13:46.187 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-16T22:13:46.187 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-16T22:13:46.242 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:13:46.356 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-16T22:13:46.371 INFO:teuthology.orchestra.run.smithi185:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-16T22:13:46.426 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:13:46.546 INFO:teuthology.orchestra.run.smithi185:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-16T22:13:46.561 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-16T22:13:46.576 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T22:13:46.689 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-16T22:13:46.705 INFO:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:46.746 INFO:teuthology.orchestra.run.smithi185:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T22:13:46.752 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-16T22:13:46.752 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-16T22:13:46.753 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-16T22:13:46.794 INFO:tasks.ceph.mon.a:Started 2020-11-16T22:13:46.795 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-16T22:13:46.795 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-16T22:13:46.797 INFO:tasks.ceph.mon.c:Started 2020-11-16T22:13:46.797 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-16T22:13:46.797 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-16T22:13:46.799 INFO:tasks.ceph.mon.b:Started 2020-11-16T22:13:46.800 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-16T22:13:46.800 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-16T22:13:46.800 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-16T22:13:46.802 INFO:tasks.ceph.mgr.y:Started 2020-11-16T22:13:46.803 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-16T22:13:46.803 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-16T22:13:46.805 INFO:tasks.ceph.mgr.x:Started 2020-11-16T22:13:46.805 INFO:tasks.ceph:Setting crush tunables to default 2020-11-16T22:13:46.805 INFO:teuthology.orchestra.run.smithi185:> sudo ceph --cluster ceph osd crush tunables default 2020-11-16T22:13:46.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 Processor -- start 2020-11-16T22:13:46.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 -- start start 2020-11-16T22:13:47.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 0x7f5848144c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480b02e0 con 0x7f58480afc40 2020-11-16T22:13:47.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480b0580 con 0x7f58480ade00 2020-11-16T22:13:47.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f5850679700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5848143fa0 con 0x7f58480b74b0 2020-11-16T22:13:47.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584ec16700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f58480ade00 0x7f5848142ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:48742/0 (socket says 172.21.15.185:48742) 2020-11-16T22:13:47.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584ec16700 1 -- 172.21.15.185:0/3419320812 learned_addr learned my addr 172.21.15.185:0/3419320812 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:13:47.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584e415700 1 --2- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 0x7f5848144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584e415700 1 -- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 msgr2=0x7f5848144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-16T22:13:47.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584e415700 1 -- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 msgr2=0x7f5848144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-16T22:13:47.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584e415700 1 --2- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 0x7f5848144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-16T22:13:47.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.011+0000 7f584e415700 1 --2- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 0x7f5848144c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-16T22:13:47.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.159+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493852168 0 0) 0x7f58480b02e0 con 0x7f58480afc40 2020-11-16T22:13:47.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.159+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5830003240 con 0x7f58480afc40 2020-11-16T22:13:47.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.159+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025987555 0 0) 0x7f58480b0580 con 0x7f58480ade00 2020-11-16T22:13:47.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.159+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58480b02e0 con 0x7f58480ade00 2020-11-16T22:13:47.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2169374200 0 0) 0x7f5830003240 con 0x7f58480afc40 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 msgr2=0x7f5848144c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 --2- 172.21.15.185:0/3419320812 >> v2:172.21.15.96:3300/0 conn(0x7f58480b74b0 0x7f5848144c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 >> v1:172.21.15.185:6789/0 conn(0x7f58480ade00 legacy=0x7f5848142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> v1:172.21.15.96:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480ad250 con 0x7f58480afc40 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v1:172.21.15.96:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f5838001530 con 0x7f58480afc40 2020-11-16T22:13:47.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 >> v1:172.21.15.96:6789/0 conn(0x7f58480afc40 legacy=0x7f58480ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5830004920 0x7f5830004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5830003480 0x7f5830003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f583000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848143fa0 con 0x7f5830003df0 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583000bc20 con 0x7f5830004920 2020-11-16T22:13:47.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583000bd60 con 0x7f5830003480 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584dc14700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5830003480 0x7f5830003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f583000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584e415700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5830004920 0x7f5830004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 -- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5830003480 msgr2=0x7f5830003900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5830003480 0x7f5830003900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 -- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5830004920 msgr2=0x7f5830004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5830004920 0x7f5830004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 -- 172.21.15.185:0/3419320812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830003240 con 0x7f5830003df0 2020-11-16T22:13:47.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 -- 172.21.15.185:0/3419320812 reap_dead start 2020-11-16T22:13:47.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.163+0000 7f584ec16700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f583000b730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5840002080 tx=0x7f5840007f70).ready entity=mon.0 client_cookie=703f3ff0d004d8f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:47.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000cbd0 con 0x7f5830003df0 2020-11-16T22:13:47.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000cd30 con 0x7f5830003df0 2020-11-16T22:13:47.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f584d413700 1 -- 172.21.15.185:0/3419320812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000cea0 con 0x7f5830003df0 2020-11-16T22:13:47.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 msgr2=0x7f583000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f583000b730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5840002080 tx=0x7f5840007f70).stop 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- 172.21.15.185:0/3419320812 shutdown_connections 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 --2- 172.21.15.185:0/3419320812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f583000b730 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- 172.21.15.185:0/3419320812 >> 172.21.15.185:0/3419320812 conn(0x7f5848067860 msgr2=0x7f5848067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- 172.21.15.185:0/3419320812 shutdown_connections 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- 172.21.15.185:0/3419320812 wait complete. 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 Processor -- start 2020-11-16T22:13:47.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.167+0000 7f5850679700 1 -- start start 2020-11-16T22:13:47.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f58481466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f58480b0fc0 0x7f5848149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481487b0 con 0x7f5830003df0 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848153390 con 0x7f5848146bd0 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5848153630 con 0x7f58480b0fc0 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55326/0 (socket says 172.21.15.185:55326) 2020-11-16T22:13:47.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 -- 172.21.15.185:0/560424788 learned_addr learned my addr 172.21.15.185:0/560424788 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:47.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584e415700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f58481466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584ec16700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f58480b0fc0 0x7f5848149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 -- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f58480b0fc0 msgr2=0x7f5848149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f58480b0fc0 0x7f5848149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 -- 172.21.15.185:0/560424788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 msgr2=0x7f58481466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f58481466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 -- 172.21.15.185:0/560424788 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848148a10 con 0x7f5848146bd0 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584e415700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f58481466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:47.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f584dc14700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f583800fb30 tx=0x7f58480b0710).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583800d8e0 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583800da40 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583800de00 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848148cb0 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480b1440 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f583801d6c0 con 0x7f5848146bd0 2020-11-16T22:13:47.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f5838013760 con 0x7f5848146bd0 2020-11-16T22:13:47.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.171+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584805d4e0 con 0x7f5848146bd0 2020-11-16T22:13:47.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.175+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f583801dae0 con 0x7f5848146bd0 2020-11-16T22:13:47.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.251+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f584814b5a0 con 0x7f5848146bd0 2020-11-16T22:13:47.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f58367fc700 1 -- 172.21.15.185:0/560424788 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f58380139a0 con 0x7f5848146bd0 2020-11-16T22:13:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 msgr2=0x7f58480b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f583800fb30 tx=0x7f58480b0710).stop 2020-11-16T22:13:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 shutdown_connections 2020-11-16T22:13:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f58480b0fc0 0x7f5848149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5830003df0 0x7f58481466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 --2- 172.21.15.185:0/560424788 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5848146bd0 0x7f58480b0ad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 >> 172.21.15.185:0/560424788 conn(0x7f5848067860 msgr2=0x7f5848142840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 shutdown_connections 2020-11-16T22:13:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:47.259+0000 7f5850679700 1 -- 172.21.15.185:0/560424788 wait complete. 2020-11-16T22:13:47.249 INFO:teuthology.orchestra.run.smithi185.stderr:adjusted tunables profile to default 2020-11-16T22:13:47.262 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-16T22:13:47.263 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:47.263 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:13:47.266 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T22:13:47.271 INFO:teuthology.orchestra.run.smithi185.stdout:bc3b03d0-fcaf-4134-b2f3-f83255339276 2020-11-16T22:13:47.272 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:47.272 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:47.272 INFO:teuthology.orchestra.run.smithi185.stderr:37 bytes copied, 8.5405e-05 s, 433 kB/s 2020-11-16T22:13:47.273 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:47.273 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:13:47.316 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T22:13:47.322 INFO:teuthology.orchestra.run.smithi185.stdout:47286fad-a235-46ea-92f4-3e0b3dff9c32 2020-11-16T22:13:47.323 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:47.323 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:47.323 INFO:teuthology.orchestra.run.smithi185.stderr:37 bytes copied, 0.000166513 s, 222 kB/s 2020-11-16T22:13:47.324 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:47.324 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:13:47.369 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T22:13:47.377 INFO:teuthology.orchestra.run.smithi185.stdout:027cf982-cd16-41e5-93f0-339e136d5781 2020-11-16T22:13:47.377 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:47.377 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:47.377 INFO:teuthology.orchestra.run.smithi185.stderr:37 bytes copied, 0.000168383 s, 220 kB/s 2020-11-16T22:13:47.378 INFO:teuthology.orchestra.run.smithi185:> set -ex 2020-11-16T22:13:47.378 INFO:teuthology.orchestra.run.smithi185:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T22:13:47.425 INFO:teuthology.orchestra.run.smithi185.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T22:13:47.432 INFO:teuthology.orchestra.run.smithi185.stdout:d27e1166-11a5-401b-b82d-8ed20760bd20 2020-11-16T22:13:47.432 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records in 2020-11-16T22:13:47.432 INFO:teuthology.orchestra.run.smithi185.stderr:0+1 records out 2020-11-16T22:13:47.432 INFO:teuthology.orchestra.run.smithi185.stderr:37 bytes copied, 0.000160412 s, 231 kB/s 2020-11-16T22:13:47.433 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:47.434 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T22:13:47.438 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T22:13:47.446 INFO:teuthology.orchestra.run.smithi096.stdout:91b77cc0-739c-4dcc-ab56-82e9b9de75fb 2020-11-16T22:13:47.447 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:47.447 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:47.447 INFO:teuthology.orchestra.run.smithi096.stderr:37 bytes copied, 0.000169411 s, 218 kB/s 2020-11-16T22:13:47.447 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:47.448 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T22:13:47.493 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T22:13:47.502 INFO:teuthology.orchestra.run.smithi096.stdout:137c0d5c-501f-4955-a1c0-cd3daf2a78a1 2020-11-16T22:13:47.502 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:47.502 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:47.502 INFO:teuthology.orchestra.run.smithi096.stderr:37 bytes copied, 0.000171997 s, 215 kB/s 2020-11-16T22:13:47.503 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:47.504 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T22:13:47.549 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T22:13:47.557 INFO:teuthology.orchestra.run.smithi096.stdout:9c13a225-67c7-4d0e-8706-b3264ac0367d 2020-11-16T22:13:47.558 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:47.558 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:47.558 INFO:teuthology.orchestra.run.smithi096.stderr:37 bytes copied, 0.000209884 s, 176 kB/s 2020-11-16T22:13:47.559 INFO:teuthology.orchestra.run.smithi096:> set -ex 2020-11-16T22:13:47.559 INFO:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T22:13:47.605 INFO:teuthology.orchestra.run.smithi096.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T22:13:47.614 INFO:teuthology.orchestra.run.smithi096.stdout:c789075c-0198-4420-9039-6f6ae5d5f906 2020-11-16T22:13:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2020-11-16T22:13:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2020-11-16T22:13:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:37 bytes copied, 0.000175814 s, 210 kB/s 2020-11-16T22:13:47.615 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new bc3b03d0-fcaf-4134-b2f3-f83255339276 0 2020-11-16T22:13:47.885 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.898+0000 7ff2d307a700 1 Processor -- start 2020-11-16T22:13:47.885 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.898+0000 7ff2d307a700 1 -- start start 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff2cc147fb0 0x7ff2cc1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 0x7ff2cc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2cc0b2f30 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2cc060fe0 con 0x7ff2cc147fb0 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2cc061280 con 0x7ff2cc05ba80 2020-11-16T22:13:47.887 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d2078700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff2cc0b1a20 0x7ff2cc1411b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52902/0 (socket says 172.21.15.96:52902) 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d2078700 1 -- 172.21.15.96:0/2960943520 learned_addr learned my addr 172.21.15.96:0/2960943520 (peer_addr_for_me v1:172.21.15.96:0/0) 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 --2- 172.21.15.96:0/2960943520 >> v2:172.21.15.96:3300/0 conn(0x7ff2cc147fb0 0x7ff2cc1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d2879700 1 --2- 172.21.15.96:0/2960943520 >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 0x7ff2cc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 -- 172.21.15.96:0/2960943520 >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 msgr2=0x7ff2cc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 --2- 172.21.15.96:0/2960943520 >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 0x7ff2cc1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 -- 172.21.15.96:0/2960943520 >> v1:172.21.15.96:6789/0 conn(0x7ff2cc0b1a20 legacy=0x7ff2cc1411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 -- 172.21.15.96:0/2960943520 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc143f90 con 0x7ff2cc147fb0 2020-11-16T22:13:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d2879700 1 --2- 172.21.15.96:0/2960943520 >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 0x7ff2cc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1877700 1 --2- 172.21.15.96:0/2960943520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc1443c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff2cc05c060 tx=0x7ff2c0006df0).ready entity=mon.0 client_cookie=8efb77eb03104923 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1076700 1 -- 172.21.15.96:0/2960943520 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00097b0 con 0x7ff2cc147fb0 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1076700 1 -- 172.21.15.96:0/2960943520 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0009910 con 0x7ff2cc147fb0 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d1076700 1 -- 172.21.15.96:0/2960943520 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0009c90 con 0x7ff2cc147fb0 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- 172.21.15.96:0/2960943520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc147fb0 msgr2=0x7ff2cc1443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 --2- 172.21.15.96:0/2960943520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc1443c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff2cc05c060 tx=0x7ff2c0006df0).stop 2020-11-16T22:13:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- 172.21.15.96:0/2960943520 shutdown_connections 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 --2- 172.21.15.96:0/2960943520 >> v2:172.21.15.185:3300/0 conn(0x7ff2cc05ba80 0x7ff2cc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 --2- 172.21.15.96:0/2960943520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc1443c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- 172.21.15.96:0/2960943520 >> 172.21.15.96:0/2960943520 conn(0x7ff2cc063df0 msgr2=0x7ff2cc0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- 172.21.15.96:0/2960943520 shutdown_connections 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.902+0000 7ff2d307a700 1 -- 172.21.15.96:0/2960943520 wait complete. 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 Processor -- start 2020-11-16T22:13:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- start start 2020-11-16T22:13:47.893 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 0x7ff2cc146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.893 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1404d0 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc140770 con 0x7ff2cc147fb0 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc13f500 con 0x7ff2cc05ba80 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.894 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37642/0 (socket says 172.21.15.96:37642) 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 -- 172.21.15.96:0/877192356 learned_addr learned my addr 172.21.15.96:0/877192356 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d2078700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 0x7ff2cc146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d2879700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 -- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 msgr2=0x7ff2cc146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 0x7ff2cc146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 -- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 msgr2=0x7ff2cc154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.895 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 -- 172.21.15.96:0/877192356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc147480 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d2078700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 0x7ff2cc146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d2879700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d1877700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ff2c000c010 tx=0x7ff2c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011dd0 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c001e460 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.896 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc146f30 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c001e820 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc13efd0 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7ff2c0025030 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff2c0028a50 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.906+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc153fb0 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.898 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.910+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7ff2c0023070 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.975 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.990+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bc3b03d0-fcaf-4134-b2f3-f83255339276", "id": 0} v 0) v1 -- 0x7ff2cc05d200 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.979 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2ca7fc700 1 -- 172.21.15.96:0/877192356 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc3b03d0-fcaf-4134-b2f3-f83255339276", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff2c0031450 con 0x7ff2cc0b1a20 2020-11-16T22:13:47.980 INFO:teuthology.orchestra.run.smithi096.stdout:0 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 msgr2=0x7ff2cc13e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ff2c000c010 tx=0x7ff2c0015000).stop 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 shutdown_connections 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2cc147fb0 0x7ff2cc154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2cc05ba80 0x7ff2cc146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.981 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 --2- 172.21.15.96:0/877192356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2cc0b1a20 0x7ff2cc13e940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:47.982 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 >> 172.21.15.96:0/877192356 conn(0x7ff2cc063df0 msgr2=0x7ff2cc066c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:47.982 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 shutdown_connections 2020-11-16T22:13:47.982 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:47.994+0000 7ff2d307a700 1 -- 172.21.15.96:0/877192356 wait complete. 2020-11-16T22:13:47.992 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 47286fad-a235-46ea-92f4-3e0b3dff9c32 1 2020-11-16T22:13:48.106 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 Processor -- start 2020-11-16T22:13:48.107 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- start start 2020-11-16T22:13:48.108 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa3d00b7690 0x7fa3d0067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.108 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa3d00b1f20 0x7fa3d00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.108 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d00b3f50 con 0x7fa3d0141690 2020-11-16T22:13:48.108 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00b41f0 con 0x7fa3d00b7690 2020-11-16T22:13:48.108 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00b4490 con 0x7fa3d00b1f20 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa3d00b7690 0x7fa3d0067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa3d00b7690 0x7fa3d0067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37646/0 (socket says 172.21.15.96:37646) 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 -- 172.21.15.96:0/420593042 learned_addr learned my addr 172.21.15.96:0/420593042 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d4cdb700 1 --2- 172.21.15.96:0/420593042 >> v2:172.21.15.185:3300/0 conn(0x7fa3d00b1f20 0x7fa3d00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 -- 172.21.15.96:0/420593042 >> v2:172.21.15.185:3300/0 conn(0x7fa3d00b1f20 msgr2=0x7fa3d00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 --2- 172.21.15.96:0/420593042 >> v2:172.21.15.185:3300/0 conn(0x7fa3d00b1f20 0x7fa3d00b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.109 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 -- 172.21.15.96:0/420593042 >> v1:172.21.15.185:6789/0 conn(0x7fa3d0141690 legacy=0x7fa3d0144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.110 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 -- 172.21.15.96:0/420593042 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00b4e20 con 0x7fa3d00b7690 2020-11-16T22:13:48.110 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3cffff700 1 --2- 172.21.15.96:0/420593042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0067420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fa3d0064720 tx=0x7fa3c8009290).ready entity=mon.0 client_cookie=ccba1f64ebe124af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.111 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3ceffd700 1 -- 172.21.15.96:0/420593042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800bbc0 con 0x7fa3d00b7690 2020-11-16T22:13:48.111 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3ceffd700 1 -- 172.21.15.96:0/420593042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800bd20 con 0x7fa3d00b7690 2020-11-16T22:13:48.111 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3ceffd700 1 -- 172.21.15.96:0/420593042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011610 con 0x7fa3d00b7690 2020-11-16T22:13:48.111 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- 172.21.15.96:0/420593042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 msgr2=0x7fa3d0067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.111 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/420593042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0067420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fa3d0064720 tx=0x7fa3c8009290).stop 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- 172.21.15.96:0/420593042 shutdown_connections 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/420593042 >> v2:172.21.15.185:3300/0 conn(0x7fa3d00b1f20 0x7fa3d00b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/420593042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0067420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- 172.21.15.96:0/420593042 >> 172.21.15.96:0/420593042 conn(0x7fa3d0064b90 msgr2=0x7fa3d0063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- 172.21.15.96:0/420593042 shutdown_connections 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.122+0000 7fa3d54dc700 1 -- 172.21.15.96:0/420593042 wait complete. 2020-11-16T22:13:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 Processor -- start 2020-11-16T22:13:48.113 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- start start 2020-11-16T22:13:48.114 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa3d00b1f20 0x7fa3d0155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.114 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.114 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa3d0141690 0x7fa3d0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0155570 con 0x7fa3d00b7690 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0111b60 con 0x7fa3d00b1f20 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0142870 con 0x7fa3d0141690 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37652/0 (socket says 172.21.15.96:37652) 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 -- 172.21.15.96:0/3081477192 learned_addr learned my addr 172.21.15.96:0/3081477192 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.115 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cffff700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa3d00b1f20 0x7fa3d0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d4cdb700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa3d0141690 0x7fa3d0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 -- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa3d0141690 msgr2=0x7fa3d0151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa3d0141690 0x7fa3d0151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 -- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa3d00b1f20 msgr2=0x7fa3d0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa3d00b1f20 0x7fa3d0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 -- 172.21.15.96:0/3081477192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d01546a0 con 0x7fa3d00b7690 2020-11-16T22:13:48.116 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3cf7fe700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fa3b800c710 tx=0x7fa3b8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8007020 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b8007180 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d0154940 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80064f0 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d01425b0 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fa3b8010070 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.126+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fa3b800e870 con 0x7fa3d00b7690 2020-11-16T22:13:48.117 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.130+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0154de0 con 0x7fa3d00b7690 2020-11-16T22:13:48.118 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.134+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fa3d0154de0 con 0x7fa3d00b7690 2020-11-16T22:13:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.218+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "47286fad-a235-46ea-92f4-3e0b3dff9c32", "id": 1} v 0) v1 -- 0x7fa3d005d200 con 0x7fa3d00b7690 2020-11-16T22:13:48.208 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.222+0000 7fa3c7fff700 1 -- 172.21.15.96:0/3081477192 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "47286fad-a235-46ea-92f4-3e0b3dff9c32", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa3b8020060 con 0x7fa3d00b7690 2020-11-16T22:13:48.209 INFO:teuthology.orchestra.run.smithi096.stdout:1 2020-11-16T22:13:48.209 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.222+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 msgr2=0x7fa3d0142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.222+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fa3b800c710 tx=0x7fa3b8007c20).stop 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 shutdown_connections 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa3d00b1f20 0x7fa3d0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa3d0141690 0x7fa3d0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 --2- 172.21.15.96:0/3081477192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d00b7690 0x7fa3d0142040 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.210 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 >> 172.21.15.96:0/3081477192 conn(0x7fa3d0064b90 msgr2=0x7fa3d0066fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.211 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 shutdown_connections 2020-11-16T22:13:48.211 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.226+0000 7fa3d54dc700 1 -- 172.21.15.96:0/3081477192 wait complete. 2020-11-16T22:13:48.221 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 027cf982-cd16-41e5-93f0-339e136d5781 2 2020-11-16T22:13:48.342 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 Processor -- start 2020-11-16T22:13:48.342 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- start start 2020-11-16T22:13:48.344 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff16c1483b0 0x7ff16c067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff16c068220 0x7ff16c13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c1477a0 con 0x7ff16c067700 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c064310 con 0x7ff16c1483b0 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c1479d0 con 0x7ff16c068220 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16affd700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff16c067700 0x7ff16c063c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52926/0 (socket says 172.21.15.96:52926) 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16affd700 1 -- 172.21.15.96:0/4168027154 learned_addr learned my addr 172.21.15.96:0/4168027154 (peer_addr_for_me v1:172.21.15.96:0/0) 2020-11-16T22:13:48.345 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 --2- 172.21.15.96:0/4168027154 >> v2:172.21.15.96:3300/0 conn(0x7ff16c1483b0 0x7ff16c067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16b7fe700 1 --2- 172.21.15.96:0/4168027154 >> v2:172.21.15.185:3300/0 conn(0x7ff16c068220 0x7ff16c13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 -- 172.21.15.96:0/4168027154 >> v2:172.21.15.185:3300/0 conn(0x7ff16c068220 msgr2=0x7ff16c13dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 --2- 172.21.15.96:0/4168027154 >> v2:172.21.15.185:3300/0 conn(0x7ff16c068220 0x7ff16c13dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 -- 172.21.15.96:0/4168027154 >> v1:172.21.15.96:6789/0 conn(0x7ff16c067700 legacy=0x7ff16c063c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff169ffb700 1 -- 172.21.15.96:0/4168027154 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324416925 0 0) 0x7ff16c1477a0 con 0x7ff16c067700 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 -- 172.21.15.96:0/4168027154 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c147c70 con 0x7ff16c1483b0 2020-11-16T22:13:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff16a7fc700 1 --2- 172.21.15.96:0/4168027154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c067ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff16c064a70 tx=0x7ff154006df0).ready entity=mon.0 client_cookie=9f9723c7f9021f0e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.347 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff169ffb700 1 -- 172.21.15.96:0/4168027154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540097b0 con 0x7ff16c1483b0 2020-11-16T22:13:48.347 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff169ffb700 1 -- 172.21.15.96:0/4168027154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154009910 con 0x7ff16c1483b0 2020-11-16T22:13:48.347 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff169ffb700 1 -- 172.21.15.96:0/4168027154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154009c90 con 0x7ff16c1483b0 2020-11-16T22:13:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- 172.21.15.96:0/4168027154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 msgr2=0x7ff16c067ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 --2- 172.21.15.96:0/4168027154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c067ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff16c064a70 tx=0x7ff154006df0).stop 2020-11-16T22:13:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- 172.21.15.96:0/4168027154 shutdown_connections 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 --2- 172.21.15.96:0/4168027154 >> v2:172.21.15.185:3300/0 conn(0x7ff16c068220 0x7ff16c13dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 --2- 172.21.15.96:0/4168027154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c067ae0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.358+0000 7ff171a1b700 1 -- 172.21.15.96:0/4168027154 >> 172.21.15.96:0/4168027154 conn(0x7ff16c061240 msgr2=0x7ff16c061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- 172.21.15.96:0/4168027154 shutdown_connections 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- 172.21.15.96:0/4168027154 wait complete. 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 Processor -- start 2020-11-16T22:13:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- start start 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff16c067700 0x7ff16c0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff16c068220 0x7ff16c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b5c00 con 0x7ff16c1483b0 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b56c0 con 0x7ff16c067700 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b5960 con 0x7ff16c068220 2020-11-16T22:13:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37666/0 (socket says 172.21.15.96:37666) 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 -- 172.21.15.96:0/2752510423 learned_addr learned my addr 172.21.15.96:0/2752510423 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16a7fc700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff16c068220 0x7ff16c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16affd700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff16c067700 0x7ff16c0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 -- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff16c068220 msgr2=0x7ff16c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff16c068220 0x7ff16c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 -- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff16c067700 msgr2=0x7ff16c0b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.352 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff16c067700 0x7ff16c0b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 -- 172.21.15.96:0/2752510423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c14b2b0 con 0x7ff16c1483b0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff16b7fe700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff1640067d0 tx=0x7ff164008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164009620 con 0x7ff16c1483b0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164009780 con 0x7ff16c1483b0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164009b20 con 0x7ff16c1483b0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c14b750 con 0x7ff16c1483b0 2020-11-16T22:13:48.353 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1463b0 con 0x7ff16c1483b0 2020-11-16T22:13:48.354 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7ff164006280 con 0x7ff16c1483b0 2020-11-16T22:13:48.354 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.362+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7ff164005da0 con 0x7ff16c1483b0 2020-11-16T22:13:48.354 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.366+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c05d4e0 con 0x7ff16c1483b0 2020-11-16T22:13:48.354 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.366+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7ff164005950 con 0x7ff16c1483b0 2020-11-16T22:13:48.436 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.450+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "027cf982-cd16-41e5-93f0-339e136d5781", "id": 2} v 0) v1 -- 0x7ff16c063060 con 0x7ff16c1483b0 2020-11-16T22:13:48.440 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.454+0000 7ff162ffd700 1 -- 172.21.15.96:0/2752510423 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "027cf982-cd16-41e5-93f0-339e136d5781", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff16402b0b0 con 0x7ff16c1483b0 2020-11-16T22:13:48.441 INFO:teuthology.orchestra.run.smithi096.stdout:2 2020-11-16T22:13:48.442 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 msgr2=0x7ff16c1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.442 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7ff1640067d0 tx=0x7ff164008040).stop 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 shutdown_connections 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff16c067700 0x7ff16c0b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff16c1483b0 0x7ff16c1570d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 --2- 172.21.15.96:0/2752510423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff16c068220 0x7ff16c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 >> 172.21.15.96:0/2752510423 conn(0x7ff16c061240 msgr2=0x7ff16c1453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 shutdown_connections 2020-11-16T22:13:48.444 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.458+0000 7ff171a1b700 1 -- 172.21.15.96:0/2752510423 wait complete. 2020-11-16T22:13:48.459 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new d27e1166-11a5-401b-b82d-8ed20760bd20 3 2020-11-16T22:13:48.572 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.586+0000 7fa5b34fe700 1 Processor -- start 2020-11-16T22:13:48.573 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.586+0000 7fa5b34fe700 1 -- start start 2020-11-16T22:13:48.574 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac0b38a0 0x7fa5ac0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa5ac05bc50 0x7fa5ac144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 0x7fa5ac13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0ad350 con 0x7fa5ac05bc50 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0b2690 con 0x7fa5ac144d60 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0b2930 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.575 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa5ac05bc50 0x7fa5ac144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa5ac05bc50 0x7fa5ac144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37672/0 (socket says 172.21.15.96:37672) 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/760262999 learned_addr learned my addr 172.21.15.96:0/760262999 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b2cfd700 1 --2- 172.21.15.96:0/760262999 >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 0x7fa5ac13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b24fc700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac0b38a0 0x7fa5ac0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/760262999 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac0b38a0 msgr2=0x7fa5ac0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac0b38a0 0x7fa5ac0b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.576 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/760262999 >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 msgr2=0x7fa5ac13fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/760262999 >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 0x7fa5ac13fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/760262999 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac1443d0 con 0x7fa5ac05bc50 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b2cfd700 1 --2- 172.21.15.96:0/760262999 >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 0x7fa5ac13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac05bc50 0x7fa5ac144870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa5ac0ac710 tx=0x7fa5a8006eb0).ready entity=mon.0 client_cookie=b7e99872b28bb415 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b14fa700 1 -- 172.21.15.96:0/760262999 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8009810 con 0x7fa5ac05bc50 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b14fa700 1 -- 172.21.15.96:0/760262999 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a8009970 con 0x7fa5ac05bc50 2020-11-16T22:13:48.577 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b14fa700 1 -- 172.21.15.96:0/760262999 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8009cf0 con 0x7fa5ac05bc50 2020-11-16T22:13:48.578 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- 172.21.15.96:0/760262999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac05bc50 msgr2=0x7fa5ac144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.578 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac05bc50 0x7fa5ac144870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa5ac0ac710 tx=0x7fa5a8006eb0).stop 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- 172.21.15.96:0/760262999 shutdown_connections 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/760262999 >> v2:172.21.15.185:3300/0 conn(0x7fa5ac144d60 0x7fa5ac13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac0b38a0 0x7fa5ac0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/760262999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac05bc50 0x7fa5ac144870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- 172.21.15.96:0/760262999 >> 172.21.15.96:0/760262999 conn(0x7fa5ac063df0 msgr2=0x7fa5ac0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- 172.21.15.96:0/760262999 shutdown_connections 2020-11-16T22:13:48.579 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.590+0000 7fa5b34fe700 1 -- 172.21.15.96:0/760262999 wait complete. 2020-11-16T22:13:48.580 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 Processor -- start 2020-11-16T22:13:48.580 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- start start 2020-11-16T22:13:48.581 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac05bc50 0x7fa5ac1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ac144d60 0x7fa5ac153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac13cc90 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac13cf30 con 0x7fa5ac144d60 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac13d1d0 con 0x7fa5ac05bc50 2020-11-16T22:13:48.582 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37678/0 (socket says 172.21.15.96:37678) 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/43131321 learned_addr learned my addr 172.21.15.96:0/43131321 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b24fc700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac05bc50 0x7fa5ac1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b2cfd700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ac144d60 0x7fa5ac153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac05bc50 msgr2=0x7fa5ac1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac05bc50 0x7fa5ac1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.583 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ac144d60 msgr2=0x7fa5ac153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ac144d60 0x7fa5ac153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 -- 172.21.15.96:0/43131321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac13d470 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b1cfb700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fa5ac05c7a0 tx=0x7fa5a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8011c80 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a8011de0 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801e6b0 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac157ab0 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac13c740 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fa5a8025030 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.594+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fa5a801d880 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.598+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac13d600 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.602+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fa5a8023070 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.669 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.686+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d27e1166-11a5-401b-b82d-8ed20760bd20", "id": 3} v 0) v1 -- 0x7fa5ac05d200 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa59e7fc700 1 -- 172.21.15.96:0/43131321 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d27e1166-11a5-401b-b82d-8ed20760bd20", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5a801edc0 con 0x7fa5ac0b38a0 2020-11-16T22:13:48.674 INFO:teuthology.orchestra.run.smithi096.stdout:3 2020-11-16T22:13:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 msgr2=0x7fa5ac142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fa5ac05c7a0 tx=0x7fa5a8015000).stop 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 shutdown_connections 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ac144d60 0x7fa5ac153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ac05bc50 0x7fa5ac1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 --2- 172.21.15.96:0/43131321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ac0b38a0 0x7fa5ac142c70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 >> 172.21.15.96:0/43131321 conn(0x7fa5ac063df0 msgr2=0x7fa5ac0ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 shutdown_connections 2020-11-16T22:13:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.690+0000 7fa5b34fe700 1 -- 172.21.15.96:0/43131321 wait complete. 2020-11-16T22:13:48.687 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 91b77cc0-739c-4dcc-ab56-82e9b9de75fb 4 2020-11-16T22:13:48.811 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 Processor -- start 2020-11-16T22:13:48.812 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 -- start start 2020-11-16T22:13:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2be4063f90 0x7f2be4145530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2be40af780 0x7f2be414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40af570 con 0x7f2be40b4c30 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40b3d00 con 0x7f2be40af780 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be8e19700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40af2d0 con 0x7f2be4063f90 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2be40af780 0x7f2be414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2be40af780 0x7f2be414b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37686/0 (socket says 172.21.15.96:37686) 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 -- 172.21.15.96:0/1459115425 learned_addr learned my addr 172.21.15.96:0/1459115425 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be37fe700 1 --2- 172.21.15.96:0/1459115425 >> v2:172.21.15.185:3300/0 conn(0x7f2be4063f90 0x7f2be4145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 -- 172.21.15.96:0/1459115425 >> v2:172.21.15.185:3300/0 conn(0x7f2be4063f90 msgr2=0x7f2be4145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 --2- 172.21.15.96:0/1459115425 >> v2:172.21.15.185:3300/0 conn(0x7f2be4063f90 0x7f2be4145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 -- 172.21.15.96:0/1459115425 >> v1:172.21.15.185:6789/0 conn(0x7f2be40b4c30 legacy=0x7f2be4145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 -- 172.21.15.96:0/1459115425 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40b3190 con 0x7f2be40af780 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be3fff700 1 --2- 172.21.15.96:0/1459115425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40af780 0x7f2be414b9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2be4065090 tx=0x7f2bdc009290).ready entity=mon.0 client_cookie=7977dd5b72ee9a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be2ffd700 1 -- 172.21.15.96:0/1459115425 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00bbc0 con 0x7f2be40af780 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be2ffd700 1 -- 172.21.15.96:0/1459115425 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc00bd20 con 0x7f2be40af780 2020-11-16T22:13:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.826+0000 7f2be2ffd700 1 -- 172.21.15.96:0/1459115425 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011610 con 0x7f2be40af780 2020-11-16T22:13:48.817 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- 172.21.15.96:0/1459115425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40af780 msgr2=0x7f2be414b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.817 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- 172.21.15.96:0/1459115425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40af780 0x7f2be414b9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2be4065090 tx=0x7f2bdc009290).stop 2020-11-16T22:13:48.817 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- 172.21.15.96:0/1459115425 shutdown_connections 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- 172.21.15.96:0/1459115425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40af780 0x7f2be414b9d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- 172.21.15.96:0/1459115425 >> v2:172.21.15.185:3300/0 conn(0x7f2be4063f90 0x7f2be4145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- 172.21.15.96:0/1459115425 >> 172.21.15.96:0/1459115425 conn(0x7f2be4067940 msgr2=0x7f2be4067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- 172.21.15.96:0/1459115425 shutdown_connections 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- 172.21.15.96:0/1459115425 wait complete. 2020-11-16T22:13:48.818 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 Processor -- start 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- start start 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2be4063f90 0x7f2be4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2be40af780 0x7f2be413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40ac470 con 0x7f2be40b4c30 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4154650 con 0x7f2be4063f90 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be8e19700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41548f0 con 0x7f2be40af780 2020-11-16T22:13:48.819 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37692/0 (socket says 172.21.15.96:37692) 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be3fff700 1 -- 172.21.15.96:0/2825352324 learned_addr learned my addr 172.21.15.96:0/2825352324 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2be37fe700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2be4063f90 0x7f2be4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.830+0000 7f2bdbfff700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2be40af780 0x7f2be413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 -- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2be40af780 msgr2=0x7f2be413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2be40af780 0x7f2be413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.820 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 -- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2be4063f90 msgr2=0x7f2be4150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.821 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2be4063f90 0x7f2be4150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.821 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 -- 172.21.15.96:0/2825352324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be41513b0 con 0x7f2be40b4c30 2020-11-16T22:13:48.821 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be3fff700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2be4067200 tx=0x7f2bdc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:48.822 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00bbc0 con 0x7f2be40b4c30 2020-11-16T22:13:48.822 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc00be90 con 0x7f2be40b4c30 2020-11-16T22:13:48.822 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be413db20 con 0x7f2be40b4c30 2020-11-16T22:13:48.822 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc010cf0 con 0x7f2be40b4c30 2020-11-16T22:13:48.822 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be4155280 con 0x7f2be40b4c30 2020-11-16T22:13:48.823 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f2bdc023030 con 0x7f2be40b4c30 2020-11-16T22:13:48.823 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f2bdc01c8e0 con 0x7f2be40b4c30 2020-11-16T22:13:48.823 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.834+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be4154ab0 con 0x7f2be40b4c30 2020-11-16T22:13:48.823 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.838+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f2bdc02b060 con 0x7f2be40b4c30 2020-11-16T22:13:48.905 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.918+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "91b77cc0-739c-4dcc-ab56-82e9b9de75fb", "id": 4} v 0) v1 -- 0x7f2be40ac5e0 con 0x7f2be40b4c30 2020-11-16T22:13:48.911 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2bdb7fe700 1 -- 172.21.15.96:0/2825352324 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91b77cc0-739c-4dcc-ab56-82e9b9de75fb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2bdc021070 con 0x7f2be40b4c30 2020-11-16T22:13:48.911 INFO:teuthology.orchestra.run.smithi096.stdout:4 2020-11-16T22:13:48.912 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 msgr2=0x7f2be4154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:48.912 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2be4067200 tx=0x7f2bdc016000).stop 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 shutdown_connections 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2be4063f90 0x7f2be4150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be40b4c30 0x7f2be4154d90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 --2- 172.21.15.96:0/2825352324 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2be40af780 0x7f2be413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 >> 172.21.15.96:0/2825352324 conn(0x7f2be4067940 msgr2=0x7f2be413e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 shutdown_connections 2020-11-16T22:13:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:48.926+0000 7f2be8e19700 1 -- 172.21.15.96:0/2825352324 wait complete. 2020-11-16T22:13:48.928 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 137c0d5c-501f-4955-a1c0-cd3daf2a78a1 5 2020-11-16T22:13:49.043 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 Processor -- start 2020-11-16T22:13:49.043 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 -- start start 2020-11-16T22:13:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 0x7fe2001414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 0x7fe2001419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe20005ba80 0x7fe2001421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000b2f30 con 0x7fe20005ba80 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe200060fe0 con 0x7fe2000b1a20 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe20532e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe200061280 con 0x7fe200145d70 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe20005ba80 0x7fe2001421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe20005ba80 0x7fe2001421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37698/0 (socket says 172.21.15.96:37698) 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 -- 172.21.15.96:0/3510368259 learned_addr learned my addr 172.21.15.96:0/3510368259 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1f7fff700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 0x7fe2001419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.046 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1fffff700 1 --2- 172.21.15.96:0/3510368259 >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 0x7fe2001414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 -- 172.21.15.96:0/3510368259 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 msgr2=0x7fe2001419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 0x7fe2001419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 -- 172.21.15.96:0/3510368259 >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 msgr2=0x7fe2001414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 --2- 172.21.15.96:0/3510368259 >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 0x7fe2001414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 -- 172.21.15.96:0/3510368259 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200060210 con 0x7fe20005ba80 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1f7fff700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 0x7fe2001419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:49.047 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1fffff700 1 --2- 172.21.15.96:0/3510368259 >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 0x7fe2001414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:49.048 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe204b2d700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe2001421c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe2000b7d40 tx=0x7fe1f8009290).ready entity=mon.0 client_cookie=a43c77d36ace51a5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.048 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1ff7fe700 1 -- 172.21.15.96:0/3510368259 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800bbc0 con 0x7fe20005ba80 2020-11-16T22:13:49.048 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1ff7fe700 1 -- 172.21.15.96:0/3510368259 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800bd20 con 0x7fe20005ba80 2020-11-16T22:13:49.048 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.058+0000 7fe1ff7fe700 1 -- 172.21.15.96:0/3510368259 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8011610 con 0x7fe20005ba80 2020-11-16T22:13:49.050 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- 172.21.15.96:0/3510368259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 msgr2=0x7fe2001421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.050 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe2001421c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe2000b7d40 tx=0x7fe1f8009290).stop 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- 172.21.15.96:0/3510368259 shutdown_connections 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe2001421c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- 172.21.15.96:0/3510368259 >> v2:172.21.15.185:3300/0 conn(0x7fe2000b1a20 0x7fe2001414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- 172.21.15.96:0/3510368259 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe200145d70 0x7fe2001419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- 172.21.15.96:0/3510368259 >> 172.21.15.96:0/3510368259 conn(0x7fe200063df0 msgr2=0x7fe2000641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- 172.21.15.96:0/3510368259 shutdown_connections 2020-11-16T22:13:49.051 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- 172.21.15.96:0/3510368259 wait complete. 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 Processor -- start 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- start start 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2000b1a20 0x7fe20013db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe200145d70 0x7fe20013a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200110780 con 0x7fe20005ba80 2020-11-16T22:13:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200110a20 con 0x7fe200145d70 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe20532e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe200110cc0 con 0x7fe2000b1a20 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37700/0 (socket says 172.21.15.96:37700) 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe1fffff700 1 -- 172.21.15.96:0/66213405 learned_addr learned my addr 172.21.15.96:0/66213405 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe1f7fff700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2000b1a20 0x7fe20013db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.062+0000 7fe204b2d700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe200145d70 0x7fe20013a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 -- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2000b1a20 msgr2=0x7fe20013db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2000b1a20 0x7fe20013db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 -- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe200145d70 msgr2=0x7fe20013a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe200145d70 0x7fe20013a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 -- 172.21.15.96:0/66213405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200110f60 con 0x7fe20005ba80 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fffff700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe1e800db60 tx=0x7fe1e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009730 con 0x7fe20005ba80 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e8009890 con 0x7fe20005ba80 2020-11-16T22:13:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2001520e0 con 0x7fe20005ba80 2020-11-16T22:13:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009c30 con 0x7fe20005ba80 2020-11-16T22:13:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000b0d50 con 0x7fe20005ba80 2020-11-16T22:13:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fe1e8006280 con 0x7fe20005ba80 2020-11-16T22:13:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fe1e8006510 con 0x7fe20005ba80 2020-11-16T22:13:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.066+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20005d200 con 0x7fe20005ba80 2020-11-16T22:13:49.056 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.070+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fe1e80058b0 con 0x7fe20005ba80 2020-11-16T22:13:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.150+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "137c0d5c-501f-4955-a1c0-cd3daf2a78a1", "id": 5} v 0) v1 -- 0x7fe2000b40d0 con 0x7fe20005ba80 2020-11-16T22:13:49.140 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.154+0000 7fe1fcff9700 1 -- 172.21.15.96:0/66213405 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "137c0d5c-501f-4955-a1c0-cd3daf2a78a1", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe1e8005a50 con 0x7fe20005ba80 2020-11-16T22:13:49.141 INFO:teuthology.orchestra.run.smithi096.stdout:5 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 msgr2=0x7fe200151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe1e800db60 tx=0x7fe1e8008040).stop 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 shutdown_connections 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe200145d70 0x7fe20013a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe20005ba80 0x7fe200151bf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.142 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 --2- 172.21.15.96:0/66213405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2000b1a20 0x7fe20013db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.143 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 >> 172.21.15.96:0/66213405 conn(0x7fe200063df0 msgr2=0x7fe20013c390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.143 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 shutdown_connections 2020-11-16T22:13:49.143 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.158+0000 7fe20532e700 1 -- 172.21.15.96:0/66213405 wait complete. 2020-11-16T22:13:49.154 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 9c13a225-67c7-4d0e-8706-b3264ac0367d 6 2020-11-16T22:13:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 Processor -- start 2020-11-16T22:13:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 -- start start 2020-11-16T22:13:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdea8063040 0x7fdea80b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 0x7fdea8144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea813de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea8144540 con 0x7fdea8063040 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b4360 con 0x7fdea80b3ed0 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdeadd3d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80b4600 con 0x7fdea8144e60 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdea8063040 0x7fdea80b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.270 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdea8063040 0x7fdea80b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37706/0 (socket says 172.21.15.96:37706) 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 -- 172.21.15.96:0/194670290 learned_addr learned my addr 172.21.15.96:0/194670290 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea6ffd700 1 --2- 172.21.15.96:0/194670290 >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 0x7fdea8144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea7fff700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea813de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 -- 172.21.15.96:0/194670290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 msgr2=0x7fdea813de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea813de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 -- 172.21.15.96:0/194670290 >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 msgr2=0x7fdea8144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.271 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 --2- 172.21.15.96:0/194670290 >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 0x7fdea8144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 -- 172.21.15.96:0/194670290 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8145310 con 0x7fdea8063040 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea6ffd700 1 --2- 172.21.15.96:0/194670290 >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 0x7fdea8144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea7fff700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea813de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.282+0000 7fdea77fe700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b2ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdea80b2fc0 tx=0x7fde98009290).ready entity=mon.0 client_cookie=655b8676a24531f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdea67fc700 1 -- 172.21.15.96:0/194670290 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800bbc0 con 0x7fdea8063040 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdea67fc700 1 -- 172.21.15.96:0/194670290 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800bd20 con 0x7fdea8063040 2020-11-16T22:13:49.272 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdea67fc700 1 -- 172.21.15.96:0/194670290 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98011610 con 0x7fdea8063040 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- 172.21.15.96:0/194670290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 msgr2=0x7fdea80b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b2ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdea80b2fc0 tx=0x7fde98009290).stop 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- 172.21.15.96:0/194670290 shutdown_connections 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea813de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/194670290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b2ad0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.275 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/194670290 >> v2:172.21.15.185:3300/0 conn(0x7fdea80b3ed0 0x7fdea8144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- 172.21.15.96:0/194670290 >> 172.21.15.96:0/194670290 conn(0x7fdea8061240 msgr2=0x7fdea8061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- 172.21.15.96:0/194670290 shutdown_connections 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- 172.21.15.96:0/194670290 wait complete. 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 Processor -- start 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 -- start start 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.276 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea80ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.286+0000 7fdeadd3d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdea8144050 0x7fdea80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8142f00 con 0x7fdea8063040 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8143bb0 con 0x7fdea8144050 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea81431a0 con 0x7fdea8144e60 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37712/0 (socket says 172.21.15.96:37712) 2020-11-16T22:13:49.277 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 -- 172.21.15.96:0/3524444449 learned_addr learned my addr 172.21.15.96:0/3524444449 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea6ffd700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea80ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea7fff700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdea8144050 0x7fdea80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 -- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 msgr2=0x7fdea80ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea80ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 -- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdea8144050 msgr2=0x7fdea80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdea8144050 0x7fdea80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.278 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 -- 172.21.15.96:0/3524444449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80b7d90 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdea77fe700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fde98009500 tx=0x7fde98016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9801c680 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9801c7e0 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea80b8260 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9801cb80 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea8142c70 con 0x7fdea8063040 2020-11-16T22:13:49.279 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fde98023030 con 0x7fdea8063040 2020-11-16T22:13:49.280 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fde98026dd0 con 0x7fdea8063040 2020-11-16T22:13:49.280 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.290+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea81439e0 con 0x7fdea8063040 2020-11-16T22:13:49.280 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.294+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fde9802b060 con 0x7fdea8063040 2020-11-16T22:13:49.361 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.374+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9c13a225-67c7-4d0e-8706-b3264ac0367d", "id": 6} v 0) v1 -- 0x7fdea805d4e0 con 0x7fdea8063040 2020-11-16T22:13:49.366 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fde877fe700 1 -- 172.21.15.96:0/3524444449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c13a225-67c7-4d0e-8706-b3264ac0367d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde98021070 con 0x7fdea8063040 2020-11-16T22:13:49.367 INFO:teuthology.orchestra.run.smithi096.stdout:6 2020-11-16T22:13:49.367 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 msgr2=0x7fdea80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fde98009500 tx=0x7fde98016000).stop 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 shutdown_connections 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdea8144050 0x7fdea80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdea8063040 0x7fdea80b78a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 --2- 172.21.15.96:0/3524444449 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdea8144e60 0x7fdea80ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.368 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 >> 172.21.15.96:0/3524444449 conn(0x7fdea8061240 msgr2=0x7fdea80b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.369 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 shutdown_connections 2020-11-16T22:13:49.369 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.382+0000 7fdeadd3d700 1 -- 172.21.15.96:0/3524444449 wait complete. 2020-11-16T22:13:49.379 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new c789075c-0198-4420-9039-6f6ae5d5f906 7 2020-11-16T22:13:49.499 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 Processor -- start 2020-11-16T22:13:49.500 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 -- start start 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13ec0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0b7c20 con 0x7f13ec13e1f0 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0b3a80 con 0x7f13ec0b3570 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f1b1b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b3d20 con 0x7f13ec13de10 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13ebfff700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f13ec13e1f0 0x7f13ec0b7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52988/0 (socket says 172.21.15.96:52988) 2020-11-16T22:13:49.503 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13ebfff700 1 -- 172.21.15.96:0/3873620133 learned_addr learned my addr 172.21.15.96:0/3873620133 (peer_addr_for_me v1:172.21.15.96:0/0) 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.514+0000 7f13f0b19700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13ec0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432572844 0 0) 0x7f13ec0b7c20 con 0x7f13ec13e1f0 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d0003240 con 0x7f13ec13e1f0 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279489169 0 0) 0x7f13ec0b3a80 con 0x7f13ec0b3570 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ec0b7c20 con 0x7f13ec0b3570 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2964161318 0 0) 0x7f13d0003240 con 0x7f13ec13e1f0 2020-11-16T22:13:49.504 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 msgr2=0x7f13ec0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13ec0666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 >> v1:172.21.15.185:6789/0 conn(0x7f13ec0b3570 legacy=0x7f13ec0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> v1:172.21.15.96:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0b3fc0 con 0x7f13ec13e1f0 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v1:172.21.15.96:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f13d80014e0 con 0x7f13ec13e1f0 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 >> v1:172.21.15.96:6789/0 conn(0x7f13ec13e1f0 legacy=0x7f13ec0b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13d00038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d0003de0 0x7f13d00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.505 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d00049d0 0x7f13d0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b3d20 con 0x7f13d00049d0 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0005370 con 0x7f13d0003de0 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d000b780 con 0x7f13ec13de10 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d00049d0 0x7f13d0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f0b19700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13d00038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13ebfff700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d0003de0 0x7f13d00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.506 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 -- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 msgr2=0x7f13d00038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13d00038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 -- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d0003de0 msgr2=0x7f13d00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d0003de0 0x7f13d00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 -- 172.21.15.96:0/3873620133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0003240 con 0x7f13d00049d0 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f0b19700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec13de10 0x7f13d00038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d00049d0 0x7f13d0004e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f13e4001fd0 tx=0x7f13e4007e80).ready entity=mon.0 client_cookie=13c25c44ab660f38 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.507 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400cc40 con 0x7f13d00049d0 2020-11-16T22:13:49.508 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e400cda0 con 0x7f13d00049d0 2020-11-16T22:13:49.508 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13eb7fe700 1 -- 172.21.15.96:0/3873620133 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4006aa0 con 0x7f13d00049d0 2020-11-16T22:13:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3873620133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d00049d0 msgr2=0x7f13d0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.510 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 --2- 172.21.15.96:0/3873620133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d00049d0 0x7f13d0004e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f13e4001fd0 tx=0x7f13e4007e80).stop 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f131a700 1 -- 172.21.15.96:0/3873620133 reap_dead start 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3873620133 shutdown_connections 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3873620133 >> 172.21.15.96:0/3873620133 conn(0x7f13ec064b90 msgr2=0x7f13ec063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3873620133 shutdown_connections 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.518+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3873620133 wait complete. 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 Processor -- start 2020-11-16T22:13:49.511 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- start start 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13ec145500 0x7f13ec13f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec154690 0x7f13ec13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1405c0 con 0x7f13ec13b2a0 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0b11b0 con 0x7f13ec145500 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec13bc10 con 0x7f13ec154690 2020-11-16T22:13:49.512 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37732/0 (socket says 172.21.15.96:37732) 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 -- 172.21.15.96:0/3413143082 learned_addr learned my addr 172.21.15.96:0/3413143082 (peer_addr_for_me v2:172.21.15.96:0/0) 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13ebfff700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec154690 0x7f13ec13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f0b19700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13ec145500 0x7f13ec13f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 -- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec154690 msgr2=0x7f13ec13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec154690 0x7f13ec13f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 -- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13ec145500 msgr2=0x7f13ec13f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13ec145500 0x7f13ec13f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 -- 172.21.15.96:0/3413143082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec13f170 con 0x7f13ec13b2a0 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f131a700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f13e4011000 tx=0x7f13e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4001170 con 0x7f13ec13b2a0 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4006870 con 0x7f13ec13b2a0 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4007270 con 0x7f13ec13b2a0 2020-11-16T22:13:49.514 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0b1410 con 0x7f13ec13b2a0 2020-11-16T22:13:49.515 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec1402b0 con 0x7f13ec13b2a0 2020-11-16T22:13:49.515 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f13e4025030 con 0x7f13ec13b2a0 2020-11-16T22:13:49.515 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.522+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f13e4012400 con 0x7f13ec13b2a0 2020-11-16T22:13:49.515 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.526+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec05d200 con 0x7f13ec13b2a0 2020-11-16T22:13:49.515 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.526+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f13e4023070 con 0x7f13ec13b2a0 2020-11-16T22:13:49.596 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.610+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c789075c-0198-4420-9039-6f6ae5d5f906", "id": 7} v 0) v1 -- 0x7f13ec140090 con 0x7f13ec13b2a0 2020-11-16T22:13:49.601 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13e8ff9700 1 -- 172.21.15.96:0/3413143082 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c789075c-0198-4420-9039-6f6ae5d5f906", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13e401e840 con 0x7f13ec13b2a0 2020-11-16T22:13:49.602 INFO:teuthology.orchestra.run.smithi096.stdout:7 2020-11-16T22:13:49.603 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 msgr2=0x7f13ec13b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:49.603 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f13e4011000 tx=0x7f13e4016000).stop 2020-11-16T22:13:49.603 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 shutdown_connections 2020-11-16T22:13:49.603 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13ec145500 0x7f13ec13f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.604 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13ec13b2a0 0x7f13ec13b720 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.604 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 --2- 172.21.15.96:0/3413143082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13ec154690 0x7f13ec13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:49.604 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 >> 172.21.15.96:0/3413143082 conn(0x7f13ec064b90 msgr2=0x7f13ec067760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:49.604 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 shutdown_connections 2020-11-16T22:13:49.604 INFO:teuthology.orchestra.run.smithi096.stderr:2020-11-16T22:13:49.618+0000 7f13f1b1b700 1 -- 172.21.15.96:0/3413143082 wait complete. 2020-11-16T22:13:49.617 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-16T22:13:49.617 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-16T22:13:49.620 INFO:tasks.ceph.osd.0:Started 2020-11-16T22:13:49.620 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-16T22:13:49.620 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-16T22:13:49.622 INFO:tasks.ceph.osd.1:Started 2020-11-16T22:13:49.623 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T22:13:49.623 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T22:13:49.624 INFO:tasks.ceph.osd.2:Started 2020-11-16T22:13:49.625 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-16T22:13:49.625 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-16T22:13:49.627 INFO:tasks.ceph.osd.3:Started 2020-11-16T22:13:49.627 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-16T22:13:49.628 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-16T22:13:49.629 INFO:tasks.ceph.osd.4:Started 2020-11-16T22:13:49.630 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-16T22:13:49.630 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-16T22:13:49.631 INFO:tasks.ceph.osd.5:Started 2020-11-16T22:13:49.632 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-16T22:13:49.632 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-16T22:13:49.634 INFO:tasks.ceph.osd.6:Started 2020-11-16T22:13:49.634 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-16T22:13:49.634 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-16T22:13:49.636 INFO:tasks.ceph.osd.7:Started 2020-11-16T22:13:49.636 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-16T22:13:50.235 INFO:tasks.ceph.osd.2.smithi185.stderr:2020-11-16T22:13:50.247+0000 7fd428d3ad80 -1 Falling back to public interface 2020-11-16T22:13:50.239 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-11-16T22:13:50.254+0000 7fd67b5c1d80 -1 Falling back to public interface 2020-11-16T22:13:50.242 INFO:tasks.ceph.osd.0.smithi185.stderr:2020-11-16T22:13:50.255+0000 7feac4868d80 -1 Falling back to public interface 2020-11-16T22:13:50.243 INFO:tasks.ceph.osd.1.smithi185.stderr:2020-11-16T22:13:50.255+0000 7f46d0b76d80 -1 Falling back to public interface 2020-11-16T22:13:50.259 INFO:tasks.ceph.osd.3.smithi185.stderr:2020-11-16T22:13:50.271+0000 7ff2420a5d80 -1 Falling back to public interface 2020-11-16T22:13:50.262 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-11-16T22:13:50.278+0000 7f857c496d80 -1 Falling back to public interface 2020-11-16T22:13:50.263 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-11-16T22:13:50.278+0000 7f62987cdd80 -1 Falling back to public interface 2020-11-16T22:13:50.263 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-11-16T22:13:50.278+0000 7f7c232c5d80 -1 Falling back to public interface 2020-11-16T22:13:50.441 INFO:teuthology.orchestra.run.smithi185:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:13:50.550 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 Processor -- start 2020-11-16T22:13:50.550 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- start start 2020-11-16T22:13:50.551 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 0x7f653c144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:50.551 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f653c142f30 0x7f653c143380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:50.551 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653c0b3020 con 0x7f653c0b7610 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c0b3280 con 0x7f653c142f30 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0b3520 con 0x7f653c067910 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65416d0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 0x7f653c144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f653c142f30 0x7f653c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f653c142f30 0x7f653c143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55364/0 (socket says 172.21.15.185:55364) 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65416d0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 0x7f653c144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:48806/0 (socket says 172.21.15.185:48806) 2020-11-16T22:13:50.552 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 -- 172.21.15.185:0/1381388438 learned_addr learned my addr 172.21.15.185:0/1381388438 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:50.553 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6540ecf700 1 -- 172.21.15.185:0/1381388438 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209972800 0 0) 0x7f653c0b3020 con 0x7f653c0b7610 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 -- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 msgr2=0x7f653c144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 --2- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 0x7f653c144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 -- 172.21.15.185:0/1381388438 >> v1:172.21.15.96:6789/0 conn(0x7f653c0b7610 legacy=0x7f653c1441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 -- 172.21.15.185:0/1381388438 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c067f40 con 0x7f653c142f30 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f65426d2700 1 --2- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c142f30 0x7f653c143380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6534009d50 tx=0x7f6534009290).ready entity=mon.1 client_cookie=7cb911cec19ddb04 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:50.554 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6540ecf700 1 -- 172.21.15.185:0/1381388438 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400bbc0 con 0x7f653c142f30 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6540ecf700 1 -- 172.21.15.185:0/1381388438 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400bd20 con 0x7f653c142f30 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6540ecf700 1 -- 172.21.15.185:0/1381388438 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534011610 con 0x7f653c142f30 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c142f30 msgr2=0x7f653c143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 --2- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c142f30 0x7f653c143380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6534009d50 tx=0x7f6534009290).stop 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- 172.21.15.185:0/1381388438 shutdown_connections 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 --2- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c142f30 0x7f653c143380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.555 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 --2- 172.21.15.185:0/1381388438 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c067910 0x7f653c144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.556 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- 172.21.15.185:0/1381388438 >> 172.21.15.185:0/1381388438 conn(0x7f653c0637b0 msgr2=0x7f653c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:50.556 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- 172.21.15.185:0/1381388438 shutdown_connections 2020-11-16T22:13:50.556 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.563+0000 7f6542ed3700 1 -- 172.21.15.185:0/1381388438 wait complete. 2020-11-16T22:13:50.557 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 Processor -- start 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- start start 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f653c067910 0x7f653c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 0x7f653c13e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0ad680 con 0x7f653c067910 2020-11-16T22:13:50.558 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c141ee0 con 0x7f653c0b7610 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c142180 con 0x7f653c142f30 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65416d0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 0x7f653c13e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:48814/0 (socket says 172.21.15.185:48814) 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 -- 172.21.15.185:0/3134950616 learned_addr learned my addr 172.21.15.185:0/3134950616 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6541ed1700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f653c067910 0x7f653c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:50.559 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 -- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 msgr2=0x7f653c13e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 0x7f653c13e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 -- 172.21.15.185:0/3134950616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f653c067910 msgr2=0x7f653c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f653c067910 0x7f653c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 -- 172.21.15.185:0/3134950616 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c142420 con 0x7f653c142f30 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65416d0700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 0x7f653c13e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f65426d2700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6534009fd0 tx=0x7f6534016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:50.560 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534011e70 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400be90 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c0af520 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534010cf0 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c0ad300 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f6534017070 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f6534026b60 con 0x7f653c142f30 2020-11-16T22:13:50.561 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.567+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c05d200 con 0x7f653c142f30 2020-11-16T22:13:50.562 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.575+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f6534015020 con 0x7f653c142f30 2020-11-16T22:13:50.633 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.643+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f653c0b17e0 con 0x7f653c142f30 2020-11-16T22:13:50.633 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6529ffb700 1 -- 172.21.15.185:0/3134950616 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7001 (secure 0 0 0) 0x7f6534010e50 con 0x7f653c142f30 2020-11-16T22:13:50.634 INFO:teuthology.misc.health.smithi185.stdout: 2020-11-16T22:13:50.634 INFO:teuthology.misc.health.smithi185.stdout:{"epoch":10,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:13:49.613920+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 msgr2=0x7f653c13ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6534009fd0 tx=0x7f6534016000).stop 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 shutdown_connections 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f653c142f30 0x7f653c13ee50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f653c067910 0x7f653c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.635 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 --2- 172.21.15.185:0/3134950616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f653c0b7610 0x7f653c13e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:50.636 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 >> 172.21.15.185:0/3134950616 conn(0x7f653c0637b0 msgr2=0x7f653c143d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:50.636 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 shutdown_connections 2020-11-16T22:13:50.636 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:50.647+0000 7f6542ed3700 1 -- 172.21.15.185:0/3134950616 wait complete. 2020-11-16T22:13:50.645 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-16T22:13:51.446 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-11-16T22:13:51.462+0000 7f7c232c5d80 -1 osd.7 0 log_to_monitors {default=true} 2020-11-16T22:13:51.642 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-11-16T22:13:51.658+0000 7fd67b5c1d80 -1 osd.4 0 log_to_monitors {default=true} 2020-11-16T22:13:51.652 INFO:tasks.ceph.osd.2.smithi185.stderr:2020-11-16T22:13:51.663+0000 7fd428d3ad80 -1 osd.2 0 log_to_monitors {default=true} 2020-11-16T22:13:51.661 INFO:tasks.ceph.osd.1.smithi185.stderr:2020-11-16T22:13:51.671+0000 7f46d0b76d80 -1 osd.1 0 log_to_monitors {default=true} 2020-11-16T22:13:51.677 INFO:tasks.ceph.osd.0.smithi185.stderr:2020-11-16T22:13:51.687+0000 7feac4868d80 -1 osd.0 0 log_to_monitors {default=true} 2020-11-16T22:13:51.695 INFO:tasks.ceph.osd.3.smithi185.stderr:2020-11-16T22:13:51.707+0000 7ff2420a5d80 -1 osd.3 0 log_to_monitors {default=true} 2020-11-16T22:13:51.717 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-11-16T22:13:51.730+0000 7f857c496d80 -1 osd.6 0 log_to_monitors {default=true} 2020-11-16T22:13:51.717 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-11-16T22:13:51.734+0000 7f62987cdd80 -1 osd.5 0 log_to_monitors {default=true} 2020-11-16T22:13:53.055 INFO:tasks.ceph.osd.2.smithi185.stderr:2020-11-16T22:13:53.067+0000 7fd420204700 -1 osd.2 0 waiting for initial osdmap 2020-11-16T22:13:53.056 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-11-16T22:13:53.070+0000 7f629149a700 -1 osd.5 0 waiting for initial osdmap 2020-11-16T22:13:53.057 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-11-16T22:13:53.070+0000 7fd672a8b700 -1 osd.4 0 waiting for initial osdmap 2020-11-16T22:13:53.057 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-11-16T22:13:53.070+0000 7f7c1a78f700 -1 osd.7 0 waiting for initial osdmap 2020-11-16T22:13:53.057 INFO:tasks.ceph.osd.0.smithi185.stderr:2020-11-16T22:13:53.067+0000 7feabbd32700 -1 osd.0 0 waiting for initial osdmap 2020-11-16T22:13:53.058 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-11-16T22:13:53.074+0000 7f8573960700 -1 osd.6 0 waiting for initial osdmap 2020-11-16T22:13:53.058 INFO:tasks.ceph.osd.1.smithi185.stderr:2020-11-16T22:13:53.071+0000 7f46c8040700 -1 osd.1 0 waiting for initial osdmap 2020-11-16T22:13:53.061 INFO:tasks.ceph.osd.3.smithi185.stderr:2020-11-16T22:13:53.075+0000 7ff23956f700 -1 osd.3 0 waiting for initial osdmap 2020-11-16T22:13:57.450 INFO:teuthology.orchestra.run.smithi185:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T22:13:57.570 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 Processor -- start 2020-11-16T22:13:57.570 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- start start 2020-11-16T22:13:57.571 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc45c0b7610 0x7fc45c067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 0x7fc45c1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c144b90 0x7fc45c147590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3020 con 0x7fc45c1441f0 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3280 con 0x7fc45c0b7610 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b3520 con 0x7fc45c144b90 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc45c0b7610 0x7fc45c067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.572 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc45c0b7610 0x7fc45c067910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55584/0 (socket says 172.21.15.185:55584) 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 -- 172.21.15.185:0/336656839 learned_addr learned my addr 172.21.15.185:0/336656839 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc46229f700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c144b90 0x7fc45c147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc46129d700 1 --2- 172.21.15.185:0/336656839 >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 0x7fc45c1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 -- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c144b90 msgr2=0x7fc45c147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c144b90 0x7fc45c147590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 -- 172.21.15.185:0/336656839 >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 msgr2=0x7fc45c1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:57.573 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 --2- 172.21.15.185:0/336656839 >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 0x7fc45c1446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.574 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 -- 172.21.15.185:0/336656839 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c067f40 con 0x7fc45c0b7610 2020-11-16T22:13:57.574 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc46129d700 1 --2- 172.21.15.185:0/336656839 >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 0x7fc45c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:13:57.575 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc461a9e700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c0b7610 0x7fc45c067910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc45c0b7d50 tx=0x7fc44c006eb0).ready entity=mon.1 client_cookie=6d486c08a6394c4c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.575 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc460a9c700 1 -- 172.21.15.185:0/336656839 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c009810 con 0x7fc45c0b7610 2020-11-16T22:13:57.575 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc460a9c700 1 -- 172.21.15.185:0/336656839 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c009970 con 0x7fc45c0b7610 2020-11-16T22:13:57.575 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc460a9c700 1 -- 172.21.15.185:0/336656839 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c009cf0 con 0x7fc45c0b7610 2020-11-16T22:13:57.575 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c0b7610 msgr2=0x7fc45c067910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c0b7610 0x7fc45c067910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc45c0b7d50 tx=0x7fc44c006eb0).stop 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- 172.21.15.185:0/336656839 shutdown_connections 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c144b90 0x7fc45c147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- 172.21.15.185:0/336656839 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c0b7610 0x7fc45c067910 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 --2- 172.21.15.185:0/336656839 >> v2:172.21.15.96:3300/0 conn(0x7fc45c1441f0 0x7fc45c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- 172.21.15.185:0/336656839 >> 172.21.15.185:0/336656839 conn(0x7fc45c0637b0 msgr2=0x7fc45c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- 172.21.15.185:0/336656839 shutdown_connections 2020-11-16T22:13:57.576 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.583+0000 7fc462aa0700 1 -- 172.21.15.185:0/336656839 wait complete. 2020-11-16T22:13:57.577 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 Processor -- start 2020-11-16T22:13:57.577 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- start start 2020-11-16T22:13:57.578 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 0x7fc45c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.578 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc45c1441f0 0x7fc45c143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c156fe0 con 0x7fc45c1441f0 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c142ea0 con 0x7fc45c144b90 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c153750 con 0x7fc45c0b7610 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc461a9e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 0x7fc45c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.579 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55592/0 (socket says 172.21.15.185:55592) 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 -- 172.21.15.185:0/1028402524 learned_addr learned my addr 172.21.15.185:0/1028402524 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46129d700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc45c1441f0 0x7fc45c143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 -- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 msgr2=0x7fc45c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 0x7fc45c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 -- 172.21.15.185:0/1028402524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc45c1441f0 msgr2=0x7fc45c143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc45c1441f0 0x7fc45c143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.580 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 -- 172.21.15.185:0/1028402524 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c1539f0 con 0x7fc45c144b90 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc461a9e700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 0x7fc45c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc46229f700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc45400eba0 tx=0x7fc454006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45400b2e0 con 0x7fc45c144b90 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45400c820 con 0x7fc45c144b90 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c153480 con 0x7fc45c144b90 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45400cbe0 con 0x7fc45c144b90 2020-11-16T22:13:57.581 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.587+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c154380 con 0x7fc45c144b90 2020-11-16T22:13:57.582 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c1430a0 con 0x7fc45c144b90 2020-11-16T22:13:57.582 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc45401c460 con 0x7fc45c144b90 2020-11-16T22:13:57.582 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc451ffb700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 0x7fc434057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.582 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fc45406db00 con 0x7fc45c144b90 2020-11-16T22:13:57.583 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc461a9e700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 0x7fc434057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.583 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.591+0000 7fc461a9e700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 0x7fc434057520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc44c00b1b0 tx=0x7fc44c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.584 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.595+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc454044190 con 0x7fc45c144b90 2020-11-16T22:13:57.740 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.751+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc45c153b80 con 0x7fc45c144b90 2020-11-16T22:13:57.742 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc451ffb700 1 -- 172.21.15.185:0/1028402524 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12152 (secure 0 0 0) 0x7fc4540477b0 con 0x7fc45c144b90 2020-11-16T22:13:57.742 INFO:teuthology.misc.health.smithi185.stdout: 2020-11-16T22:13:57.743 INFO:teuthology.misc.health.smithi185.stdout:{"epoch":15,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:13:56.087497+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:13:57.744 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 msgr2=0x7fc434057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.744 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 0x7fc434057520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc44c00b1b0 tx=0x7fc44c015000).stop 2020-11-16T22:13:57.744 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 msgr2=0x7fc45c153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc45400eba0 tx=0x7fc454006df0).stop 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 shutdown_connections 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc434055060 0x7fc434057520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc45c144b90 0x7fc45c153e90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc45c0b7610 0x7fc45c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.745 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 --2- 172.21.15.185:0/1028402524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc45c1441f0 0x7fc45c143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.746 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 >> 172.21.15.185:0/1028402524 conn(0x7fc45c0637b0 msgr2=0x7fc45c060950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:57.746 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 shutdown_connections 2020-11-16T22:13:57.746 INFO:teuthology.misc.health.smithi185.stderr:2020-11-16T22:13:57.755+0000 7fc462aa0700 1 -- 172.21.15.185:0/1028402524 wait complete. 2020-11-16T22:13:57.756 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-16T22:13:57.756 INFO:tasks.ceph:Creating RBD pool 2020-11-16T22:13:57.756 INFO:teuthology.orchestra.run.smithi185:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-16T22:13:57.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.895+0000 7fc8eec4a700 1 Processor -- start 2020-11-16T22:13:57.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.895+0000 7fc8eec4a700 1 -- start start 2020-11-16T22:13:57.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc8e80ba800 0x7fc8e80b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc8e8143470 0x7fc8e81438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e80b7290 con 0x7fc8e80b5d70 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e80b4960 con 0x7fc8e8143470 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e80b4bf0 con 0x7fc8e80ba800 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc8e80ba800 0x7fc8e80b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc8e80ba800 0x7fc8e80b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55598/0 (socket says 172.21.15.185:55598) 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 -- 172.21.15.185:0/1258882786 learned_addr learned my addr 172.21.15.185:0/1258882786 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:57.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ee449700 1 --2- 172.21.15.185:0/1258882786 >> v2:172.21.15.96:3300/0 conn(0x7fc8e8143470 0x7fc8e81438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 -- 172.21.15.185:0/1258882786 >> v2:172.21.15.96:3300/0 conn(0x7fc8e8143470 msgr2=0x7fc8e81438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 --2- 172.21.15.185:0/1258882786 >> v2:172.21.15.96:3300/0 conn(0x7fc8e8143470 0x7fc8e81438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 -- 172.21.15.185:0/1258882786 >> v1:172.21.15.96:6789/0 conn(0x7fc8e80b5d70 legacy=0x7fc8e80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 -- 172.21.15.185:0/1258882786 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80b74c0 con 0x7fc8e80ba800 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ed447700 1 --2- 172.21.15.185:0/1258882786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80ba800 0x7fc8e80b4470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc8e8080bf0 tx=0x7fc8d8006eb0).ready entity=mon.1 client_cookie=a0a42ebf7363f22d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ecc46700 1 -- 172.21.15.185:0/1258882786 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8009890 con 0x7fc8e80ba800 2020-11-16T22:13:57.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ecc46700 1 -- 172.21.15.185:0/1258882786 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d80099f0 con 0x7fc8e80ba800 2020-11-16T22:13:57.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8ecc46700 1 -- 172.21.15.185:0/1258882786 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8009d70 con 0x7fc8e80ba800 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1258882786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80ba800 msgr2=0x7fc8e80b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1258882786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80ba800 0x7fc8e80b4470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc8e8080bf0 tx=0x7fc8d8006eb0).stop 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1258882786 shutdown_connections 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1258882786 >> v2:172.21.15.96:3300/0 conn(0x7fc8e8143470 0x7fc8e81438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1258882786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80ba800 0x7fc8e80b4470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1258882786 >> 172.21.15.185:0/1258882786 conn(0x7fc8e805ba80 msgr2=0x7fc8e805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:57.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1258882786 shutdown_connections 2020-11-16T22:13:57.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.899+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1258882786 wait complete. 2020-11-16T22:13:57.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 Processor -- start 2020-11-16T22:13:57.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- start start 2020-11-16T22:13:57.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc8e80ba800 0x7fc8e80af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8143470 0x7fc8e8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80ae9c0 con 0x7fc8e8143470 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80afde0 con 0x7fc8e80b5d70 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80afa90 con 0x7fc8e80ba800 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55604/0 (socket says 172.21.15.185:55604) 2020-11-16T22:13:57.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 -- 172.21.15.185:0/1786864912 learned_addr learned my addr 172.21.15.185:0/1786864912 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8ed447700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc8e80ba800 0x7fc8e80af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8ee449700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8143470 0x7fc8e8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 -- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc8e80ba800 msgr2=0x7fc8e80af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc8e80ba800 0x7fc8e80af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 -- 172.21.15.185:0/1786864912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8143470 msgr2=0x7fc8e8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8143470 0x7fc8e8152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:57.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 -- 172.21.15.185:0/1786864912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80b0010 con 0x7fc8e80b5d70 2020-11-16T22:13:57.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8edc48700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc8e400c790 tx=0x7fc8e4007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400e560 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e400e6e0 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e80b1530 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400eaa0 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.903+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e8148230 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e80b6df0 con 0x7fc8e80b5d70 2020-11-16T22:13:57.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc8e4010070 con 0x7fc8e80b5d70 2020-11-16T22:13:57.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8d5ffb700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 0x7fc8c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:57.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fc8e406ce10 con 0x7fc8e80b5d70 2020-11-16T22:13:57.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8ed447700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 0x7fc8c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:57.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.907+0000 7fc8ed447700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 0x7fc8c0057520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc8d8006ef0 tx=0x7fc8d8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:57.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:57.911+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc8e80b6df0 con 0x7fc8e80b5d70 2020-11-16T22:13:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.063+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc8e80b6df0 con 0x7fc8e80b5d70 2020-11-16T22:13:58.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.923+0000 7fc8d5ffb700 1 -- 172.21.15.185:0/1786864912 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc8e80b6df0 con 0x7fc8e80b5d70 2020-11-16T22:13:58.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 msgr2=0x7fc8c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:58.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 0x7fc8c0057520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc8d8006ef0 tx=0x7fc8d8015000).stop 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 msgr2=0x7fc8e8154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc8e400c790 tx=0x7fc8e4007ba0).stop 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 shutdown_connections 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc8c0055060 0x7fc8c0057520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc8e80b5d70 0x7fc8e8154120 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8143470 0x7fc8e8152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 --2- 172.21.15.185:0/1786864912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc8e80ba800 0x7fc8e80af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 >> 172.21.15.185:0/1786864912 conn(0x7fc8e805ba80 msgr2=0x7fc8e80b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 shutdown_connections 2020-11-16T22:13:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:58.927+0000 7fc8eec4a700 1 -- 172.21.15.185:0/1786864912 wait complete. 2020-11-16T22:13:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'rbd' created 2020-11-16T22:13:58.930 INFO:teuthology.orchestra.run.smithi185:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-16T22:13:59.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.063+0000 7f2bbe09e700 1 Processor -- start 2020-11-16T22:13:59.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.063+0000 7f2bbe09e700 1 -- start start 2020-11-16T22:13:59.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bb80b41f0 0x7f2bb80b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2bb8067f80 0x7f2bb8142030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb81456d0 con 0x7f2bb8068430 2020-11-16T22:13:59.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb8145970 con 0x7f2bb80b41f0 2020-11-16T22:13:59.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb8145c10 con 0x7f2bb8067f80 2020-11-16T22:13:59.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb7fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2bb8068430 0x7f2bb80ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49062/0 (socket says 172.21.15.185:49062) 2020-11-16T22:13:59.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb7fff700 1 -- 172.21.15.185:0/1473239294 learned_addr learned my addr 172.21.15.185:0/1473239294 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:13:59.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/1473239294 >> v2:172.21.15.185:3300/0 conn(0x7f2bb8067f80 0x7f2bb8142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb77fe700 1 --2- 172.21.15.185:0/1473239294 >> v2:172.21.15.96:3300/0 conn(0x7f2bb80b41f0 0x7f2bb80b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 -- 172.21.15.185:0/1473239294 >> v2:172.21.15.96:3300/0 conn(0x7f2bb80b41f0 msgr2=0x7f2bb80b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:13:59.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/1473239294 >> v2:172.21.15.96:3300/0 conn(0x7f2bb80b41f0 0x7f2bb80b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 -- 172.21.15.185:0/1473239294 >> v1:172.21.15.185:6789/0 conn(0x7f2bb8068430 legacy=0x7f2bb80ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 -- 172.21.15.185:0/1473239294 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8146350 con 0x7f2bb8067f80 2020-11-16T22:13:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/1473239294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb8142030 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2bb80ad410 tx=0x7f2ba0006df0).ready entity=mon.1 client_cookie=c2d99b14ad45a9c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb67fc700 1 -- 172.21.15.185:0/1473239294 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00096b0 con 0x7f2bb8067f80 2020-11-16T22:13:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb67fc700 1 -- 172.21.15.185:0/1473239294 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0009810 con 0x7f2bb8067f80 2020-11-16T22:13:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bb67fc700 1 -- 172.21.15.185:0/1473239294 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0009b90 con 0x7f2bb8067f80 2020-11-16T22:13:59.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 -- 172.21.15.185:0/1473239294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 msgr2=0x7f2bb8142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.067+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/1473239294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb8142030 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2bb80ad410 tx=0x7f2ba0006df0).stop 2020-11-16T22:13:59.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 -- 172.21.15.185:0/1473239294 shutdown_connections 2020-11-16T22:13:59.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/1473239294 >> v2:172.21.15.96:3300/0 conn(0x7f2bb80b41f0 0x7f2bb80b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/1473239294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb8142030 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 -- 172.21.15.185:0/1473239294 >> 172.21.15.185:0/1473239294 conn(0x7f2bb8061240 msgr2=0x7f2bb8061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:59.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 -- 172.21.15.185:0/1473239294 shutdown_connections 2020-11-16T22:13:59.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 -- 172.21.15.185:0/1473239294 wait complete. 2020-11-16T22:13:59.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 Processor -- start 2020-11-16T22:13:59.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.071+0000 7f2bbe09e700 1 -- start start 2020-11-16T22:13:59.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb8068430 0x7f2bb80634e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 0x7f2bb80639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8065240 con 0x7f2bb80b41f0 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb809cd40 con 0x7f2bb8067f80 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb809d0f0 con 0x7f2bb8068430 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55626/0 (socket says 172.21.15.185:55626) 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 -- 172.21.15.185:0/3559771896 learned_addr learned my addr 172.21.15.185:0/3559771896 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb8068430 0x7f2bb80634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb7fff700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 0x7f2bb80639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 -- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb8068430 msgr2=0x7f2bb80634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb8068430 0x7f2bb80634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 -- 172.21.15.185:0/3559771896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 msgr2=0x7f2bb80639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 0x7f2bb80639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 -- 172.21.15.185:0/3559771896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb809d390 con 0x7f2bb8067f80 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb7fff700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 0x7f2bb80639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb77fe700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f2ba800db60 tx=0x7f2ba8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8009790 con 0x7f2bb8067f80 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba80098f0 con 0x7f2bb8067f80 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb81402c0 con 0x7f2bb8067f80 2020-11-16T22:13:59.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8009c90 con 0x7f2bb8067f80 2020-11-16T22:13:59.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8142280 con 0x7f2bb8067f80 2020-11-16T22:13:59.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2ba8013480 con 0x7f2bb8067f80 2020-11-16T22:13:59.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 0x7f2b94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:13:59.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb809a360 con 0x7f2bb8067f80 2020-11-16T22:13:59.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f2ba806de20 con 0x7f2bb8067f80 2020-11-16T22:13:59.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 0x7f2b94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:13:59.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.075+0000 7f2bb6ffd700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 0x7f2b94057520 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2ba000a010 tx=0x7f2ba0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:13:59.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.087+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ba8047d90 con 0x7f2bb8067f80 2020-11-16T22:13:59.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.243+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2bb805d4e0 con 0x7f2bb8067f80 2020-11-16T22:13:59.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.935+0000 7f2baf7fe700 1 -- 172.21.15.185:0/3559771896 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2ba8047980 con 0x7f2bb8067f80 2020-11-16T22:13:59.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 msgr2=0x7f2b94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 0x7f2b94057520 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2ba000a010 tx=0x7f2ba0015000).stop 2020-11-16T22:13:59.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 msgr2=0x7f2bb813fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f2ba800db60 tx=0x7f2ba8008040).stop 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 shutdown_connections 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b94055060 0x7f2b94057520 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb8067f80 0x7f2bb813fdd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb80b41f0 0x7f2bb80639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 --2- 172.21.15.185:0/3559771896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb8068430 0x7f2bb80634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:13:59.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 >> 172.21.15.185:0/3559771896 conn(0x7f2bb8061240 msgr2=0x7f2bb80adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:13:59.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 shutdown_connections 2020-11-16T22:13:59.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:13:59.939+0000 7f2bbe09e700 1 -- 172.21.15.185:0/3559771896 wait complete. 2020-11-16T22:13:59.930 INFO:teuthology.orchestra.run.smithi185.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-16T22:13:59.943 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-16T22:13:59.944 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:13:59.944 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:13:59.946 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:13:59.946 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:13:59.947 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:13:59.948 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:13:59.949 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:13:59.949 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:13:59.951 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:13:59.951 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-16T22:13:59.951 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:00.012 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:14:00.023 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:14:00.038 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:14:00.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.083+0000 7f436acf9700 1 Processor -- start 2020-11-16T22:14:00.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.083+0000 7f436acf9700 1 -- start start 2020-11-16T22:14:00.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.083+0000 7f436acf9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43640b2950 0x7f43640b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.083+0000 7f436acf9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.083+0000 7f436acf9700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4364065fe0 con 0x7f4364067070 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4364062e70 con 0x7f43640b2950 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4364066280 con 0x7f43641417d0 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4368a95700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43640b2950 0x7f43640b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49136/0 (socket says 172.21.15.185:49136) 2020-11-16T22:14:00.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 -- 172.21.15.185:0/2693476874 learned_addr learned my addr 172.21.15.185:0/2693476874 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:00.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 -- 172.21.15.185:0/2693476874 >> v2:172.21.15.185:3300/0 conn(0x7f43640b2950 msgr2=0x7f43640b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 --2- 172.21.15.185:0/2693476874 >> v2:172.21.15.185:3300/0 conn(0x7f43640b2950 0x7f43640b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 -- 172.21.15.185:0/2693476874 >> v1:172.21.15.96:6789/0 conn(0x7f4364067070 legacy=0x7f4364062760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 -- 172.21.15.185:0/2693476874 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364066520 con 0x7f43641417d0 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f4363fff700 1 --2- 172.21.15.185:0/2693476874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f4350006fd0 tx=0x7f4350006eb0).ready entity=mon.2 client_cookie=9bac7474bf73f40c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f43637fe700 1 -- 172.21.15.185:0/2693476874 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350009810 con 0x7f43641417d0 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f43637fe700 1 -- 172.21.15.185:0/2693476874 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4350009970 con 0x7f43641417d0 2020-11-16T22:14:00.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f43637fe700 1 -- 172.21.15.185:0/2693476874 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350009cf0 con 0x7f43641417d0 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- 172.21.15.185:0/2693476874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 msgr2=0x7f43640669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 --2- 172.21.15.185:0/2693476874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f4350006fd0 tx=0x7f4350006eb0).stop 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- 172.21.15.185:0/2693476874 shutdown_connections 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 --2- 172.21.15.185:0/2693476874 >> v2:172.21.15.185:3300/0 conn(0x7f43640b2950 0x7f43640b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 --2- 172.21.15.185:0/2693476874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43641417d0 0x7f43640669c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- 172.21.15.185:0/2693476874 >> 172.21.15.185:0/2693476874 conn(0x7f43640b4410 msgr2=0x7f4364060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:00.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- 172.21.15.185:0/2693476874 shutdown_connections 2020-11-16T22:14:00.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- 172.21.15.185:0/2693476874 wait complete. 2020-11-16T22:14:00.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 Processor -- start 2020-11-16T22:14:00.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.087+0000 7f436acf9700 1 -- start start 2020-11-16T22:14:00.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43641417d0 0x7f4364130a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364143130 con 0x7f43641417d0 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364144890 con 0x7f43640b2950 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4364144b30 con 0x7f4364067070 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4363fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49140/0 (socket says 172.21.15.185:49140) 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4363fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55698/0 (socket says 172.21.15.185:55698) 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 -- 172.21.15.185:0/4014851867 learned_addr learned my addr 172.21.15.185:0/4014851867 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4369296700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43641417d0 0x7f4364130a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 -- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 msgr2=0x7f4364130520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:00.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 -- 172.21.15.185:0/4014851867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43641417d0 msgr2=0x7f4364130a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43641417d0 0x7f4364130a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 -- 172.21.15.185:0/4014851867 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364144dd0 con 0x7f4364067070 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4363fff700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4368a95700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f435800eba0 tx=0x7f4358006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800c9b0 con 0x7f4364067070 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435800cb10 con 0x7f4364067070 2020-11-16T22:14:00.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4364132450 con 0x7f4364067070 2020-11-16T22:14:00.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800ce90 con 0x7f4364067070 2020-11-16T22:14:00.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4364142e50 con 0x7f4364067070 2020-11-16T22:14:00.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436405d4e0 con 0x7f4364067070 2020-11-16T22:14:00.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4358012600 con 0x7f4364067070 2020-11-16T22:14:00.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 0x7f433c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:00.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f435806ded0 con 0x7f4364067070 2020-11-16T22:14:00.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.091+0000 7f4363fff700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 0x7f433c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:00.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.095+0000 7f4363fff700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 0x7f433c057520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f43640b1430 tx=0x7f4350015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:00.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.099+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4358044460 con 0x7f4364067070 2020-11-16T22:14:00.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.255+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4364061e90 con 0x7f4364067070 2020-11-16T22:14:00.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.255+0000 7f4360ff9700 1 -- 172.21.15.185:0/4014851867 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13685 (secure 0 0 0) 0x7f4358047a80 con 0x7f4364067070 2020-11-16T22:14:00.244 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:00.245 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":17,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:13:59.923486+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:04.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 msgr2=0x7f433c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 0x7f433c057520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f43640b1430 tx=0x7f4350015000).stop 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 msgr2=0x7f4364131f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f435800eba0 tx=0x7f4358006df0).stop 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 shutdown_connections 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f433c055060 0x7f433c057520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43641417d0 0x7f4364130a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4364067070 0x7f4364131f60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 --2- 172.21.15.185:0/4014851867 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43640b2950 0x7f4364130520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 >> 172.21.15.185:0/4014851867 conn(0x7f43640b4410 msgr2=0x7f43640b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:04.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 shutdown_connections 2020-11-16T22:14:04.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:00.259+0000 7f436acf9700 1 -- 172.21.15.185:0/4014851867 wait complete. 2020-11-16T22:14:04.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:14:04.518 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:04.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 Processor -- start 2020-11-16T22:14:04.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- start start 2020-11-16T22:14:04.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff4280b3920 0x7ff428148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42814c660 con 0x7ff4280b13d0 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4280ac120 con 0x7ff4280b3920 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280ac3c0 con 0x7ff4280b3080 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42ecd1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff4280b3920 0x7ff428148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49150/0 (socket says 172.21.15.185:49150) 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 -- 172.21.15.185:0/2483139839 learned_addr learned my addr 172.21.15.185:0/2483139839 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:04.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 -- 172.21.15.185:0/2483139839 >> v2:172.21.15.185:3300/0 conn(0x7ff4280b3920 msgr2=0x7ff428148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 --2- 172.21.15.185:0/2483139839 >> v2:172.21.15.185:3300/0 conn(0x7ff4280b3920 0x7ff428148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 -- 172.21.15.185:0/2483139839 >> v1:172.21.15.96:6789/0 conn(0x7ff4280b13d0 legacy=0x7ff4280b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 -- 172.21.15.185:0/2483139839 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4280acd50 con 0x7ff4280b3080 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42e4d0700 1 --2- 172.21.15.185:0/2483139839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff424006fd0 tx=0x7ff424006eb0).ready entity=mon.2 client_cookie=dc2dd96c1b86d134 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42d4ce700 1 -- 172.21.15.185:0/2483139839 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424009810 con 0x7ff4280b3080 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42d4ce700 1 -- 172.21.15.185:0/2483139839 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff424009970 con 0x7ff4280b3080 2020-11-16T22:14:04.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42d4ce700 1 -- 172.21.15.185:0/2483139839 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424009cf0 con 0x7ff4280b3080 2020-11-16T22:14:04.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- 172.21.15.185:0/2483139839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 msgr2=0x7ff4280ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/2483139839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff424006fd0 tx=0x7ff424006eb0).stop 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- 172.21.15.185:0/2483139839 shutdown_connections 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/2483139839 >> v2:172.21.15.185:3300/0 conn(0x7ff4280b3920 0x7ff428148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/2483139839 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280ac860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- 172.21.15.185:0/2483139839 >> 172.21.15.185:0/2483139839 conn(0x7ff428060130 msgr2=0x7ff428060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- 172.21.15.185:0/2483139839 shutdown_connections 2020-11-16T22:14:04.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.671+0000 7ff42f4d2700 1 -- 172.21.15.185:0/2483139839 wait complete. 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 Processor -- start 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- start start 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4280b13d0 0x7ff4280aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4280b3920 0x7ff4281114e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428110a30 con 0x7ff4280b13d0 2020-11-16T22:14:04.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428110170 con 0x7ff4280b3920 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280ae8f0 con 0x7ff4280b3080 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49158/0 (socket says 172.21.15.185:49158) 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42ecd1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4280b3920 0x7ff4281114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 -- 172.21.15.185:0/3370016036 learned_addr learned my addr 172.21.15.185:0/3370016036 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42e4d0700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4280b13d0 0x7ff4280aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 -- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4280b3920 msgr2=0x7ff4281114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4280b3920 0x7ff4281114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 -- 172.21.15.185:0/3370016036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4280b13d0 msgr2=0x7ff4280aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4280b13d0 0x7ff4280aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 -- 172.21.15.185:0/3370016036 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4280afd00 con 0x7ff4280b3080 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42dccf700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff41c00c740 tx=0x7ff41c007bc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c007020 con 0x7ff4280b3080 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c007180 con 0x7ff4280b3080 2020-11-16T22:14:04.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4280affa0 con 0x7ff4280b3080 2020-11-16T22:14:04.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0064f0 con 0x7ff4280b3080 2020-11-16T22:14:04.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4280b6cb0 con 0x7ff4280b3080 2020-11-16T22:14:04.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.675+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4280aed90 con 0x7ff4280b3080 2020-11-16T22:14:04.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.679+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff41c010070 con 0x7ff4280b3080 2020-11-16T22:14:04.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.679+0000 7ff41a7fc700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 0x7ff400057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.679+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff41c06cd10 con 0x7ff4280b3080 2020-11-16T22:14:04.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.679+0000 7ff42e4d0700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 0x7ff400057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.679+0000 7ff42e4d0700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 0x7ff400057520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff42400b1b0 tx=0x7ff424015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.683+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff41c0432a0 con 0x7ff4280b3080 2020-11-16T22:14:04.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.835+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff42805d200 con 0x7ff4280b3080 2020-11-16T22:14:04.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.835+0000 7ff41a7fc700 1 -- 172.21.15.185:0/3370016036 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff41c0468c0 con 0x7ff4280b3080 2020-11-16T22:14:04.824 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:04.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 msgr2=0x7ff400057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 0x7ff400057520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff42400b1b0 tx=0x7ff424015000).stop 2020-11-16T22:14:04.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 msgr2=0x7ff4280b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff41c00c740 tx=0x7ff41c007bc0).stop 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 shutdown_connections 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff400055060 0x7ff400057520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4280b3920 0x7ff4281114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4280b13d0 0x7ff4280aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 --2- 172.21.15.185:0/3370016036 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4280b3080 0x7ff4280b6400 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 >> 172.21.15.185:0/3370016036 conn(0x7ff428060130 msgr2=0x7ff428146710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:04.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 shutdown_connections 2020-11-16T22:14:04.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.839+0000 7ff42f4d2700 1 -- 172.21.15.185:0/3370016036 wait complete. 2020-11-16T22:14:04.838 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:04.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 Processor -- start 2020-11-16T22:14:04.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 -- start start 2020-11-16T22:14:04.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f57380b74b0 0x7f5738144c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57380afc40 0x7f57380ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57380b02e0 con 0x7f57380adbe0 2020-11-16T22:14:04.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5738143fa0 con 0x7f57380b74b0 2020-11-16T22:14:04.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573fb3f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57380b00d0 con 0x7f57380afc40 2020-11-16T22:14:04.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57380afc40 0x7f57380ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57380afc40 0x7f57380ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55720/0 (socket says 172.21.15.185:55720) 2020-11-16T22:14:04.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 -- 172.21.15.185:0/3869755685 learned_addr learned my addr 172.21.15.185:0/3869755685 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:04.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d8db700 1 --2- 172.21.15.185:0/3869755685 >> v2:172.21.15.96:3300/0 conn(0x7f57380b74b0 0x7f5738144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 -- 172.21.15.185:0/3869755685 >> v2:172.21.15.96:3300/0 conn(0x7f57380b74b0 msgr2=0x7f5738144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:04.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 --2- 172.21.15.185:0/3869755685 >> v2:172.21.15.96:3300/0 conn(0x7f57380b74b0 0x7f5738144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 -- 172.21.15.185:0/3869755685 >> v1:172.21.15.96:6789/0 conn(0x7f57380adbe0 legacy=0x7f5738147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 -- 172.21.15.185:0/3869755685 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380b0580 con 0x7f57380afc40 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573d0da700 1 --2- 172.21.15.185:0/3869755685 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57380ad6f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f57381450f0 tx=0x7f572c006e50).ready entity=mon.1 client_cookie=5c6f554eb0d77b40 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573c8d9700 1 -- 172.21.15.185:0/3869755685 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c006030 con 0x7f57380afc40 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573c8d9700 1 -- 172.21.15.185:0/3869755685 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c006190 con 0x7f57380afc40 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.979+0000 7f573c8d9700 1 -- 172.21.15.185:0/3869755685 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c006510 con 0x7f57380afc40 2020-11-16T22:14:04.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/3869755685 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 msgr2=0x7f57380ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- 172.21.15.185:0/3869755685 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57380ad6f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f57381450f0 tx=0x7f572c006e50).stop 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/3869755685 shutdown_connections 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- 172.21.15.185:0/3869755685 >> v2:172.21.15.96:3300/0 conn(0x7f57380b74b0 0x7f5738144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- 172.21.15.185:0/3869755685 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57380ad6f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/3869755685 >> 172.21.15.185:0/3869755685 conn(0x7f5738067860 msgr2=0x7f5738067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/3869755685 shutdown_connections 2020-11-16T22:14:04.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/3869755685 wait complete. 2020-11-16T22:14:04.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 Processor -- start 2020-11-16T22:14:04.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- start start 2020-11-16T22:14:04.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 0x7f573813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5738145ab0 con 0x7f57380b74b0 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5738146990 con 0x7f57380afc40 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5738146c30 con 0x7f57380adbe0 2020-11-16T22:14:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d0da700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d0da700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55726/0 (socket says 172.21.15.185:55726) 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49172/0 (socket says 172.21.15.185:49172) 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d0da700 1 -- 172.21.15.185:0/2207862028 learned_addr learned my addr 172.21.15.185:0/2207862028 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573e0dc700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 0x7f573813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 -- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 msgr2=0x7f57381462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 -- 172.21.15.185:0/2207862028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 msgr2=0x7f573813e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 0x7f573813e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 -- 172.21.15.185:0/2207862028 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738156cd0 con 0x7f57380adbe0 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d0da700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573e0dc700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 0x7f573813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573d8db700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f573400eba0 tx=0x7f5734006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400c8e0 con 0x7f57380adbe0 2020-11-16T22:14:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57380b2c80 con 0x7f57380adbe0 2020-11-16T22:14:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400ca40 con 0x7f57380adbe0 2020-11-16T22:14:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400cdc0 con 0x7f57380adbe0 2020-11-16T22:14:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.983+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573813ee80 con 0x7f57380adbe0 2020-11-16T22:14:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57381467e0 con 0x7f57380adbe0 2020-11-16T22:14:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5734012600 con 0x7f57380adbe0 2020-11-16T22:14:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f5729ffb700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 0x7f5710057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f573406de10 con 0x7f57380adbe0 2020-11-16T22:14:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f573d0da700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 0x7f5710057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.987+0000 7f573d0da700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 0x7f5710057550 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f57381423d0 tx=0x7f572c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:04.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:04.995+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5734044420 con 0x7f57380adbe0 2020-11-16T22:14:05.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f573805d4e0 con 0x7f57380adbe0 2020-11-16T22:14:05.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f5729ffb700 1 -- 172.21.15.185:0/2207862028 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5734047a40 con 0x7f57380adbe0 2020-11-16T22:14:05.134 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:05.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 msgr2=0x7f5710057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 0x7f5710057550 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f57381423d0 tx=0x7f572c015000).stop 2020-11-16T22:14:05.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 msgr2=0x7f5738153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f573400eba0 tx=0x7f5734006df0).stop 2020-11-16T22:14:05.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 shutdown_connections 2020-11-16T22:14:05.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5710055090 0x7f5710057550 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57380b74b0 0x7f573813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57380adbe0 0x7f5738153520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 --2- 172.21.15.185:0/2207862028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57380afc40 0x7f57381462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 >> 172.21.15.185:0/2207862028 conn(0x7f5738067860 msgr2=0x7f5738142d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:05.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 shutdown_connections 2020-11-16T22:14:05.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.147+0000 7f573fb3f700 1 -- 172.21.15.185:0/2207862028 wait complete. 2020-11-16T22:14:05.153 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:14:05.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 Processor -- start 2020-11-16T22:14:05.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- start start 2020-11-16T22:14:05.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f090c0b4d40 0x7f090c068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c1418d0 0x7f090c13de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c0b7e10 con 0x7f090c0675b0 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0b7670 con 0x7f090c0b4d40 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b78d0 con 0x7f090c1418d0 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0910c00700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f090c0675b0 0x7f090c0b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49172/0 (socket says 172.21.15.185:49172) 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0910c00700 1 -- 172.21.15.185:0/1443782224 learned_addr learned my addr 172.21.15.185:0/1443782224 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 --2- 172.21.15.185:0/1443782224 >> v2:172.21.15.185:3300/0 conn(0x7f090c0b4d40 0x7f090c068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911401700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c1418d0 0x7f090c13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090b7fe700 1 -- 172.21.15.185:0/1443782224 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387950830 0 0) 0x7f090c0b7e10 con 0x7f090c0675b0 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090b7fe700 1 -- 172.21.15.185:0/1443782224 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec003240 con 0x7f090c0675b0 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 -- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c1418d0 msgr2=0x7f090c13de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c1418d0 0x7f090c13de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 -- 172.21.15.185:0/1443782224 >> v1:172.21.15.185:6789/0 conn(0x7f090c0675b0 legacy=0x7f090c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 -- 172.21.15.185:0/1443782224 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0b7b70 con 0x7f090c0b4d40 2020-11-16T22:14:05.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090bfff700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c0b4d40 0x7f090c068460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f08f400b9a0 tx=0x7f08f4006df0).ready entity=mon.1 client_cookie=9f914431f5e5bbac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090b7fe700 1 -- 172.21.15.185:0/1443782224 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4009640 con 0x7f090c0b4d40 2020-11-16T22:14:05.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090b7fe700 1 -- 172.21.15.185:0/1443782224 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f40097a0 con 0x7f090c0b4d40 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f090b7fe700 1 -- 172.21.15.185:0/1443782224 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4009b20 con 0x7f090c0b4d40 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c0b4d40 msgr2=0x7f090c068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c0b4d40 0x7f090c068460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f08f400b9a0 tx=0x7f08f4006df0).stop 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- 172.21.15.185:0/1443782224 shutdown_connections 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c1418d0 0x7f090c13de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 --2- 172.21.15.185:0/1443782224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c0b4d40 0x7f090c068460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- 172.21.15.185:0/1443782224 >> 172.21.15.185:0/1443782224 conn(0x7f090c05ba80 msgr2=0x7f090c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- 172.21.15.185:0/1443782224 shutdown_connections 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.283+0000 7f0911c02700 1 -- 172.21.15.185:0/1443782224 wait complete. 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 Processor -- start 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- start start 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 0x7f090c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 0x7f090c1270e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b0ee0 con 0x7f090c0675b0 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b1180 con 0x7f090c1418d0 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c061a70 con 0x7f090c0b4d40 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911401700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 0x7f090c1270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911401700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 0x7f090c1270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55742/0 (socket says 172.21.15.185:55742) 2020-11-16T22:14:05.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49188/0 (socket says 172.21.15.185:49188) 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911401700 1 -- 172.21.15.185:0/24373308 learned_addr learned my addr 172.21.15.185:0/24373308 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0910c00700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 0x7f090c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 -- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 msgr2=0x7f090c1270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 0x7f090c1270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 -- 172.21.15.185:0/24373308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 msgr2=0x7f090c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 0x7f090c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 -- 172.21.15.185:0/24373308 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0b1420 con 0x7f090c0b4d40 2020-11-16T22:14:05.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0910c00700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 0x7f090c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:05.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f090bfff700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f08f4012000 tx=0x7f08f4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4011cc0 con 0x7f090c0b4d40 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4011e20 con 0x7f090c0b4d40 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c1275d0 con 0x7f090c0b4d40 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f401d680 con 0x7f090c0b4d40 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.287+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c131320 con 0x7f090c0b4d40 2020-11-16T22:14:05.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c05d200 con 0x7f090c0b4d40 2020-11-16T22:14:05.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f08f4024030 con 0x7f090c0b4d40 2020-11-16T22:14:05.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0908ff9700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 0x7f08e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0910c00700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 0x7f08e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f08f406eea0 con 0x7f090c0b4d40 2020-11-16T22:14:05.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.291+0000 7f0910c00700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 0x7f08e8057520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f08fc000f40 tx=0x7f08fc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.299+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f08f4045430 con 0x7f090c0b4d40 2020-11-16T22:14:05.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.451+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f090c131f50 con 0x7f090c0b4d40 2020-11-16T22:14:05.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0908ff9700 1 -- 172.21.15.185:0/24373308 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f08f4022070 con 0x7f090c0b4d40 2020-11-16T22:14:05.442 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:05.442 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:14:05.442 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:05.442 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:14:05.442 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:05.443 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:14:05.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 msgr2=0x7f08e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 0x7f08e8057520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f08fc000f40 tx=0x7f08fc009040).stop 2020-11-16T22:14:05.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 msgr2=0x7f090c061580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f08f4012000 tx=0x7f08f4015000).stop 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 shutdown_connections 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f08e8055060 0x7f08e8057520 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f090c1418d0 0x7f090c1270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f090c0675b0 0x7f090c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 --2- 172.21.15.185:0/24373308 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f090c0b4d40 0x7f090c061580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 >> 172.21.15.185:0/24373308 conn(0x7f090c05ba80 msgr2=0x7f090c0b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:05.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 shutdown_connections 2020-11-16T22:14:05.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.455+0000 7f0911c02700 1 -- 172.21.15.185:0/24373308 wait complete. 2020-11-16T22:14:05.455 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:14:05.456 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:14:05.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 Processor -- start 2020-11-16T22:14:05.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 -- start start 2020-11-16T22:14:05.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f02a40b4470 0x7f02a40b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a40b7290 con 0x7f02a41475d0 2020-11-16T22:14:05.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a40b4bf0 con 0x7f02a40b4470 2020-11-16T22:14:05.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a91c8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b74c0 con 0x7f02a40b5d70 2020-11-16T22:14:05.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49192/0 (socket says 172.21.15.185:49192) 2020-11-16T22:14:05.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 -- 172.21.15.185:0/2503244185 learned_addr learned my addr 172.21.15.185:0/2503244185 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:05.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a37fe700 1 --2- 172.21.15.185:0/2503244185 >> v2:172.21.15.96:3300/0 conn(0x7f02a40b4470 0x7f02a40b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 -- 172.21.15.185:0/2503244185 >> v2:172.21.15.96:3300/0 conn(0x7f02a40b4470 msgr2=0x7f02a40b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:05.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 --2- 172.21.15.185:0/2503244185 >> v2:172.21.15.96:3300/0 conn(0x7f02a40b4470 0x7f02a40b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 -- 172.21.15.185:0/2503244185 >> v1:172.21.15.96:6789/0 conn(0x7f02a41475d0 legacy=0x7f02a4147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 -- 172.21.15.185:0/2503244185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40b7760 con 0x7f02a40b5d70 2020-11-16T22:14:05.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.599+0000 7f02a3fff700 1 --2- 172.21.15.185:0/2503244185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0298006fd0 tx=0x7f0298006eb0).ready entity=mon.2 client_cookie=f4e6fbcf7da3c5d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a2ffd700 1 -- 172.21.15.185:0/2503244185 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298009810 con 0x7f02a40b5d70 2020-11-16T22:14:05.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a2ffd700 1 -- 172.21.15.185:0/2503244185 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298009970 con 0x7f02a40b5d70 2020-11-16T22:14:05.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a2ffd700 1 -- 172.21.15.185:0/2503244185 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298009cf0 con 0x7f02a40b5d70 2020-11-16T22:14:05.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- 172.21.15.185:0/2503244185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 msgr2=0x7f02a40ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- 172.21.15.185:0/2503244185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0298006fd0 tx=0x7f0298006eb0).stop 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- 172.21.15.185:0/2503244185 shutdown_connections 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- 172.21.15.185:0/2503244185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b5d70 0x7f02a40ba800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- 172.21.15.185:0/2503244185 >> v2:172.21.15.96:3300/0 conn(0x7f02a40b4470 0x7f02a40b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- 172.21.15.185:0/2503244185 >> 172.21.15.185:0/2503244185 conn(0x7f02a405ba80 msgr2=0x7f02a405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- 172.21.15.185:0/2503244185 shutdown_connections 2020-11-16T22:14:05.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- 172.21.15.185:0/2503244185 wait complete. 2020-11-16T22:14:05.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 Processor -- start 2020-11-16T22:14:05.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- start start 2020-11-16T22:14:05.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f02a40b4470 0x7f02a413f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02a41475d0 0x7f02a40b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b0370 con 0x7f02a41475d0 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b0610 con 0x7f02a40b4470 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a91c8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b08b0 con 0x7f02a40b1240 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a89c7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a89c7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49202/0 (socket says 172.21.15.185:49202) 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a89c7700 1 -- 172.21.15.185:0/775872642 learned_addr learned my addr 172.21.15.185:0/775872642 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f02a40b4470 0x7f02a413f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.603+0000 7f02a37fe700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02a41475d0 0x7f02a40b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 -- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f02a40b4470 msgr2=0x7f02a413f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f02a40b4470 0x7f02a413f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 -- 172.21.15.185:0/775872642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02a41475d0 msgr2=0x7f02a40b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02a41475d0 0x7f02a40b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 -- 172.21.15.185:0/775872642 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a413f5e0 con 0x7f02a40b1240 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a89c7700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f029c00c710 tx=0x7f029c007af0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e490 con 0x7f02a40b1240 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00e610 con 0x7f02a40b1240 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a40aec60 con 0x7f02a40b1240 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e9d0 con 0x7f02a40b1240 2020-11-16T22:14:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a413e7a0 con 0x7f02a40b1240 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f029c010070 con 0x7f02a40b1240 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a40b16c0 con 0x7f02a40b1240 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 0x7f027c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f029c06cec0 con 0x7f02a40b1240 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a3fff700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 0x7f027c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.607+0000 7f02a3fff700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 0x7f027c057520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f02a405c5f0 tx=0x7f0298015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:05.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.619+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f029c043340 con 0x7f02a40b1240 2020-11-16T22:14:05.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:05.771+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f02a405d200 con 0x7f02a40b1240 2020-11-16T22:14:06.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.175+0000 7f0283fff700 1 -- 172.21.15.185:0/775872642 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7f029c046960 con 0x7f02a40b1240 2020-11-16T22:14:06.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 msgr2=0x7f027c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 0x7f027c057520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f02a405c5f0 tx=0x7f0298015000).stop 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 msgr2=0x7f02a413e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f029c00c710 tx=0x7f029c007af0).stop 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 shutdown_connections 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f027c055060 0x7f027c057520 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f02a40b4470 0x7f02a413f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f02a40b1240 0x7f02a413e090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 --2- 172.21.15.185:0/775872642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02a41475d0 0x7f02a40b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 >> 172.21.15.185:0/775872642 conn(0x7f02a405ba80 msgr2=0x7f02a4147ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:06.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 shutdown_connections 2020-11-16T22:14:06.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.179+0000 7f02a91c8700 1 -- 172.21.15.185:0/775872642 wait complete. 2020-11-16T22:14:06.168 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:14:06.181 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:14:06.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.323+0000 7f7250ecf700 1 Processor -- start 2020-11-16T22:14:06.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.323+0000 7f7250ecf700 1 -- start start 2020-11-16T22:14:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.323+0000 7f7250ecf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f724c0b4c30 0x7f724c145c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.323+0000 7f7250ecf700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c0af570 con 0x7f724c063f90 2020-11-16T22:14:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c0b3d00 con 0x7f724c0af780 2020-11-16T22:14:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c0af2d0 con 0x7f724c0b4c30 2020-11-16T22:14:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724affd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f724c0b4c30 0x7f724c145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724bfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f724c0af780 0x7f724c14b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49204/0 (socket says 172.21.15.185:49204) 2020-11-16T22:14:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724bfff700 1 -- 172.21.15.185:0/2995657534 learned_addr learned my addr 172.21.15.185:0/2995657534 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724affd700 1 -- 172.21.15.185:0/2995657534 >> v1:172.21.15.185:6789/0 conn(0x7f724c0af780 legacy=0x7f724c14b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724affd700 1 -- 172.21.15.185:0/2995657534 >> v1:172.21.15.96:6789/0 conn(0x7f724c063f90 legacy=0x7f724c145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724a7fc700 1 -- 172.21.15.185:0/2995657534 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721914701 0 0) 0x7f724c0b3d00 con 0x7f724c0af780 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724affd700 1 -- 172.21.15.185:0/2995657534 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c0b3190 con 0x7f724c0b4c30 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724affd700 1 --2- 172.21.15.185:0/2995657534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0b4c30 0x7f724c145c40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f724000b9a0 tx=0x7f7240006df0).ready entity=mon.1 client_cookie=de3508065e7319cc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724a7fc700 1 -- 172.21.15.185:0/2995657534 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400097b0 con 0x7f724c0b4c30 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724a7fc700 1 -- 172.21.15.185:0/2995657534 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240009910 con 0x7f724c0b4c30 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f724a7fc700 1 -- 172.21.15.185:0/2995657534 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240009c90 con 0x7f724c0b4c30 2020-11-16T22:14:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- 172.21.15.185:0/2995657534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0b4c30 msgr2=0x7f724c145c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 --2- 172.21.15.185:0/2995657534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0b4c30 0x7f724c145c40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f724000b9a0 tx=0x7f7240006df0).stop 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- 172.21.15.185:0/2995657534 shutdown_connections 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 --2- 172.21.15.185:0/2995657534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0b4c30 0x7f724c145c40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- 172.21.15.185:0/2995657534 >> 172.21.15.185:0/2995657534 conn(0x7f724c067940 msgr2=0x7f724c067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- 172.21.15.185:0/2995657534 shutdown_connections 2020-11-16T22:14:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.327+0000 7f7250ecf700 1 -- 172.21.15.185:0/2995657534 wait complete. 2020-11-16T22:14:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 Processor -- start 2020-11-16T22:14:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 -- start start 2020-11-16T22:14:06.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f724c063f90 0x7f724c144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 0x7f724c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c0ace40 con 0x7f724c063f90 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c0ad0e0 con 0x7f724c0af780 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f7250ecf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c0ad380 con 0x7f724c0b4c30 2020-11-16T22:14:06.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:06.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55768/0 (socket says 172.21.15.185:55768) 2020-11-16T22:14:06.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 -- 172.21.15.185:0/1926792643 learned_addr learned my addr 172.21.15.185:0/1926792643 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:06.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724bfff700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 0x7f724c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:06.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724b7fe700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f724c063f90 0x7f724c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:06.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 -- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 msgr2=0x7f724c150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 0x7f724c150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 -- 172.21.15.185:0/1926792643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f724c063f90 msgr2=0x7f724c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:06.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f724c063f90 0x7f724c144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 -- 172.21.15.185:0/1926792643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c144e60 con 0x7f724c0af780 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724bfff700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 0x7f724c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.331+0000 7f724affd700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7240010000 tx=0x7f7240015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240011dd0 con 0x7f724c0af780 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724001e460 con 0x7f724c0af780 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c0b94e0 con 0x7f724c0af780 2020-11-16T22:14:06.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724001e7e0 con 0x7f724c0af780 2020-11-16T22:14:06.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c0ac880 con 0x7f724c0af780 2020-11-16T22:14:06.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7240025030 con 0x7f724c0af780 2020-11-16T22:14:06.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c0ad540 con 0x7f724c0af780 2020-11-16T22:14:06.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 0x7f7224057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:06.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f724006f470 con 0x7f724c0af780 2020-11-16T22:14:06.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f724b7fe700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 0x7f7224057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:06.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.335+0000 7f724b7fe700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 0x7f7224057520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f723c003300 tx=0x7f723c007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:06.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.343+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7240045950 con 0x7f724c0af780 2020-11-16T22:14:06.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:06.495+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f724c05d200 con 0x7f724c0af780 2020-11-16T22:14:07.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.179+0000 7f722b7fe700 1 -- 172.21.15.185:0/1926792643 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v20) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7240023070 con 0x7f724c0af780 2020-11-16T22:14:07.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 msgr2=0x7f7224057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 0x7f7224057520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f723c003300 tx=0x7f723c007b90).stop 2020-11-16T22:14:07.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 msgr2=0x7f724c0ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7240010000 tx=0x7f7240015000).stop 2020-11-16T22:14:07.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 shutdown_connections 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7224055060 0x7f7224057520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f724c0b4c30 0x7f724c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f724c063f90 0x7f724c144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 --2- 172.21.15.185:0/1926792643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f724c0af780 0x7f724c0ac1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 >> 172.21.15.185:0/1926792643 conn(0x7f724c067940 msgr2=0x7f724c141d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 shutdown_connections 2020-11-16T22:14:07.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.187+0000 7f7250ecf700 1 -- 172.21.15.185:0/1926792643 wait complete. 2020-11-16T22:14:07.176 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:14:07.189 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:14:07.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 Processor -- start 2020-11-16T22:14:07.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- start start 2020-11-16T22:14:07.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 0x7f75a0061450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a00b4d60 con 0x7f75a00b8830 2020-11-16T22:14:07.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a00630b0 con 0x7f75a0062c10 2020-11-16T22:14:07.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00b41a0 con 0x7f75a00b48e0 2020-11-16T22:14:07.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49224/0 (socket says 172.21.15.185:49224) 2020-11-16T22:14:07.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 -- 172.21.15.185:0/3697104046 learned_addr learned my addr 172.21.15.185:0/3697104046 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:07.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f759f7fe700 1 --2- 172.21.15.185:0/3697104046 >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 0x7f75a0061450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 -- 172.21.15.185:0/3697104046 >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 msgr2=0x7f75a0061450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 --2- 172.21.15.185:0/3697104046 >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 0x7f75a0061450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 -- 172.21.15.185:0/3697104046 >> v1:172.21.15.96:6789/0 conn(0x7f75a00b8830 legacy=0x7f75a01441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 -- 172.21.15.185:0/3697104046 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00b4440 con 0x7f75a00b48e0 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f759f7fe700 1 --2- 172.21.15.185:0/3697104046 >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 0x7f75a0061450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:07.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a4ce1700 1 --2- 172.21.15.185:0/3697104046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7598009d50 tx=0x7f7598009290).ready entity=mon.2 client_cookie=a1bc2de98be67516 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f759effd700 1 -- 172.21.15.185:0/3697104046 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759800bc80 con 0x7f75a00b48e0 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f759effd700 1 -- 172.21.15.185:0/3697104046 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759800bde0 con 0x7f75a00b48e0 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f759effd700 1 -- 172.21.15.185:0/3697104046 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598011610 con 0x7f75a00b48e0 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- 172.21.15.185:0/3697104046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 msgr2=0x7f75a0143140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3697104046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7598009d50 tx=0x7f7598009290).stop 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 -- 172.21.15.185:0/3697104046 shutdown_connections 2020-11-16T22:14:07.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.323+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3697104046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0143140 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3697104046 >> v2:172.21.15.96:3300/0 conn(0x7f75a0062c10 0x7f75a0061450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- 172.21.15.185:0/3697104046 >> 172.21.15.185:0/3697104046 conn(0x7f75a0060130 msgr2=0x7f75a0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- 172.21.15.185:0/3697104046 shutdown_connections 2020-11-16T22:14:07.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- 172.21.15.185:0/3697104046 wait complete. 2020-11-16T22:14:07.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 Processor -- start 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- start start 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a00b8830 0x7f75a0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00ae3f0 con 0x7f75a00b8830 2020-11-16T22:14:07.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0140dc0 con 0x7f75a0062c10 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a54e2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00aed20 con 0x7f75a00b48e0 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55782/0 (socket says 172.21.15.185:55782) 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759ffff700 1 -- 172.21.15.185:0/3815446634 learned_addr learned my addr 172.21.15.185:0/3815446634 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f75a4ce1700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a00b8830 0x7f75a0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759ffff700 1 -- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 msgr2=0x7f75a0153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.327+0000 7f759ffff700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759ffff700 1 -- 172.21.15.185:0/3815446634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a00b8830 msgr2=0x7f75a0154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759ffff700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a00b8830 0x7f75a0154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759ffff700 1 -- 172.21.15.185:0/3815446634 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a0140880 con 0x7f75a0062c10 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759ffff700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f759400c6e0 tx=0x7f7594007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594007020 con 0x7f75a0062c10 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594007180 con 0x7f75a0062c10 2020-11-16T22:14:07.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a0140b20 con 0x7f75a0062c10 2020-11-16T22:14:07.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400e560 con 0x7f75a0062c10 2020-11-16T22:14:07.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a013ea00 con 0x7f75a0062c10 2020-11-16T22:14:07.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a00675b0 con 0x7f75a0062c10 2020-11-16T22:14:07.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7594010070 con 0x7f75a0062c10 2020-11-16T22:14:07.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 0x7f7578057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f759406d060 con 0x7f75a0062c10 2020-11-16T22:14:07.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759f7fe700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 0x7f7578057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.331+0000 7f759f7fe700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 0x7f7578057520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f75a0061720 tx=0x7f7590008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.335+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f75a00675b0 con 0x7f75a0062c10 2020-11-16T22:14:07.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.535+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f75a00675b0 con 0x7f75a0062c10 2020-11-16T22:14:07.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.551+0000 7f757ffff700 1 -- 172.21.15.185:0/3815446634 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f75a00675b0 con 0x7f75a0062c10 2020-11-16T22:14:07.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.551+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 msgr2=0x7f7578057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.551+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 0x7f7578057520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f75a0061720 tx=0x7f7590008040).stop 2020-11-16T22:14:07.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.551+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 msgr2=0x7f75a013e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.551+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f759400c6e0 tx=0x7f7594007c50).stop 2020-11-16T22:14:07.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 shutdown_connections 2020-11-16T22:14:07.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7578055060 0x7f7578057520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75a0062c10 0x7f75a013e510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a00b8830 0x7f75a0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 --2- 172.21.15.185:0/3815446634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75a00b48e0 0x7f75a0153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 >> 172.21.15.185:0/3815446634 conn(0x7f75a0060130 msgr2=0x7f75a00b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 shutdown_connections 2020-11-16T22:14:07.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.555+0000 7f75a54e2700 1 -- 172.21.15.185:0/3815446634 wait complete. 2020-11-16T22:14:07.548 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 3 and data pool 4 2020-11-16T22:14:07.562 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:07.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 Processor -- start 2020-11-16T22:14:07.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 -- start start 2020-11-16T22:14:07.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 0x7f8540066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8540061e60 0x7f85401442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85400abf90 con 0x7f85400acc10 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400ac230 con 0x7f8540061e60 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f85449dd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400ac4d0 con 0x7f85400669a0 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 0x7f8540066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f85400acc10 0x7f8540147550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49228/0 (socket says 172.21.15.185:49228) 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853f59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8540061e60 0x7f85401442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853e59c700 1 -- 172.21.15.185:0/1173812169 learned_addr learned my addr 172.21.15.185:0/1173812169 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:07.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853dd9b700 1 -- 172.21.15.185:0/1173812169 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897056686 0 0) 0x7f85400abf90 con 0x7f85400acc10 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853dd9b700 1 -- 172.21.15.185:0/1173812169 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8524003240 con 0x7f85400acc10 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 -- 172.21.15.185:0/1173812169 >> v2:172.21.15.185:3300/0 conn(0x7f8540061e60 msgr2=0x7f85401442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 --2- 172.21.15.185:0/1173812169 >> v2:172.21.15.185:3300/0 conn(0x7f8540061e60 0x7f85401442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 -- 172.21.15.185:0/1173812169 >> v1:172.21.15.185:6789/0 conn(0x7f85400acc10 legacy=0x7f8540147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 -- 172.21.15.185:0/1173812169 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400ac770 con 0x7f85400669a0 2020-11-16T22:14:07.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853ed9d700 1 --2- 172.21.15.185:0/1173812169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 0x7f8540066d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f853800b9a0 tx=0x7f8538006df0).ready entity=mon.2 client_cookie=d312ad0a91f5cd88 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853dd9b700 1 -- 172.21.15.185:0/1173812169 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538009680 con 0x7f85400669a0 2020-11-16T22:14:07.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853dd9b700 1 -- 172.21.15.185:0/1173812169 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85380097e0 con 0x7f85400669a0 2020-11-16T22:14:07.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.683+0000 7f853dd9b700 1 -- 172.21.15.185:0/1173812169 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538009b60 con 0x7f85400669a0 2020-11-16T22:14:07.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- 172.21.15.185:0/1173812169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 msgr2=0x7f8540066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 --2- 172.21.15.185:0/1173812169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 0x7f8540066d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f853800b9a0 tx=0x7f8538006df0).stop 2020-11-16T22:14:07.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- 172.21.15.185:0/1173812169 shutdown_connections 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 --2- 172.21.15.185:0/1173812169 >> v2:172.21.15.185:3300/0 conn(0x7f8540061e60 0x7f85401442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 --2- 172.21.15.185:0/1173812169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85400669a0 0x7f8540066d80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- 172.21.15.185:0/1173812169 >> 172.21.15.185:0/1173812169 conn(0x7f8540060130 msgr2=0x7f8540060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- 172.21.15.185:0/1173812169 shutdown_connections 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- 172.21.15.185:0/1173812169 wait complete. 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 Processor -- start 2020-11-16T22:14:07.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.687+0000 7f85449dd700 1 -- start start 2020-11-16T22:14:07.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85400669a0 0x7f85401400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 0x7f85401405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85401479d0 con 0x7f85400669a0 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540145e90 con 0x7f8540145520 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8540146130 con 0x7f8540061e60 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49240/0 (socket says 172.21.15.185:49240) 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 -- 172.21.15.185:0/3430911298 learned_addr learned my addr 172.21.15.185:0/3430911298 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:07.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853f59e700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 0x7f85401405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853e59c700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85400669a0 0x7f85401400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 -- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 msgr2=0x7f85401405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 0x7f85401405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 -- 172.21.15.185:0/3430911298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85400669a0 msgr2=0x7f85401400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85400669a0 0x7f85401400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 -- 172.21.15.185:0/3430911298 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400b1660 con 0x7f8540061e60 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853f59e700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 0x7f85401405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f853ed9d700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f853800c010 tx=0x7f8538015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538011c40 con 0x7f8540061e60 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8538011da0 con 0x7f8540061e60 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853801d680 con 0x7f8540061e60 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8540140ad0 con 0x7f8540061e60 2020-11-16T22:14:07.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8540146890 con 0x7f8540061e60 2020-11-16T22:14:07.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.691+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f853800f070 con 0x7f8540061e60 2020-11-16T22:14:07.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.695+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8540146a80 con 0x7f8540061e60 2020-11-16T22:14:07.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.695+0000 7f8536ffd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 0x7f8510057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.695+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f853806fa90 con 0x7f8540061e60 2020-11-16T22:14:07.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.695+0000 7f853e59c700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 0x7f8510057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.695+0000 7f853e59c700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 0x7f8510057520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8528005920 tx=0x7f8528009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.703+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8538045cf0 con 0x7f8540061e60 2020-11-16T22:14:07.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.847+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85401477f0 con 0x7f8540061e60 2020-11-16T22:14:07.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.847+0000 7f8536ffd700 1 -- 172.21.15.185:0/3430911298 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f8538006980 con 0x7f8540061e60 2020-11-16T22:14:07.837 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:07.837 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":21,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:07.542554+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:07.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 msgr2=0x7f8510057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 0x7f8510057520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8528005920 tx=0x7f8528009040).stop 2020-11-16T22:14:07.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 msgr2=0x7f85400b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f853800c010 tx=0x7f8538015000).stop 2020-11-16T22:14:07.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 shutdown_connections 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8510055060 0x7f8510057520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8540145520 0x7f85401405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8540061e60 0x7f85400b1170 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 --2- 172.21.15.185:0/3430911298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85400669a0 0x7f85401400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 >> 172.21.15.185:0/3430911298 conn(0x7f8540060130 msgr2=0x7f8540067490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 shutdown_connections 2020-11-16T22:14:07.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.851+0000 7f85449dd700 1 -- 172.21.15.185:0/3430911298 wait complete. 2020-11-16T22:14:07.851 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:07.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.983+0000 7f09efc3c700 1 Processor -- start 2020-11-16T22:14:07.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.983+0000 7f09efc3c700 1 -- start start 2020-11-16T22:14:07.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 0x7f09e80669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e8065fe0 con 0x7f09e8067070 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e8062e70 con 0x7f09e81417d0 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8066280 con 0x7f09e80b2950 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49248/0 (socket says 172.21.15.185:49248) 2020-11-16T22:14:07.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 -- 172.21.15.185:0/1827001162 learned_addr learned my addr 172.21.15.185:0/1827001162 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:07.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed1d7700 1 --2- 172.21.15.185:0/1827001162 >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 0x7f09e80669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 -- 172.21.15.185:0/1827001162 >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 msgr2=0x7f09e80669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:07.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 --2- 172.21.15.185:0/1827001162 >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 0x7f09e80669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 -- 172.21.15.185:0/1827001162 >> v1:172.21.15.96:6789/0 conn(0x7f09e8067070 legacy=0x7f09e8062760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 -- 172.21.15.185:0/1827001162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8066520 con 0x7f09e80b2950 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed1d7700 1 --2- 172.21.15.185:0/1827001162 >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 0x7f09e80669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ed9d8700 1 --2- 172.21.15.185:0/1827001162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f09e4006fd0 tx=0x7f09e4006eb0).ready entity=mon.2 client_cookie=6cadbe4c529bdf56 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ec9d6700 1 -- 172.21.15.185:0/1827001162 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40098d0 con 0x7f09e80b2950 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ec9d6700 1 -- 172.21.15.185:0/1827001162 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e4009a30 con 0x7f09e80b2950 2020-11-16T22:14:07.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09ec9d6700 1 -- 172.21.15.185:0/1827001162 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4009db0 con 0x7f09e80b2950 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- 172.21.15.185:0/1827001162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 msgr2=0x7f09e80b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 --2- 172.21.15.185:0/1827001162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f09e4006fd0 tx=0x7f09e4006eb0).stop 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- 172.21.15.185:0/1827001162 shutdown_connections 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 --2- 172.21.15.185:0/1827001162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e80b2950 0x7f09e80b2d30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 --2- 172.21.15.185:0/1827001162 >> v2:172.21.15.96:3300/0 conn(0x7f09e81417d0 0x7f09e80669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.987+0000 7f09efc3c700 1 -- 172.21.15.185:0/1827001162 >> 172.21.15.185:0/1827001162 conn(0x7f09e80b4410 msgr2=0x7f09e8060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:07.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.991+0000 7f09efc3c700 1 -- 172.21.15.185:0/1827001162 shutdown_connections 2020-11-16T22:14:07.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.991+0000 7f09efc3c700 1 -- 172.21.15.185:0/1827001162 wait complete. 2020-11-16T22:14:07.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.991+0000 7f09efc3c700 1 Processor -- start 2020-11-16T22:14:07.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.991+0000 7f09efc3c700 1 -- start start 2020-11-16T22:14:07.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f09e80b2950 0x7f09e813a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09e81417d0 0x7f09e81420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8130490 con 0x7f09e81417d0 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8131490 con 0x7f09e80b2950 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8131730 con 0x7f09e8067070 2020-11-16T22:14:07.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49254/0 (socket says 172.21.15.185:49254) 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed1d7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f09e80b2950 0x7f09e813a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 -- 172.21.15.185:0/804174313 learned_addr learned my addr 172.21.15.185:0/804174313 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ee1d9700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09e81417d0 0x7f09e81420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 -- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f09e80b2950 msgr2=0x7f09e813a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f09e80b2950 0x7f09e813a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 -- 172.21.15.185:0/804174313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09e81417d0 msgr2=0x7f09e81420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:07.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09e81417d0 0x7f09e81420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:07.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 -- 172.21.15.185:0/804174313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e81319d0 con 0x7f09e8067070 2020-11-16T22:14:07.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09ed9d8700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f09e4009a00 tx=0x7f09e4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4011c80 con 0x7f09e8067070 2020-11-16T22:14:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e4011de0 con 0x7f09e8067070 2020-11-16T22:14:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e401e670 con 0x7f09e8067070 2020-11-16T22:14:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e8131c70 con 0x7f09e8067070 2020-11-16T22:14:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.995+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e8143dd0 con 0x7f09e8067070 2020-11-16T22:14:07.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e805d4e0 con 0x7f09e8067070 2020-11-16T22:14:07.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f09e4025030 con 0x7f09e8067070 2020-11-16T22:14:07.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09d5ffb700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 0x7f09c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:07.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09e406f2c0 con 0x7f09e8067070 2020-11-16T22:14:07.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09ed1d7700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 0x7f09c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:07.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:07.999+0000 7f09ed1d7700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 0x7f09c0057520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f09e8143b90 tx=0x7f09d8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:07.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.007+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f09e40455d0 con 0x7f09e8067070 2020-11-16T22:14:08.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.159+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09e809ce10 con 0x7f09e8067070 2020-11-16T22:14:08.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09d5ffb700 1 -- 172.21.15.185:0/804174313 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f09e40311e0 con 0x7f09e8067070 2020-11-16T22:14:08.150 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:08.150 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":21,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:07.542554+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:08.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 msgr2=0x7f09c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 0x7f09c0057520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f09e8143b90 tx=0x7f09d8008040).stop 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 msgr2=0x7f09e812ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f09e4009a00 tx=0x7f09e4015000).stop 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 shutdown_connections 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f09c0055060 0x7f09c0057520 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09e81417d0 0x7f09e81420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f09e8067070 0x7f09e812ffa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 --2- 172.21.15.185:0/804174313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f09e80b2950 0x7f09e813a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 >> 172.21.15.185:0/804174313 conn(0x7f09e80b4410 msgr2=0x7f09e80b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:08.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 shutdown_connections 2020-11-16T22:14:08.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.163+0000 7f09efc3c700 1 -- 172.21.15.185:0/804174313 wait complete. 2020-11-16T22:14:08.165 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:14:08.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 Processor -- start 2020-11-16T22:14:08.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 -- start start 2020-11-16T22:14:08.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce900b7440 0x7fce900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900ad490 con 0x7fce90144860 2020-11-16T22:14:08.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900ad730 con 0x7fce900afc60 2020-11-16T22:14:08.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce97ce5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce900ad9d0 con 0x7fce900b7440 2020-11-16T22:14:08.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce95280700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fce900afc60 0x7fce9013dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49258/0 (socket says 172.21.15.185:49258) 2020-11-16T22:14:08.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce95280700 1 -- 172.21.15.185:0/370960580 learned_addr learned my addr 172.21.15.185:0/370960580 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:08.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce95a81700 1 --2- 172.21.15.185:0/370960580 >> v2:172.21.15.96:3300/0 conn(0x7fce900b7440 0x7fce900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583375785 0 0) 0x7fce900ad730 con 0x7fce900afc60 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce78003240 con 0x7fce900afc60 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241261289 0 0) 0x7fce900ad490 con 0x7fce90144860 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.303+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce900ad730 con 0x7fce90144860 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 562707982 0 0) 0x7fce78003240 con 0x7fce900afc60 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 >> v2:172.21.15.96:3300/0 conn(0x7fce900b7440 msgr2=0x7fce900b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 --2- 172.21.15.185:0/370960580 >> v2:172.21.15.96:3300/0 conn(0x7fce900b7440 0x7fce900b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 >> v1:172.21.15.96:6789/0 conn(0x7fce90144860 legacy=0x7fce900ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9013e5b0 con 0x7fce900afc60 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fce7c0014e0 con 0x7fce900afc60 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 >> v1:172.21.15.185:6789/0 conn(0x7fce900afc60 legacy=0x7fce9013dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce780038c0 0x7fce78003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce780044e0 0x7fce78004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 0x7fce7800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900ad9d0 con 0x7fce780044e0 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780053b0 con 0x7fce780038c0 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7800bca0 con 0x7fce78004e50 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce95a81700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce780038c0 0x7fce78003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce95280700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce780044e0 0x7fce78004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 0x7fce7800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 -- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce780038c0 msgr2=0x7fce78003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:08.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce780038c0 0x7fce78003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 -- 172.21.15.185:0/370960580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce780044e0 msgr2=0x7fce78004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce780044e0 0x7fce78004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 -- 172.21.15.185:0/370960580 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78003240 con 0x7fce78004e50 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce95a81700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce780038c0 0x7fce78003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 -- 172.21.15.185:0/370960580 reap_dead start 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce96282700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 0x7fce7800b730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fce88001fd0 tx=0x7fce88007e80).ready entity=mon.2 client_cookie=dfd0b6c37fde5514 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800cb10 con 0x7fce78004e50 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8800cc70 con 0x7fce78004e50 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce94a7f700 1 -- 172.21.15.185:0/370960580 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88006870 con 0x7fce78004e50 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 -- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 msgr2=0x7fce7800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 0x7fce7800b730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fce88001fd0 tx=0x7fce88007e80).stop 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 -- 172.21.15.185:0/370960580 shutdown_connections 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 --2- 172.21.15.185:0/370960580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce78004e50 0x7fce7800b730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 -- 172.21.15.185:0/370960580 >> 172.21.15.185:0/370960580 conn(0x7fce90067860 msgr2=0x7fce90067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:08.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 -- 172.21.15.185:0/370960580 shutdown_connections 2020-11-16T22:14:08.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.307+0000 7fce97ce5700 1 -- 172.21.15.185:0/370960580 wait complete. 2020-11-16T22:14:08.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 Processor -- start 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- start start 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce90156760 0x7fce90140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9013f910 con 0x7fce90156760 2020-11-16T22:14:08.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90147380 con 0x7fce78004e50 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce901482c0 con 0x7fce900b3aa0 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95280700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55830/0 (socket says 172.21.15.185:55830) 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95280700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49276/0 (socket says 172.21.15.185:49276) 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 -- 172.21.15.185:0/3437070894 learned_addr learned my addr 172.21.15.185:0/3437070894 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:08.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce96282700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce90156760 0x7fce90140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 -- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 msgr2=0x7fce90140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 -- 172.21.15.185:0/3437070894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce90156760 msgr2=0x7fce90140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce90156760 0x7fce90140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 -- 172.21.15.185:0/3437070894 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce901475b0 con 0x7fce78004e50 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95280700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce95a81700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fce80011c90 tx=0x7fce80006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8000fa30 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8000fb90 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce90147850 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80014440 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.311+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce900b3690 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce900b4930 con 0x7fce78004e50 2020-11-16T22:14:08.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce80024070 con 0x7fce78004e50 2020-11-16T22:14:08.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce8dffb700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 0x7fce64057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce8006e7d0 con 0x7fce78004e50 2020-11-16T22:14:08.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce95280700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 0x7fce64057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.315+0000 7fce95280700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 0x7fce64057520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fce90140110 tx=0x7fce7c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.323+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fce80044ab0 con 0x7fce78004e50 2020-11-16T22:14:08.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.519+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fce9005d4e0 con 0x7fce78004e50 2020-11-16T22:14:08.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce8dffb700 1 -- 172.21.15.185:0/3437070894 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fce800480d0 con 0x7fce78004e50 2020-11-16T22:14:08.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 msgr2=0x7fce64057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 0x7fce64057520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fce90140110 tx=0x7fce7c009040).stop 2020-11-16T22:14:08.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 msgr2=0x7fce900b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fce80011c90 tx=0x7fce80006df0).stop 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 shutdown_connections 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce64055060 0x7fce64057520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce78004e50 0x7fce900b3120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce90156760 0x7fce90140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 --2- 172.21.15.185:0/3437070894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce900b3aa0 0x7fce90140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 >> 172.21.15.185:0/3437070894 conn(0x7fce90067860 msgr2=0x7fce90067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:08.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 shutdown_connections 2020-11-16T22:14:08.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.559+0000 7fce97ce5700 1 -- 172.21.15.185:0/3437070894 wait complete. 2020-11-16T22:14:08.562 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:14:08.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 Processor -- start 2020-11-16T22:14:08.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 -- start start 2020-11-16T22:14:08.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c08141690 0x7f5c08067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c080b3f50 con 0x7f5c080b7690 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c080b41f0 con 0x7f5c08141690 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c07fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b4490 con 0x7f5c080b1f20 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c077fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c077fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49286/0 (socket says 172.21.15.185:49286) 2020-11-16T22:14:08.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c077fe700 1 -- 172.21.15.185:0/1331221287 learned_addr learned my addr 172.21.15.185:0/1331221287 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:08.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.703+0000 7f5c067fc700 1 --2- 172.21.15.185:0/1331221287 >> v2:172.21.15.96:3300/0 conn(0x7f5c08141690 0x7f5c08067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c077fe700 1 -- 172.21.15.185:0/1331221287 >> v2:172.21.15.96:3300/0 conn(0x7f5c08141690 msgr2=0x7f5c08067420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:08.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c077fe700 1 --2- 172.21.15.185:0/1331221287 >> v2:172.21.15.96:3300/0 conn(0x7f5c08141690 0x7f5c08067420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c077fe700 1 -- 172.21.15.185:0/1331221287 >> v1:172.21.15.96:6789/0 conn(0x7f5c080b7690 legacy=0x7f5c08144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c077fe700 1 -- 172.21.15.185:0/1331221287 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080b4e20 con 0x7f5c080b1f20 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c077fe700 1 --2- 172.21.15.185:0/1331221287 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5c00009d50 tx=0x7f5c00009290).ready entity=mon.2 client_cookie=4a5e216f2a52c9f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c05ffb700 1 -- 172.21.15.185:0/1331221287 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bbc0 con 0x7f5c080b1f20 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c05ffb700 1 -- 172.21.15.185:0/1331221287 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000bd20 con 0x7f5c080b1f20 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c05ffb700 1 -- 172.21.15.185:0/1331221287 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011610 con 0x7f5c080b1f20 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- 172.21.15.185:0/1331221287 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 msgr2=0x7f5c080b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 --2- 172.21.15.185:0/1331221287 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5c00009d50 tx=0x7f5c00009290).stop 2020-11-16T22:14:08.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- 172.21.15.185:0/1331221287 shutdown_connections 2020-11-16T22:14:08.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 --2- 172.21.15.185:0/1331221287 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c080b1f20 0x7f5c080b4930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 --2- 172.21.15.185:0/1331221287 >> v2:172.21.15.96:3300/0 conn(0x7f5c08141690 0x7f5c08067420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- 172.21.15.185:0/1331221287 >> 172.21.15.185:0/1331221287 conn(0x7f5c08064b90 msgr2=0x7f5c08063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:08.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- 172.21.15.185:0/1331221287 shutdown_connections 2020-11-16T22:14:08.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- 172.21.15.185:0/1331221287 wait complete. 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 Processor -- start 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.707+0000 7f5c07fff700 1 -- start start 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c080b7690 0x7f5c080b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c08141690 0x7f5c080b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b5d00 con 0x7f5c080b7690 2020-11-16T22:14:08.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b6b60 con 0x7f5c080b1f20 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b5f10 con 0x7f5c08141690 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c077fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c08141690 0x7f5c080b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55844/0 (socket says 172.21.15.185:55844) 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 -- 172.21.15.185:0/4076025517 learned_addr learned my addr 172.21.15.185:0/4076025517 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c067fc700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c080b7690 0x7f5c080b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 -- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c08141690 msgr2=0x7f5c080b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c08141690 0x7f5c080b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 -- 172.21.15.185:0/4076025517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c080b7690 msgr2=0x7f5c080b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c080b7690 0x7f5c080b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 -- 172.21.15.185:0/4076025517 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080b61b0 con 0x7f5c080b1f20 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c06ffd700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5bfc00c6e0 tx=0x7f5bfc007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc007020 con 0x7f5c080b1f20 2020-11-16T22:14:08.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc007180 con 0x7f5c080b1f20 2020-11-16T22:14:08.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0813bbd0 con 0x7f5c080b1f20 2020-11-16T22:14:08.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0064f0 con 0x7f5c080b1f20 2020-11-16T22:14:08.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0813eac0 con 0x7f5c080b1f20 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0805d200 con 0x7f5c080b1f20 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.711+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5bfc010070 con 0x7f5c080b1f20 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.715+0000 7f5beeffd700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 0x7f5be4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.715+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bfc06d0a0 con 0x7f5c080b1f20 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.715+0000 7f5c067fc700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 0x7f5be4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:08.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.715+0000 7f5c067fc700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 0x7f5be4057520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f5c0813f200 tx=0x7f5bf8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:08.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.719+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5bfc043300 con 0x7f5c080b1f20 2020-11-16T22:14:08.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:08.919+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f5c0813f920 con 0x7f5c080b1f20 2020-11-16T22:14:09.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.567+0000 7f5beeffd700 1 -- 172.21.15.185:0/4076025517 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5bfc046920 con 0x7f5c080b1f20 2020-11-16T22:14:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 msgr2=0x7f5be4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 0x7f5be4057520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f5c0813f200 tx=0x7f5bf8006df0).stop 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 msgr2=0x7f5c0813b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5bfc00c6e0 tx=0x7f5bfc007c50).stop 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 shutdown_connections 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5be4055060 0x7f5be4057520 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5c080b1f20 0x7f5c0813b6e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5c08141690 0x7f5c080b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 --2- 172.21.15.185:0/4076025517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c080b7690 0x7f5c080b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 >> 172.21.15.185:0/4076025517 conn(0x7f5c08064b90 msgr2=0x7f5c08140c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:09.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 shutdown_connections 2020-11-16T22:14:09.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.571+0000 7f5c07fff700 1 -- 172.21.15.185:0/4076025517 wait complete. 2020-11-16T22:14:09.579 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2020-11-16T22:14:09.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.711+0000 7fc4d092d700 1 Processor -- start 2020-11-16T22:14:09.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.711+0000 7fc4d092d700 1 -- start start 2020-11-16T22:14:09.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 0x7fc4cc0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc4cc0af7f0 0x7fc4cc145350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc0b47b0 con 0x7fc4cc0af7f0 2020-11-16T22:14:09.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc0b4a50 con 0x7fc4cc063fd0 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0b4cf0 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cb7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 0x7fc4cc0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49300/0 (socket says 172.21.15.185:49300) 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 -- 172.21.15.185:0/209320581 learned_addr learned my addr 172.21.15.185:0/209320581 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4caffd700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.96:3300/0 conn(0x7fc4cc0af7f0 0x7fc4cc145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 -- 172.21.15.185:0/209320581 >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 msgr2=0x7fc4cc0b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:09.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 0x7fc4cc0b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 -- 172.21.15.185:0/209320581 >> v2:172.21.15.96:3300/0 conn(0x7fc4cc0af7f0 msgr2=0x7fc4cc145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:09.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.96:3300/0 conn(0x7fc4cc0af7f0 0x7fc4cc145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 -- 172.21.15.185:0/209320581 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0b4430 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cb7fe700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 0x7fc4cc0b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:09.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4cbfff700 1 --2- 172.21.15.185:0/209320581 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc4cc05c620 tx=0x7fc4bc006eb0).ready entity=mon.2 client_cookie=17ba492b2f75d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:09.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4ca7fc700 1 -- 172.21.15.185:0/209320581 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc009720 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4ca7fc700 1 -- 172.21.15.185:0/209320581 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc009880 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4ca7fc700 1 -- 172.21.15.185:0/209320581 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc009c00 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- 172.21.15.185:0/209320581 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 msgr2=0x7fc4cc145840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- 172.21.15.185:0/209320581 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc4cc05c620 tx=0x7fc4bc006eb0).stop 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- 172.21.15.185:0/209320581 shutdown_connections 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- 172.21.15.185:0/209320581 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc0b3ec0 0x7fc4cc145840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.185:3300/0 conn(0x7fc4cc063fd0 0x7fc4cc0b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 --2- 172.21.15.185:0/209320581 >> v2:172.21.15.96:3300/0 conn(0x7fc4cc0af7f0 0x7fc4cc145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- 172.21.15.185:0/209320581 >> 172.21.15.185:0/209320581 conn(0x7fc4cc067940 msgr2=0x7fc4cc067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:09.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- 172.21.15.185:0/209320581 shutdown_connections 2020-11-16T22:14:09.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- 172.21.15.185:0/209320581 wait complete. 2020-11-16T22:14:09.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 Processor -- start 2020-11-16T22:14:09.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.715+0000 7fc4d092d700 1 -- start start 2020-11-16T22:14:09.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4cc0af7f0 0x7fc4cc13f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4cc0b3ec0 0x7fc4cc13a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc146290 con 0x7fc4cc0af7f0 2020-11-16T22:14:09.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc139d10 con 0x7fc4cc0b3ec0 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc151d10 con 0x7fc4cc063fd0 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49302/0 (socket says 172.21.15.185:49302) 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 -- 172.21.15.185:0/580958634 learned_addr learned my addr 172.21.15.185:0/580958634 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cbfff700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4cc0b3ec0 0x7fc4cc13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4caffd700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4cc0af7f0 0x7fc4cc13f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 -- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4cc0b3ec0 msgr2=0x7fc4cc13a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4cc0b3ec0 0x7fc4cc13a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 -- 172.21.15.185:0/580958634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4cc0af7f0 msgr2=0x7fc4cc13f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4cc0af7f0 0x7fc4cc13f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 -- 172.21.15.185:0/580958634 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc145ff0 con 0x7fc4cc063fd0 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4cb7fe700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc4c4009fa0 tx=0x7fc4c40091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400cd40 con 0x7fc4cc063fd0 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400cea0 con 0x7fc4cc063fd0 2020-11-16T22:14:09.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc151a20 con 0x7fc4cc063fd0 2020-11-16T22:14:09.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4012600 con 0x7fc4cc063fd0 2020-11-16T22:14:09.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.719+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc0acd60 con 0x7fc4cc063fd0 2020-11-16T22:14:09.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc4c4022070 con 0x7fc4cc063fd0 2020-11-16T22:14:09.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc151ed0 con 0x7fc4cc063fd0 2020-11-16T22:14:09.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4c37fe700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 0x7fc4a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:09.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4c406e710 con 0x7fc4cc063fd0 2020-11-16T22:14:09.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4caffd700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 0x7fc4a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:09.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.723+0000 7fc4caffd700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 0x7fc4a8057520 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc4cc0b1630 tx=0x7fc4b4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:09.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.727+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc4cc151ed0 con 0x7fc4cc063fd0 2020-11-16T22:14:09.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:09.927+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc4cc0ac110 con 0x7fc4cc063fd0 2020-11-16T22:14:10.136 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:14:10.150+0000 7fdcc80a3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:14:10.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.579+0000 7fc4c37fe700 1 -- 172.21.15.185:0/580958634 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc4c4044970 con 0x7fc4cc063fd0 2020-11-16T22:14:10.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 msgr2=0x7fc4a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 0x7fc4a8057520 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc4cc0b1630 tx=0x7fc4b4008040).stop 2020-11-16T22:14:10.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 msgr2=0x7fc4cc13ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc4c4009fa0 tx=0x7fc4c40091d0).stop 2020-11-16T22:14:10.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 shutdown_connections 2020-11-16T22:14:10.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4a8055060 0x7fc4a8057520 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4cc0b3ec0 0x7fc4cc13a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc4cc063fd0 0x7fc4cc13ed10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 --2- 172.21.15.185:0/580958634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4cc0af7f0 0x7fc4cc13f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 >> 172.21.15.185:0/580958634 conn(0x7fc4cc067940 msgr2=0x7fc4cc0b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:10.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 shutdown_connections 2020-11-16T22:14:10.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.583+0000 7fc4d092d700 1 -- 172.21.15.185:0/580958634 wait complete. 2020-11-16T22:14:10.591 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:10.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 Processor -- start 2020-11-16T22:14:10.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 -- start start 2020-11-16T22:14:10.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6bac1446c0 0x7f6bac062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:10.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac067930 con 0x7f6bac067de0 2020-11-16T22:14:10.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0632e0 con 0x7f6bac062900 2020-11-16T22:14:10.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb3a02700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac066ae0 con 0x7f6bac1446c0 2020-11-16T22:14:10.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb179e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6bac062900 0x7f6bac067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49314/0 (socket says 172.21.15.185:49314) 2020-11-16T22:14:10.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb179e700 1 -- 172.21.15.185:0/3742535236 learned_addr learned my addr 172.21.15.185:0/3742535236 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:10.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb0f9d700 1 --2- 172.21.15.185:0/3742535236 >> v2:172.21.15.185:3300/0 conn(0x7f6bac1446c0 0x7f6bac062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:10.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb0f9d700 1 -- 172.21.15.185:0/3742535236 >> v1:172.21.15.185:6789/0 conn(0x7f6bac062900 legacy=0x7f6bac067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb0f9d700 1 -- 172.21.15.185:0/3742535236 >> v1:172.21.15.96:6789/0 conn(0x7f6bac067de0 legacy=0x7f6bac1408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6b9ffff700 1 -- 172.21.15.185:0/3742535236 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160885776 0 0) 0x7f6bac0632e0 con 0x7f6bac062900 2020-11-16T22:14:10.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb0f9d700 1 -- 172.21.15.185:0/3742535236 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac066d80 con 0x7f6bac1446c0 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6bb0f9d700 1 --2- 172.21.15.185:0/3742535236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac062ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6ba000cb30 tx=0x7f6ba0006e80).ready entity=mon.1 client_cookie=e8abeb4d3b1e2c0e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6b9ffff700 1 -- 172.21.15.185:0/3742535236 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000a7e0 con 0x7f6bac1446c0 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6b9ffff700 1 -- 172.21.15.185:0/3742535236 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba000a940 con 0x7f6bac1446c0 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.723+0000 7f6b9ffff700 1 -- 172.21.15.185:0/3742535236 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000acc0 con 0x7f6bac1446c0 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- 172.21.15.185:0/3742535236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 msgr2=0x7f6bac062ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/3742535236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac062ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6ba000cb30 tx=0x7f6ba0006e80).stop 2020-11-16T22:14:10.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- 172.21.15.185:0/3742535236 shutdown_connections 2020-11-16T22:14:10.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/3742535236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac062ce0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- 172.21.15.185:0/3742535236 >> 172.21.15.185:0/3742535236 conn(0x7f6bac0b4410 msgr2=0x7f6bac060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:10.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- 172.21.15.185:0/3742535236 shutdown_connections 2020-11-16T22:14:10.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- 172.21.15.185:0/3742535236 wait complete. 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 Processor -- start 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- start start 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 0x7f6bac157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6bac067de0 0x7f6bac141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac156c20 con 0x7f6bac062900 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac156ec0 con 0x7f6bac1446c0 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb3a02700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac157160 con 0x7f6bac067de0 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.727+0000 7f6bb1f9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55886/0 (socket says 172.21.15.185:55886) 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb0f9d700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6bac067de0 0x7f6bac141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 -- 172.21.15.185:0/1577025164 learned_addr learned my addr 172.21.15.185:0/1577025164 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:10.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb179e700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 0x7f6bac157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 -- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6bac067de0 msgr2=0x7f6bac141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6bac067de0 0x7f6bac141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 -- 172.21.15.185:0/1577025164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 msgr2=0x7f6bac157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 0x7f6bac157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 -- 172.21.15.185:0/1577025164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac13eb90 con 0x7f6bac1446c0 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb179e700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 0x7f6bac157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:10.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb1f9f700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6ba400ca90 tx=0x7f6ba4007ea0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:10.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400a730 con 0x7f6bac1446c0 2020-11-16T22:14:10.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba400a890 con 0x7f6bac1446c0 2020-11-16T22:14:10.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac146d60 con 0x7f6bac1446c0 2020-11-16T22:14:10.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400ac10 con 0x7f6bac1446c0 2020-11-16T22:14:10.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac13f680 con 0x7f6bac1446c0 2020-11-16T22:14:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac157320 con 0x7f6bac1446c0 2020-11-16T22:14:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6ba4019030 con 0x7f6bac1446c0 2020-11-16T22:14:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 0x7f6b84057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6ba406d510 con 0x7f6bac1446c0 2020-11-16T22:14:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.731+0000 7f6bb179e700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 0x7f6b84057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.735+0000 7f6bb179e700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 0x7f6b84057520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6ba80011a0 tx=0x7f6ba8009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:10.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.739+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6ba4043820 con 0x7f6bac1446c0 2020-11-16T22:14:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bac05d4e0 con 0x7f6bac1446c0 2020-11-16T22:14:10.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6b9d7fa700 1 -- 172.21.15.185:0/1577025164 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+2511 (secure 0 0 0) 0x7f6ba4046e40 con 0x7f6bac1446c0 2020-11-16T22:14:10.930 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:10.930 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:10.585745+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:creating","state_seq":2,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:10.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 msgr2=0x7f6b84057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 0x7f6b84057520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6ba80011a0 tx=0x7f6ba8009040).stop 2020-11-16T22:14:10.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 msgr2=0x7f6bac13e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6ba400ca90 tx=0x7f6ba4007ea0).stop 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 shutdown_connections 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b84055060 0x7f6b84057520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6bac1446c0 0x7f6bac13e6a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bac062900 0x7f6bac157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 --2- 172.21.15.185:0/1577025164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6bac067de0 0x7f6bac141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:10.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 >> 172.21.15.185:0/1577025164 conn(0x7f6bac0b4410 msgr2=0x7f6bac0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:10.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 shutdown_connections 2020-11-16T22:14:10.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:10.943+0000 7f6bb3a02700 1 -- 172.21.15.185:0/1577025164 wait complete. 2020-11-16T22:14:10.934 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 9 2020-11-16T22:14:10.945 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:11.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 Processor -- start 2020-11-16T22:14:11.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 -- start start 2020-11-16T22:14:11.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fddc409cb40 0x7fddc4146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fddc4064350 0x7fddc41445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40b4220 con 0x7fddc40b1a00 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc4146560 con 0x7fddc409cb40 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc8b26700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40b44c0 con 0x7fddc4064350 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fddc4064350 0x7fddc41445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fddc4064350 0x7fddc41445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55892/0 (socket says 172.21.15.185:55892) 2020-11-16T22:14:11.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 -- 172.21.15.185:0/639379747 learned_addr learned my addr 172.21.15.185:0/639379747 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc259c700 1 --2- 172.21.15.185:0/639379747 >> v2:172.21.15.96:3300/0 conn(0x7fddc409cb40 0x7fddc4146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 -- 172.21.15.185:0/639379747 >> v2:172.21.15.96:3300/0 conn(0x7fddc409cb40 msgr2=0x7fddc4146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:11.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 --2- 172.21.15.185:0/639379747 >> v2:172.21.15.96:3300/0 conn(0x7fddc409cb40 0x7fddc4146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 -- 172.21.15.185:0/639379747 >> v1:172.21.15.96:6789/0 conn(0x7fddc40b1a00 legacy=0x7fddc4063c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 -- 172.21.15.185:0/639379747 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40b4760 con 0x7fddc4064350 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc2d9d700 1 --2- 172.21.15.185:0/639379747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc41445b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fddc40b40d0 tx=0x7fddbc009290).ready entity=mon.1 client_cookie=1863813cf7e5ed4d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc159a700 1 -- 172.21.15.185:0/639379747 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00bbc0 con 0x7fddc4064350 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc159a700 1 -- 172.21.15.185:0/639379747 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc00bd20 con 0x7fddc4064350 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.079+0000 7fddc159a700 1 -- 172.21.15.185:0/639379747 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc011610 con 0x7fddc4064350 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- 172.21.15.185:0/639379747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 msgr2=0x7fddc41445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- 172.21.15.185:0/639379747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc41445b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fddc40b40d0 tx=0x7fddbc009290).stop 2020-11-16T22:14:11.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- 172.21.15.185:0/639379747 shutdown_connections 2020-11-16T22:14:11.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- 172.21.15.185:0/639379747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc41445b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- 172.21.15.185:0/639379747 >> v2:172.21.15.96:3300/0 conn(0x7fddc409cb40 0x7fddc4146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- 172.21.15.185:0/639379747 >> 172.21.15.185:0/639379747 conn(0x7fddc4061240 msgr2=0x7fddc4061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- 172.21.15.185:0/639379747 shutdown_connections 2020-11-16T22:14:11.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- 172.21.15.185:0/639379747 wait complete. 2020-11-16T22:14:11.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 Processor -- start 2020-11-16T22:14:11.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- start start 2020-11-16T22:14:11.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddc409cb40 0x7fddc4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc40b1a00 0x7fddc4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4157680 con 0x7fddc40b1a00 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4099390 con 0x7fddc4064350 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc8b26700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4099630 con 0x7fddc409cb40 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55896/0 (socket says 172.21.15.185:55896) 2020-11-16T22:14:11.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 -- 172.21.15.185:0/3157011130 learned_addr learned my addr 172.21.15.185:0/3157011130 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc1d9b700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddc409cb40 0x7fddc4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc2d9d700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc40b1a00 0x7fddc4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 -- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddc409cb40 msgr2=0x7fddc4156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddc409cb40 0x7fddc4156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 -- 172.21.15.185:0/3157011130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc40b1a00 msgr2=0x7fddc4157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc40b1a00 0x7fddc4157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.083+0000 7fddc259c700 1 -- 172.21.15.185:0/3157011130 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40998d0 con 0x7fddc4064350 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc259c700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fddb400ebd0 tx=0x7fddb4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400c8e0 con 0x7fddc4064350 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400ca40 con 0x7fddc4064350 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4156800 con 0x7fddc4064350 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400cdc0 con 0x7fddc4064350 2020-11-16T22:14:11.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc408eab0 con 0x7fddc4064350 2020-11-16T22:14:11.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc405d4e0 con 0x7fddc4064350 2020-11-16T22:14:11.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fddb4012600 con 0x7fddc4064350 2020-11-16T22:14:11.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 0x7fdda0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fddb406e060 con 0x7fddc4064350 2020-11-16T22:14:11.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc1d9b700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 0x7fdda0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.087+0000 7fddc1d9b700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 0x7fdda0057520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fddc40b3610 tx=0x7fddac007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.095+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fddb4044260 con 0x7fddc4064350 2020-11-16T22:14:11.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.247+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddc40626c0 con 0x7fddc4064350 2020-11-16T22:14:11.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.247+0000 7fddba7fc700 1 -- 172.21.15.185:0/3157011130 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fddb4047880 con 0x7fddc4064350 2020-11-16T22:14:11.237 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:11.237 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:11.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 msgr2=0x7fdda0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 0x7fdda0057520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fddc40b3610 tx=0x7fddac007b90).stop 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 msgr2=0x7fddc4148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fddb400ebd0 tx=0x7fddb4006df0).stop 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 shutdown_connections 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdda0055060 0x7fdda0057520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddc4064350 0x7fddc4148d70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc40b1a00 0x7fddc4157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 --2- 172.21.15.185:0/3157011130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddc409cb40 0x7fddc4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 >> 172.21.15.185:0/3157011130 conn(0x7fddc4061240 msgr2=0x7fddc40b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 shutdown_connections 2020-11-16T22:14:11.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.251+0000 7fddc8b26700 1 -- 172.21.15.185:0/3157011130 wait complete. 2020-11-16T22:14:11.252 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:11.253 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-16T22:14:11.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 Processor -- start 2020-11-16T22:14:11.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 -- start start 2020-11-16T22:14:11.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 0x7f2f7c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 0x7f2f7c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c061fe0 con 0x7f2f7c0ad110 2020-11-16T22:14:11.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b2b60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f82abb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b0fe0 con 0x7f2f7c0b3090 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f822ba700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2f7c0ad110 0x7f2f7c1477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49344/0 (socket says 172.21.15.185:49344) 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f822ba700 1 -- 172.21.15.185:0/2175917182 learned_addr learned my addr 172.21.15.185:0/2175917182 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2175917182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 0x7f2f7c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f812b8700 1 --2- 172.21.15.185:0/2175917182 >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 0x7f2f7c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f80ab7700 1 -- 172.21.15.185:0/2175917182 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061427786 0 0) 0x7f2f7c061fe0 con 0x7f2f7c0ad110 2020-11-16T22:14:11.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f80ab7700 1 -- 172.21.15.185:0/2175917182 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f5c003240 con 0x7f2f7c0ad110 2020-11-16T22:14:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 -- 172.21.15.185:0/2175917182 >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 msgr2=0x7f2f7c14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2175917182 >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 0x7f2f7c14b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 -- 172.21.15.185:0/2175917182 >> v1:172.21.15.185:6789/0 conn(0x7f2f7c0ad110 legacy=0x7f2f7c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 -- 172.21.15.185:0/2175917182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0b1280 con 0x7f2f7c0b3090 2020-11-16T22:14:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f812b8700 1 --2- 172.21.15.185:0/2175917182 >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 0x7f2f7c14b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:11.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2175917182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 0x7f2f7c0b1720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2f74009c90 tx=0x7f2f740091d0).ready entity=mon.2 client_cookie=3f60e9c7e9a5b01f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f80ab7700 1 -- 172.21.15.185:0/2175917182 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400b9c0 con 0x7f2f7c0b3090 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f80ab7700 1 -- 172.21.15.185:0/2175917182 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7400bb20 con 0x7f2f7c0b3090 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.391+0000 7f2f80ab7700 1 -- 172.21.15.185:0/2175917182 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400bea0 con 0x7f2f7c0b3090 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 -- 172.21.15.185:0/2175917182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 msgr2=0x7f2f7c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2175917182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 0x7f2f7c0b1720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2f74009c90 tx=0x7f2f740091d0).stop 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 -- 172.21.15.185:0/2175917182 shutdown_connections 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2175917182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b3090 0x7f2f7c0b1720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2175917182 >> v2:172.21.15.96:3300/0 conn(0x7f2f7c0b1d30 0x7f2f7c14b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 -- 172.21.15.185:0/2175917182 >> 172.21.15.185:0/2175917182 conn(0x7f2f7c060130 msgr2=0x7f2f7c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 -- 172.21.15.185:0/2175917182 shutdown_connections 2020-11-16T22:14:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 -- 172.21.15.185:0/2175917182 wait complete. 2020-11-16T22:14:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.395+0000 7f2f82abb700 1 Processor -- start 2020-11-16T22:14:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- start start 2020-11-16T22:14:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 0x7f2f7c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1135e0 con 0x7f2f7c0b3090 2020-11-16T22:14:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c141e40 con 0x7f2f7c0ad110 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1420e0 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f81ab9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49356/0 (socket says 172.21.15.185:49356) 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f81ab9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55910/0 (socket says 172.21.15.185:55910) 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 -- 172.21.15.185:0/2496324246 learned_addr learned my addr 172.21.15.185:0/2496324246 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f822ba700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 0x7f2f7c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 -- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 msgr2=0x7f2f7c1126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 -- 172.21.15.185:0/2496324246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 msgr2=0x7f2f7c145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 0x7f2f7c145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 -- 172.21.15.185:0/2496324246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c142380 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f822ba700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 0x7f2f7c145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:11.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f812b8700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2f6400ef50 tx=0x7f2f64006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400b580 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6400cb30 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c145e60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f640123f0 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.399+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c147c90 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c0b5e60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2f64012760 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f79ffb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 0x7f2f58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 0x7f2f58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2f6406e380 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.403+0000 7f2f81ab9700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 0x7f2f58057520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2f74009440 tx=0x7f2f74016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.407+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2f7c0b5e60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.563+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f7c0b5e60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.563+0000 7f2f79ffb700 1 -- 172.21.15.185:0/2496324246 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f2f7c0b5e60 con 0x7f2f7c0b1d30 2020-11-16T22:14:11.552 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:11.552 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:11.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 msgr2=0x7f2f58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 0x7f2f58057520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2f74009440 tx=0x7f2f74016000).stop 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 msgr2=0x7f2f7c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f2f6400ef50 tx=0x7f2f64006df0).stop 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 shutdown_connections 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2f58055060 0x7f2f58057520 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2f7c0ad110 0x7f2f7c1126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f7c0b3090 0x7f2f7c145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 --2- 172.21.15.185:0/2496324246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2f7c0b1d30 0x7f2f7c145480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 >> 172.21.15.185:0/2496324246 conn(0x7f2f7c060130 msgr2=0x7f2f7c0b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 shutdown_connections 2020-11-16T22:14:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.567+0000 7f2f82abb700 1 -- 172.21.15.185:0/2496324246 wait complete. 2020-11-16T22:14:11.567 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:11.568 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:11.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.695+0000 7fb81a1aa700 1 Processor -- start 2020-11-16T22:14:11.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.695+0000 7fb81a1aa700 1 -- start start 2020-11-16T22:14:11.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8140b3890 0x7fb8140b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8140ad620 con 0x7fb8141410d0 2020-11-16T22:14:11.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb814141650 con 0x7fb8140b3890 2020-11-16T22:14:11.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141418f0 con 0x7fb8140adc40 2020-11-16T22:14:11.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49368/0 (socket says 172.21.15.185:49368) 2020-11-16T22:14:11.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb8137fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8140b3890 0x7fb8140b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 -- 172.21.15.185:0/3120415070 learned_addr learned my addr 172.21.15.185:0/3120415070 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 -- 172.21.15.185:0/3120415070 >> v2:172.21.15.96:3300/0 conn(0x7fb8140b3890 msgr2=0x7fb8140b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:11.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 --2- 172.21.15.185:0/3120415070 >> v2:172.21.15.96:3300/0 conn(0x7fb8140b3890 0x7fb8140b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 -- 172.21.15.185:0/3120415070 >> v1:172.21.15.96:6789/0 conn(0x7fb8141410d0 legacy=0x7fb8140aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 -- 172.21.15.185:0/3120415070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814141b90 con 0x7fb8140adc40 2020-11-16T22:14:11.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb813fff700 1 --2- 172.21.15.185:0/3120415070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb80c009d50 tx=0x7fb80c009290).ready entity=mon.2 client_cookie=33e2eb809e1b8d2a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb8127fc700 1 -- 172.21.15.185:0/3120415070 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00bbc0 con 0x7fb8140adc40 2020-11-16T22:14:11.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb8127fc700 1 -- 172.21.15.185:0/3120415070 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00bd20 con 0x7fb8140adc40 2020-11-16T22:14:11.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb8127fc700 1 -- 172.21.15.185:0/3120415070 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c011610 con 0x7fb8140adc40 2020-11-16T22:14:11.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- 172.21.15.185:0/3120415070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 msgr2=0x7fb8140ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/3120415070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb80c009d50 tx=0x7fb80c009290).stop 2020-11-16T22:14:11.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- 172.21.15.185:0/3120415070 shutdown_connections 2020-11-16T22:14:11.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/3120415070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140adc40 0x7fb8140ad130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/3120415070 >> v2:172.21.15.96:3300/0 conn(0x7fb8140b3890 0x7fb8140b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- 172.21.15.185:0/3120415070 >> 172.21.15.185:0/3120415070 conn(0x7fb814063f60 msgr2=0x7fb814064340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- 172.21.15.185:0/3120415070 shutdown_connections 2020-11-16T22:14:11.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.699+0000 7fb81a1aa700 1 -- 172.21.15.185:0/3120415070 wait complete. 2020-11-16T22:14:11.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 Processor -- start 2020-11-16T22:14:11.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 -- start start 2020-11-16T22:14:11.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140b3890 0x7fb814066f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8141410d0 0x7fb814151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814155860 con 0x7fb8141410d0 2020-11-16T22:14:11.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814067760 con 0x7fb8140adc40 2020-11-16T22:14:11.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb81a1aa700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81413bf10 con 0x7fb8140b3890 2020-11-16T22:14:11.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb8137fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55926/0 (socket says 172.21.15.185:55926) 2020-11-16T22:14:11.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 -- 172.21.15.185:0/1704101169 learned_addr learned my addr 172.21.15.185:0/1704101169 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.703+0000 7fb812ffd700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140b3890 0x7fb814066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb813fff700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8141410d0 0x7fb814151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 -- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140b3890 msgr2=0x7fb814066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140b3890 0x7fb814066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 -- 172.21.15.185:0/1704101169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8141410d0 msgr2=0x7fb814151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8141410d0 0x7fb814151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 -- 172.21.15.185:0/1704101169 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81413c1b0 con 0x7fb8140adc40 2020-11-16T22:14:11.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb8137fe700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb80400ebd0 tx=0x7fb804006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400c8e0 con 0x7fb8140adc40 2020-11-16T22:14:11.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400ca40 con 0x7fb8140adc40 2020-11-16T22:14:11.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81413c450 con 0x7fb8140adc40 2020-11-16T22:14:11.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400cdc0 con 0x7fb8140adc40 2020-11-16T22:14:11.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81413cff0 con 0x7fb8140adc40 2020-11-16T22:14:11.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81405d4e0 con 0x7fb8140adc40 2020-11-16T22:14:11.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb804012600 con 0x7fb8140adc40 2020-11-16T22:14:11.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 0x7fb7f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb812ffd700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 0x7fb7f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.707+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb80406e060 con 0x7fb8140adc40 2020-11-16T22:14:11.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.711+0000 7fb812ffd700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 0x7fb7f0057520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb8140b3030 tx=0x7fb7fc007b00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:11.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.715+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb804044260 con 0x7fb8140adc40 2020-11-16T22:14:11.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.863+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb8140633d0 con 0x7fb8140adc40 2020-11-16T22:14:11.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb80b7fe700 1 -- 172.21.15.185:0/1704101169 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb804047880 con 0x7fb8140adc40 2020-11-16T22:14:11.853 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 msgr2=0x7fb7f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 0x7fb7f0057520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb8140b3030 tx=0x7fb7fc007b00).stop 2020-11-16T22:14:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 msgr2=0x7fb814155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb80400ebd0 tx=0x7fb804006df0).stop 2020-11-16T22:14:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 shutdown_connections 2020-11-16T22:14:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb7f0055060 0x7fb7f0057520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8140adc40 0x7fb814155370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8141410d0 0x7fb814151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 --2- 172.21.15.185:0/1704101169 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8140b3890 0x7fb814066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 >> 172.21.15.185:0/1704101169 conn(0x7fb814063f60 msgr2=0x7fb8140b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 shutdown_connections 2020-11-16T22:14:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:11.867+0000 7fb81a1aa700 1 -- 172.21.15.185:0/1704101169 wait complete. 2020-11-16T22:14:11.868 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:11.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 Processor -- start 2020-11-16T22:14:11.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 -- start start 2020-11-16T22:14:11.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc063fd0 0x7fecfc0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 0x7fecfc145350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fecfc0b3ec0 0x7fecfc145840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:11.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecfc0b4870 con 0x7fecfc0af7f0 2020-11-16T22:14:11.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecfc0b4a50 con 0x7fecfc0b3ec0 2020-11-16T22:14:11.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.007+0000 7fed00c87700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0b4cf0 con 0x7fecfc063fd0 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fecfc0b3ec0 0x7fecfc145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc063fd0 0x7fecfc0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fecfc0b3ec0 0x7fecfc145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55936/0 (socket says 172.21.15.185:55936) 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 -- 172.21.15.185:0/1630926593 learned_addr learned my addr 172.21.15.185:0/1630926593 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfaffd700 1 --2- 172.21.15.185:0/1630926593 >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 0x7fecfc145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:11.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 -- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc063fd0 msgr2=0x7fecfc0b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc063fd0 0x7fecfc0b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 -- 172.21.15.185:0/1630926593 >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 msgr2=0x7fecfc145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- 172.21.15.185:0/1630926593 >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 0x7fecfc145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 -- 172.21.15.185:0/1630926593 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0b4430 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfaffd700 1 --2- 172.21.15.185:0/1630926593 >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 0x7fecfc145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0b3ec0 0x7fecfc145840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fecfc0b0f50 tx=0x7fecf4006eb0).ready entity=mon.1 client_cookie=ab1de8c244c34aca server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfa7fc700 1 -- 172.21.15.185:0/1630926593 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4009810 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfa7fc700 1 -- 172.21.15.185:0/1630926593 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf4009970 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfa7fc700 1 -- 172.21.15.185:0/1630926593 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4009cf0 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0b3ec0 msgr2=0x7fecfc145840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0b3ec0 0x7fecfc145840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fecfc0b0f50 tx=0x7fecf4006eb0).stop 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- 172.21.15.185:0/1630926593 shutdown_connections 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0b3ec0 0x7fecfc145840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- 172.21.15.185:0/1630926593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc063fd0 0x7fecfc0b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- 172.21.15.185:0/1630926593 >> v2:172.21.15.96:3300/0 conn(0x7fecfc0af7f0 0x7fecfc145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- 172.21.15.185:0/1630926593 >> 172.21.15.185:0/1630926593 conn(0x7fecfc067940 msgr2=0x7fecfc067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- 172.21.15.185:0/1630926593 shutdown_connections 2020-11-16T22:14:12.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- 172.21.15.185:0/1630926593 wait complete. 2020-11-16T22:14:12.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 Processor -- start 2020-11-16T22:14:12.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- start start 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecfc063fd0 0x7fecfc0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 0x7fecfc0ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 0x7fecfc12e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0b9180 con 0x7fecfc063fd0 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0ad9f0 con 0x7fecfc0af7f0 2020-11-16T22:14:12.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fed00c87700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc12f9d0 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfaffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 0x7fecfc0ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 0x7fecfc12e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfaffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 0x7fecfc0ac790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55940/0 (socket says 172.21.15.185:55940) 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfaffd700 1 -- 172.21.15.185:0/3249995011 learned_addr learned my addr 172.21.15.185:0/3249995011 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.011+0000 7fecfb7fe700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecfc063fd0 0x7fecfc0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 -- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 msgr2=0x7fecfc0ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 0x7fecfc0ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 -- 172.21.15.185:0/3249995011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecfc063fd0 msgr2=0x7fecfc0ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecfc063fd0 0x7fecfc0ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 -- 172.21.15.185:0/3249995011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc12f100 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecfbfff700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 0x7fecfc12e050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fecf4012000 tx=0x7fecf4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4006b20 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf401d5d0 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc0b8880 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf401d970 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc0ad700 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fecf4025030 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc05d200 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 0x7feccc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.015+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fecf406f410 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.019+0000 7fecfb7fe700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 0x7feccc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.019+0000 7fecfb7fe700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 0x7feccc057520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fecec009960 tx=0x7fecec0091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.023+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fecf4045670 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.175+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fecfc0b8820 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fecf37fe700 1 -- 172.21.15.185:0/3249995011 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fecf4023020 con 0x7fecfc0b3ec0 2020-11-16T22:14:12.165 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:12.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 msgr2=0x7feccc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 0x7feccc057520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fecec009960 tx=0x7fecec0091d0).stop 2020-11-16T22:14:12.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 msgr2=0x7fecfc12e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 0x7fecfc12e050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fecf4012000 tx=0x7fecf4015000).stop 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 shutdown_connections 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feccc055060 0x7feccc057520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecfc0b3ec0 0x7fecfc12e050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecfc063fd0 0x7fecfc0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 --2- 172.21.15.185:0/3249995011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecfc0af7f0 0x7fecfc0ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 >> 172.21.15.185:0/3249995011 conn(0x7fecfc067940 msgr2=0x7fecfc1394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 shutdown_connections 2020-11-16T22:14:12.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.179+0000 7fed00c87700 1 -- 172.21.15.185:0/3249995011 wait complete. 2020-11-16T22:14:12.180 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:12.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 Processor -- start 2020-11-16T22:14:12.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 -- start start 2020-11-16T22:14:12.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 0x7f3f780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f78144d50 con 0x7f3f7813dea0 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f780ad490 con 0x7f3f780b7440 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f7e87f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780ad730 con 0x7f3f780afc60 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f77fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 0x7f3f780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49394/0 (socket says 172.21.15.185:49394) 2020-11-16T22:14:12.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 -- 172.21.15.185:0/3709736731 learned_addr learned my addr 172.21.15.185:0/3709736731 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 -- 172.21.15.185:0/3709736731 >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 msgr2=0x7f3f780b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:12.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 --2- 172.21.15.185:0/3709736731 >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 0x7f3f780b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 -- 172.21.15.185:0/3709736731 >> v1:172.21.15.96:6789/0 conn(0x7f3f7813dea0 legacy=0x7f3f780ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 -- 172.21.15.185:0/3709736731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780ad9d0 con 0x7f3f780afc60 2020-11-16T22:14:12.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f77fff700 1 --2- 172.21.15.185:0/3709736731 >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 0x7f3f780b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:12.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f777fe700 1 --2- 172.21.15.185:0/3709736731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3f6c006fd0 tx=0x7f3f6c006eb0).ready entity=mon.2 client_cookie=1bd367da394132db server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f76ffd700 1 -- 172.21.15.185:0/3709736731 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c009810 con 0x7f3f780afc60 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f76ffd700 1 -- 172.21.15.185:0/3709736731 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c009970 con 0x7f3f780afc60 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.323+0000 7f3f76ffd700 1 -- 172.21.15.185:0/3709736731 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c009cf0 con 0x7f3f780afc60 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- 172.21.15.185:0/3709736731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 msgr2=0x7f3f78144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/3709736731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f3f6c006fd0 tx=0x7f3f6c006eb0).stop 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- 172.21.15.185:0/3709736731 shutdown_connections 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/3709736731 >> v2:172.21.15.185:3300/0 conn(0x7f3f780b7440 0x7f3f780b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/3709736731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78144860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- 172.21.15.185:0/3709736731 >> 172.21.15.185:0/3709736731 conn(0x7f3f78067860 msgr2=0x7f3f78067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- 172.21.15.185:0/3709736731 shutdown_connections 2020-11-16T22:14:12.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- 172.21.15.185:0/3709736731 wait complete. 2020-11-16T22:14:12.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 Processor -- start 2020-11-16T22:14:12.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.327+0000 7f3f7e87f700 1 -- start start 2020-11-16T22:14:12.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 0x7f3f78140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780b4d20 con 0x7f3f7813dea0 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78153800 con 0x7f3f780b7440 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78157800 con 0x7f3f780afc60 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f777fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f777fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55956/0 (socket says 172.21.15.185:55956) 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49398/0 (socket says 172.21.15.185:49398) 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f777fe700 1 -- 172.21.15.185:0/1224608135 learned_addr learned my addr 172.21.15.185:0/1224608135 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7ce1c700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 0x7f3f78140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 -- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 msgr2=0x7f3f78157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 -- 172.21.15.185:0/1224608135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 msgr2=0x7f3f78140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 0x7f3f78140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 -- 172.21.15.185:0/1224608135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78113ca0 con 0x7f3f780afc60 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f777fe700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7ce1c700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 0x7f3f78140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f77fff700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3f6800ed20 tx=0x7f3f68006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6800eee0 con 0x7f3f780afc60 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6800cbb0 con 0x7f3f780afc60 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680123f0 con 0x7f3f780afc60 2020-11-16T22:14:12.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78113f40 con 0x7f3f780afc60 2020-11-16T22:14:12.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f78154240 con 0x7f3f780afc60 2020-11-16T22:14:12.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f78143df0 con 0x7f3f780afc60 2020-11-16T22:14:12.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.331+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3f680127e0 con 0x7f3f780afc60 2020-11-16T22:14:12.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.335+0000 7f3f57fff700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 0x7f3f50057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.335+0000 7f3f777fe700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 0x7f3f50057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.335+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3f6806e450 con 0x7f3f780afc60 2020-11-16T22:14:12.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.335+0000 7f3f777fe700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 0x7f3f50057520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3f781486b0 tx=0x7f3f6c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.339+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3f78143df0 con 0x7f3f780afc60 2020-11-16T22:14:12.424 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:12.430 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:12.433 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:12.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.487+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f78141000 con 0x7f3f780afc60 2020-11-16T22:14:12.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.487+0000 7f3f57fff700 1 -- 172.21.15.185:0/1224608135 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3f68044540 con 0x7f3f780afc60 2020-11-16T22:14:12.476 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:12.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 msgr2=0x7f3f50057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 0x7f3f50057520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3f781486b0 tx=0x7f3f6c015000).stop 2020-11-16T22:14:12.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 msgr2=0x7f3f78153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3f6800ed20 tx=0x7f3f68006df0).stop 2020-11-16T22:14:12.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 shutdown_connections 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3f50055060 0x7f3f50057520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7813dea0 0x7f3f78140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3f780afc60 0x7f3f78153310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 --2- 172.21.15.185:0/1224608135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3f780b7440 0x7f3f78157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 >> 172.21.15.185:0/1224608135 conn(0x7f3f78067860 msgr2=0x7f3f78146ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 shutdown_connections 2020-11-16T22:14:12.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.491+0000 7f3f7e87f700 1 -- 172.21.15.185:0/1224608135 wait complete. 2020-11-16T22:14:12.491 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:12.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.675+0000 7fe76b3b3700 1 Processor -- start 2020-11-16T22:14:12.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.675+0000 7fe76b3b3700 1 -- start start 2020-11-16T22:14:12.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76b3b3700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe7640b38a0 0x7fe7640b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76b3b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764141130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76b3b3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640b2690 con 0x7fe76405bc50 2020-11-16T22:14:12.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76b3b3700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640ad350 con 0x7fe7640b38a0 2020-11-16T22:14:12.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76b3b3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b2930 con 0x7fe764140cc0 2020-11-16T22:14:12.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe769bb0700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe76405bc50 0x7fe764144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49402/0 (socket says 172.21.15.185:49402) 2020-11-16T22:14:12.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe769bb0700 1 -- 172.21.15.185:0/2502063138 learned_addr learned my addr 172.21.15.185:0/2502063138 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:12.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76a3b1700 1 --2- 172.21.15.185:0/2502063138 >> v2:172.21.15.96:3300/0 conn(0x7fe7640b38a0 0x7fe7640b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe7693af700 1 -- 172.21.15.185:0/2502063138 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055202097 0 0) 0x7fe7640b2690 con 0x7fe76405bc50 2020-11-16T22:14:12.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe7693af700 1 -- 172.21.15.185:0/2502063138 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe74c003240 con 0x7fe76405bc50 2020-11-16T22:14:12.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 -- 172.21.15.185:0/2502063138 >> v2:172.21.15.96:3300/0 conn(0x7fe7640b38a0 msgr2=0x7fe7640b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 --2- 172.21.15.185:0/2502063138 >> v2:172.21.15.96:3300/0 conn(0x7fe7640b38a0 0x7fe7640b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 -- 172.21.15.185:0/2502063138 >> v1:172.21.15.185:6789/0 conn(0x7fe76405bc50 legacy=0x7fe764144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 -- 172.21.15.185:0/2502063138 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7641443d0 con 0x7fe764140cc0 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe76abb2700 1 --2- 172.21.15.185:0/2502063138 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764141130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe75c00b9d0 tx=0x7fe75c006df0).ready entity=mon.2 client_cookie=c5e362ed0f5d3afb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe7693af700 1 -- 172.21.15.185:0/2502063138 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c009640 con 0x7fe764140cc0 2020-11-16T22:14:12.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe7693af700 1 -- 172.21.15.185:0/2502063138 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c0097a0 con 0x7fe764140cc0 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.679+0000 7fe7693af700 1 -- 172.21.15.185:0/2502063138 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c009b20 con 0x7fe764140cc0 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2502063138 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 msgr2=0x7fe764141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2502063138 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764141130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe75c00b9d0 tx=0x7fe75c006df0).stop 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2502063138 shutdown_connections 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2502063138 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764141130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2502063138 >> v2:172.21.15.96:3300/0 conn(0x7fe7640b38a0 0x7fe7640b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2502063138 >> 172.21.15.185:0/2502063138 conn(0x7fe764063df0 msgr2=0x7fe7640641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2502063138 shutdown_connections 2020-11-16T22:14:12.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2502063138 wait complete. 2020-11-16T22:14:12.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 Processor -- start 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- start start 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe76405bc50 0x7fe764141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764148e50 con 0x7fe76405bc50 2020-11-16T22:14:12.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641490f0 con 0x7fe7640b38a0 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe764149390 con 0x7fe764140cc0 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76abb2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55970/0 (socket says 172.21.15.185:55970) 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 -- 172.21.15.185:0/2211038952 learned_addr learned my addr 172.21.15.185:0/2211038952 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76a3b1700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe76405bc50 0x7fe764141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 -- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 msgr2=0x7fe764142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 -- 172.21.15.185:0/2211038952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe76405bc50 msgr2=0x7fe764141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe76405bc50 0x7fe764141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 -- 172.21.15.185:0/2211038952 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764141170 con 0x7fe7640b38a0 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76abb2700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe769bb0700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe75400c6e0 tx=0x7fe754007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540072e0 con 0x7fe7640b38a0 2020-11-16T22:14:12.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754007440 con 0x7fe7640b38a0 2020-11-16T22:14:12.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe764141b00 con 0x7fe7640b38a0 2020-11-16T22:14:12.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754005800 con 0x7fe7640b38a0 2020-11-16T22:14:12.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.683+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764143120 con 0x7fe7640b38a0 2020-11-16T22:14:12.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe764141300 con 0x7fe7640b38a0 2020-11-16T22:14:12.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe754010070 con 0x7fe7640b38a0 2020-11-16T22:14:12.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe7527fc700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 0x7fe73c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe75406d430 con 0x7fe7640b38a0 2020-11-16T22:14:12.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe76a3b1700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 0x7fe73c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.687+0000 7fe76a3b1700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 0x7fe73c057520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe7600099f0 tx=0x7fe7600091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.695+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe754043690 con 0x7fe7640b38a0 2020-11-16T22:14:12.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.847+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe76405d200 con 0x7fe7640b38a0 2020-11-16T22:14:12.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.847+0000 7fe7527fc700 1 -- 172.21.15.185:0/2211038952 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe754007c10 con 0x7fe7640b38a0 2020-11-16T22:14:12.836 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:12.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 msgr2=0x7fe73c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 0x7fe73c057520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe7600099f0 tx=0x7fe7600091d0).stop 2020-11-16T22:14:12.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 msgr2=0x7fe76413df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe75400c6e0 tx=0x7fe754007ad0).stop 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 shutdown_connections 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe73c055060 0x7fe73c057520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe764140cc0 0x7fe764142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe76405bc50 0x7fe764141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 --2- 172.21.15.185:0/2211038952 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe7640b38a0 0x7fe76413df70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 >> 172.21.15.185:0/2211038952 conn(0x7fe764063df0 msgr2=0x7fe764144790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 shutdown_connections 2020-11-16T22:14:12.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.851+0000 7fe76b3b3700 1 -- 172.21.15.185:0/2211038952 wait complete. 2020-11-16T22:14:12.851 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-16T22:14:12.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-16T22:14:12.852 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-16T22:14:12.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.983+0000 7f7ab7fff700 1 Processor -- start 2020-11-16T22:14:12.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.983+0000 7f7ab7fff700 1 -- start start 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7ab80682b0 0x7f7ab8143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab8067510 con 0x7f7ab8067070 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80677b0 con 0x7f7ab80682b0 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80b17a0 con 0x7f7ab80b7470 2020-11-16T22:14:12.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7ab80682b0 0x7f7ab8143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49420/0 (socket says 172.21.15.185:49420) 2020-11-16T22:14:12.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab67fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7ab8067070 0x7f7ab80b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49416/0 (socket says 172.21.15.185:49416) 2020-11-16T22:14:12.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab6ffd700 1 -- 172.21.15.185:0/16354095 learned_addr learned my addr 172.21.15.185:0/16354095 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352746143 0 0) 0x7f7ab8067510 con 0x7f7ab8067070 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa4003240 con 0x7f7ab8067070 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 -- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 msgr2=0x7f7ab80b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3199637974 0 0) 0x7f7aa4003240 con 0x7f7ab8067070 2020-11-16T22:14:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 -- 172.21.15.185:0/16354095 >> v1:172.21.15.185:6789/0 conn(0x7f7ab8067070 legacy=0x7f7ab80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 -- 172.21.15.185:0/16354095 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80b1a40 con 0x7f7ab80682b0 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab6ffd700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab77fe700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8143a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f7ab000b9a0 tx=0x7f7ab0006df0).ready entity=mon.1 client_cookie=4cf92271bca55edc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0014e0 con 0x7f7ab80682b0 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab0011400 con 0x7f7ab80682b0 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab5ffb700 1 -- 172.21.15.185:0/16354095 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0011780 con 0x7f7ab80682b0 2020-11-16T22:14:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 msgr2=0x7f7ab8143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8143a20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f7ab000b9a0 tx=0x7f7ab0006df0).stop 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- 172.21.15.185:0/16354095 shutdown_connections 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8143a20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/16354095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab80b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- 172.21.15.185:0/16354095 >> 172.21.15.185:0/16354095 conn(0x7f7ab80637b0 msgr2=0x7f7ab80b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- 172.21.15.185:0/16354095 shutdown_connections 2020-11-16T22:14:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.987+0000 7f7ab7fff700 1 -- 172.21.15.185:0/16354095 wait complete. 2020-11-16T22:14:12.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.991+0000 7f7ab7fff700 1 Processor -- start 2020-11-16T22:14:12.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.991+0000 7f7ab7fff700 1 -- start start 2020-11-16T22:14:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab8067070 0x7f7ab80adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8131160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8150890 con 0x7f7ab8067070 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8132000 con 0x7f7ab80682b0 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80acd70 con 0x7f7ab80b7470 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49430/0 (socket says 172.21.15.185:49430) 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 -- 172.21.15.185:0/469485106 learned_addr learned my addr 172.21.15.185:0/469485106 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8131160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:55984/0 (socket says 172.21.15.185:55984) 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab6ffd700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab8067070 0x7f7ab80adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 -- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 msgr2=0x7f7ab8131160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8131160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 -- 172.21.15.185:0/469485106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab8067070 msgr2=0x7f7ab80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab8067070 0x7f7ab80adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 -- 172.21.15.185:0/469485106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8156f70 con 0x7f7ab80b7470 2020-11-16T22:14:12.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab77fe700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7ab8140490 tx=0x7f7ab0013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0010b60 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab0010cc0 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab80acfa0 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab001e690 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab80b04e0 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab8157100 con 0x7f7ab80b7470 2020-11-16T22:14:12.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7ab0025030 con 0x7f7ab80b7470 2020-11-16T22:14:12.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 0x7f7a94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:12.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7ab006f190 con 0x7f7ab80b7470 2020-11-16T22:14:12.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.995+0000 7f7ab6ffd700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 0x7f7a94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:12.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:12.999+0000 7f7ab6ffd700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 0x7f7a94057520 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7aa8006d90 tx=0x7f7aa8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:12.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.003+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7ab00453f0 con 0x7f7ab80b7470 2020-11-16T22:14:13.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.191+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7ab805d200 con 0x7f7ab80b7470 2020-11-16T22:14:13.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.199+0000 7f7a9effd700 1 -- 172.21.15.185:0/469485106 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+132413 (secure 0 0 0) 0x7f7ab0023030 con 0x7f7ab80b7470 2020-11-16T22:14:13.188 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 msgr2=0x7f7a94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 0x7f7a94057520 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7aa8006d90 tx=0x7f7aa8008040).stop 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 msgr2=0x7f7ab8131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7ab8140490 tx=0x7f7ab0013000).stop 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 shutdown_connections 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7a94055060 0x7f7a94057520 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ab80b7470 0x7f7ab8131650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab8067070 0x7f7ab80adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 --2- 172.21.15.185:0/469485106 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ab80682b0 0x7f7ab8131160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 >> 172.21.15.185:0/469485106 conn(0x7f7ab80637b0 msgr2=0x7f7ab8141850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:13.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 shutdown_connections 2020-11-16T22:14:13.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.203+0000 7f7ab7fff700 1 -- 172.21.15.185:0/469485106 wait complete. 2020-11-16T22:14:13.209 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":5,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6832","nonce":15035},{"type":"v1","addr":"172.21.15.185:6833","nonce":15035}]},"active_addr":"172.21.15.185:6833/15035","active_change":"2020-11-16T22:13:51.033175+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.185:0","nonce":131473828}]},{"addrvec":[{"type":"v2","addr":"172.21.15.185:0","nonce":261771711}]},{"addrvec":[{"type":"v2","addr":"172.21.15.185:0","nonce":3020121457}]}]}} 2020-11-16T22:14:13.824 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-16T22:14:13.825 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-16T22:14:13.825 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:13.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 Processor -- start 2020-11-16T22:14:13.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 -- start start 2020-11-16T22:14:13.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 0x7ff9840b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff984063020 0x7ff984144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9841484c0 con 0x7ff984147db0 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9841441c0 con 0x7ff984063020 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98a3c2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff984144460 con 0x7ff9840b0300 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 0x7ff9840b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:13.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98895f700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff984147db0 0x7ff98413dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49432/0 (socket says 172.21.15.185:49432) 2020-11-16T22:14:13.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff98895f700 1 -- 172.21.15.185:0/1277467637 learned_addr learned my addr 172.21.15.185:0/1277467637 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:13.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff9837fe700 1 --2- 172.21.15.185:0/1277467637 >> v2:172.21.15.96:3300/0 conn(0x7ff984063020 0x7ff984144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:13.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff982ffd700 1 -- 172.21.15.185:0/1277467637 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580087005 0 0) 0x7ff9841484c0 con 0x7ff984147db0 2020-11-16T22:14:13.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff982ffd700 1 -- 172.21.15.185:0/1277467637 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff964003240 con 0x7ff984147db0 2020-11-16T22:14:13.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 -- 172.21.15.185:0/1277467637 >> v2:172.21.15.96:3300/0 conn(0x7ff984063020 msgr2=0x7ff984144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 --2- 172.21.15.185:0/1277467637 >> v2:172.21.15.96:3300/0 conn(0x7ff984063020 0x7ff984144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 -- 172.21.15.185:0/1277467637 >> v1:172.21.15.185:6789/0 conn(0x7ff984147db0 legacy=0x7ff98413dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 -- 172.21.15.185:0/1277467637 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984144700 con 0x7ff9840b0300 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.967+0000 7ff983fff700 1 --2- 172.21.15.185:0/1277467637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 0x7ff9840b06e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff96c00b9a0 tx=0x7ff96c006df0).ready entity=mon.2 client_cookie=18f76959526d9ba2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff982ffd700 1 -- 172.21.15.185:0/1277467637 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c009710 con 0x7ff9840b0300 2020-11-16T22:14:13.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff982ffd700 1 -- 172.21.15.185:0/1277467637 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c009870 con 0x7ff9840b0300 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff982ffd700 1 -- 172.21.15.185:0/1277467637 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c009bf0 con 0x7ff9840b0300 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1277467637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 msgr2=0x7ff9840b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1277467637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 0x7ff9840b06e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff96c00b9a0 tx=0x7ff96c006df0).stop 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1277467637 shutdown_connections 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1277467637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff9840b0300 0x7ff9840b06e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1277467637 >> v2:172.21.15.96:3300/0 conn(0x7ff984063020 0x7ff984144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1277467637 >> 172.21.15.185:0/1277467637 conn(0x7ff9840b4410 msgr2=0x7ff984060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:13.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1277467637 shutdown_connections 2020-11-16T22:14:13.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1277467637 wait complete. 2020-11-16T22:14:13.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 Processor -- start 2020-11-16T22:14:13.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- start start 2020-11-16T22:14:13.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 0x7ff984142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff984147db0 0x7ff984140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98408fa50 con 0x7ff9840b0300 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98408fcf0 con 0x7ff984147db0 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.971+0000 7ff98a3c2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98408ff90 con 0x7ff984063020 2020-11-16T22:14:13.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff98895f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff984147db0 0x7ff984140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49440/0 (socket says 172.21.15.185:49440) 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 -- 172.21.15.185:0/1917830746 learned_addr learned my addr 172.21.15.185:0/1917830746 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff9837fe700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 0x7ff984142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 -- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff984147db0 msgr2=0x7ff984140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff984147db0 0x7ff984140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 -- 172.21.15.185:0/1917830746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 msgr2=0x7ff984142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:13.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 0x7ff984142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:13.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 -- 172.21.15.185:0/1917830746 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98413ffc0 con 0x7ff984063020 2020-11-16T22:14:13.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff9837fe700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 0x7ff984142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:14.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff983fff700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff96c006e30 tx=0x7ff96c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c011e30 con 0x7ff984063020 2020-11-16T22:14:14.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c01d450 con 0x7ff984063020 2020-11-16T22:14:14.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9840b1a40 con 0x7ff984063020 2020-11-16T22:14:14.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c01d7f0 con 0x7ff984063020 2020-11-16T22:14:14.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98408f600 con 0x7ff984063020 2020-11-16T22:14:14.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98408f850 con 0x7ff984063020 2020-11-16T22:14:14.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.975+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff96c00f070 con 0x7ff984063020 2020-11-16T22:14:14.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.979+0000 7ff97bfff700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 0x7ff960057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.979+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff96c06efc0 con 0x7ff984063020 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.979+0000 7ff9837fe700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 0x7ff960057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.979+0000 7ff9837fe700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 0x7ff960057520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff98413ff70 tx=0x7ff974008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:13.983+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff96c0452d0 con 0x7ff984063020 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.139+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff98405d4e0 con 0x7ff984063020 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff97bfff700 1 -- 172.21.15.185:0/1917830746 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7ff96c02f1d0 con 0x7ff984063020 2020-11-16T22:14:14.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 msgr2=0x7ff960057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 0x7ff960057520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff98413ff70 tx=0x7ff974008040).stop 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 msgr2=0x7ff98413fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff96c006e30 tx=0x7ff96c015000).stop 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 shutdown_connections 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff960055060 0x7ff960057520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff984147db0 0x7ff984140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff984063020 0x7ff98413fa80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 --2- 172.21.15.185:0/1917830746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9840b0300 0x7ff984142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 >> 172.21.15.185:0/1917830746 conn(0x7ff9840b4410 msgr2=0x7ff984060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:14.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 shutdown_connections 2020-11-16T22:14:14.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.143+0000 7ff98a3c2700 1 -- 172.21.15.185:0/1917830746 wait complete. 2020-11-16T22:14:14.172 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:14.172 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:14.374 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-16T22:14:14.375 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:14.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.519+0000 7fe2d1a5c700 1 Processor -- start 2020-11-16T22:14:14.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 -- start start 2020-11-16T22:14:14.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe2cc0b7470 0x7fe2cc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b2920 0x7fe2cc13e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc066ad0 con 0x7fe2cc13e720 2020-11-16T22:14:14.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc140d10 con 0x7fe2cc0b7470 2020-11-16T22:14:14.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d1a5c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc144d90 con 0x7fe2cc0b2920 2020-11-16T22:14:14.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe2cc0b7470 0x7fe2cc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe2cc0b7470 0x7fe2cc0b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56004/0 (socket says 172.21.15.185:56004) 2020-11-16T22:14:14.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 -- 172.21.15.185:0/2893447545 learned_addr learned my addr 172.21.15.185:0/2893447545 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:14.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2cbfff700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b2920 0x7fe2cc13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 -- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b2920 msgr2=0x7fe2cc13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:14.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b2920 0x7fe2cc13e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 -- 172.21.15.185:0/2893447545 >> v1:172.21.15.96:6789/0 conn(0x7fe2cc13e720 legacy=0x7fe2cc141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 -- 172.21.15.185:0/2893447545 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc144ff0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2d0a5a700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc0b7470 0x7fe2cc0b7850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe2bc009d50 tx=0x7fe2bc009290).ready entity=mon.1 client_cookie=90a37b9d7785b3ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2cb7fe700 1 -- 172.21.15.185:0/2893447545 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00bc50 con 0x7fe2cc0b7470 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2cb7fe700 1 -- 172.21.15.185:0/2893447545 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2bc00bdb0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.523+0000 7fe2cb7fe700 1 -- 172.21.15.185:0/2893447545 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc011610 con 0x7fe2cc0b7470 2020-11-16T22:14:14.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc0b7470 msgr2=0x7fe2cc0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc0b7470 0x7fe2cc0b7850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe2bc009d50 tx=0x7fe2bc009290).stop 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/2893447545 shutdown_connections 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc0b7470 0x7fe2cc0b7850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/2893447545 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b2920 0x7fe2cc13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/2893447545 >> 172.21.15.185:0/2893447545 conn(0x7fe2cc0637b0 msgr2=0x7fe2cc0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/2893447545 shutdown_connections 2020-11-16T22:14:14.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/2893447545 wait complete. 2020-11-16T22:14:14.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 Processor -- start 2020-11-16T22:14:14.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.527+0000 7fe2d1a5c700 1 -- start start 2020-11-16T22:14:14.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc0b2920 0x7fe2cc12e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 0x7fe2cc13a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc1460f0 con 0x7fe2cc0b2920 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc1559a0 con 0x7fe2cc13e720 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc154ad0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d125b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 0x7fe2cc13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49456/0 (socket says 172.21.15.185:49456) 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 -- 172.21.15.185:0/1013035941 learned_addr learned my addr 172.21.15.185:0/1013035941 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d0a5a700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc0b2920 0x7fe2cc12e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 -- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 msgr2=0x7fe2cc13a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 0x7fe2cc13a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 -- 172.21.15.185:0/1013035941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc0b2920 msgr2=0x7fe2cc12e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc0b2920 0x7fe2cc12e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 -- 172.21.15.185:0/1013035941 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc154f90 con 0x7fe2cc0b7470 2020-11-16T22:14:14.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d125b700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 0x7fe2cc13a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:14.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2cbfff700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe2c000eba0 tx=0x7fe2c0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000c8e0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c000ca40 con 0x7fe2cc0b7470 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc155230 con 0x7fe2cc0b7470 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000cdc0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc12e840 con 0x7fe2cc0b7470 2020-11-16T22:14:14.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc1553c0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.531+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe2c0012600 con 0x7fe2cc0b7470 2020-11-16T22:14:14.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.535+0000 7fe2c8ff9700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 0x7fe2a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.535+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2c006e090 con 0x7fe2cc0b7470 2020-11-16T22:14:14.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.535+0000 7fe2d0a5a700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 0x7fe2a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.535+0000 7fe2d0a5a700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 0x7fe2a4057520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe2bc0092d0 tx=0x7fe2bc016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.539+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe2c0044290 con 0x7fe2cc0b7470 2020-11-16T22:14:14.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.695+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2cc05d200 con 0x7fe2cc0b7470 2020-11-16T22:14:14.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.695+0000 7fe2c8ff9700 1 -- 172.21.15.185:0/1013035941 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fe2c00478b0 con 0x7fe2cc0b7470 2020-11-16T22:14:14.683 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:14.684 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:14.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 msgr2=0x7fe2a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 0x7fe2a4057520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe2bc0092d0 tx=0x7fe2bc016000).stop 2020-11-16T22:14:14.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 msgr2=0x7fe2cc1441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe2c000eba0 tx=0x7fe2c0006df0).stop 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 shutdown_connections 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe2a4055060 0x7fe2a4057520 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe2cc13e720 0x7fe2cc13a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc0b2920 0x7fe2cc12e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 --2- 172.21.15.185:0/1013035941 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2cc0b7470 0x7fe2cc1441a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 >> 172.21.15.185:0/1013035941 conn(0x7fe2cc0637b0 msgr2=0x7fe2cc13d650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:14.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 shutdown_connections 2020-11-16T22:14:14.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.699+0000 7fe2d1a5c700 1 -- 172.21.15.185:0/1013035941 wait complete. 2020-11-16T22:14:14.703 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-16T22:14:14.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 Processor -- start 2020-11-16T22:14:14.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 -- start start 2020-11-16T22:14:14.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f257c0b72e0 0x7f257c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c147f90 con 0x7f257c0acee0 2020-11-16T22:14:14.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.839+0000 7f2580ef1700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c0af020 con 0x7f257c0b72e0 2020-11-16T22:14:14.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b4810 con 0x7f257c0b64b0 2020-11-16T22:14:14.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49464/0 (socket says 172.21.15.185:49464) 2020-11-16T22:14:14.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 -- 172.21.15.185:0/2384410737 learned_addr learned my addr 172.21.15.185:0/2384410737 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:14.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257b7fe700 1 --2- 172.21.15.185:0/2384410737 >> v2:172.21.15.96:3300/0 conn(0x7f257c0b72e0 0x7f257c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 -- 172.21.15.185:0/2384410737 >> v2:172.21.15.96:3300/0 conn(0x7f257c0b72e0 msgr2=0x7f257c0b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 --2- 172.21.15.185:0/2384410737 >> v2:172.21.15.96:3300/0 conn(0x7f257c0b72e0 0x7f257c0b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 -- 172.21.15.185:0/2384410737 >> v1:172.21.15.96:6789/0 conn(0x7f257c0acee0 legacy=0x7f257c144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 -- 172.21.15.185:0/2384410737 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c148460 con 0x7f257c0b64b0 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257affd700 1 --2- 172.21.15.185:0/2384410737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f256c006fd0 tx=0x7f256c006eb0).ready entity=mon.2 client_cookie=6bd1451640d9f132 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257a7fc700 1 -- 172.21.15.185:0/2384410737 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0098d0 con 0x7f257c0b64b0 2020-11-16T22:14:14.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257a7fc700 1 -- 172.21.15.185:0/2384410737 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c009a30 con 0x7f257c0b64b0 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f257a7fc700 1 -- 172.21.15.185:0/2384410737 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c009db0 con 0x7f257c0b64b0 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- 172.21.15.185:0/2384410737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 msgr2=0x7f257c0b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 --2- 172.21.15.185:0/2384410737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f256c006fd0 tx=0x7f256c006eb0).stop 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- 172.21.15.185:0/2384410737 shutdown_connections 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 --2- 172.21.15.185:0/2384410737 >> v2:172.21.15.96:3300/0 conn(0x7f257c0b72e0 0x7f257c0b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 --2- 172.21.15.185:0/2384410737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0b64b0 0x7f257c0b6aa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- 172.21.15.185:0/2384410737 >> 172.21.15.185:0/2384410737 conn(0x7f257c067860 msgr2=0x7f257c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:14.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- 172.21.15.185:0/2384410737 shutdown_connections 2020-11-16T22:14:14.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 -- 172.21.15.185:0/2384410737 wait complete. 2020-11-16T22:14:14.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.843+0000 7f2580ef1700 1 Processor -- start 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- start start 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 0x7f257c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f257c0b64b0 0x7f257c1165b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f257c0b72e0 0x7f257c116aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b3340 con 0x7f257c0b64b0 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b3900 con 0x7f257c0b72e0 2020-11-16T22:14:14.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b3ba0 con 0x7f257c0acee0 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 0x7f257c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 0x7f257c152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49468/0 (socket says 172.21.15.185:49468) 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 -- 172.21.15.185:0/3479199281 learned_addr learned my addr 172.21.15.185:0/3479199281 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257bfff700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f257c0b72e0 0x7f257c116aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257affd700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f257c0b64b0 0x7f257c1165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 -- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f257c0b72e0 msgr2=0x7f257c116aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f257c0b72e0 0x7f257c116aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 -- 172.21.15.185:0/3479199281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f257c0b64b0 msgr2=0x7f257c1165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f257c0b64b0 0x7f257c1165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c153e80 con 0x7f257c0acee0 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f257b7fe700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 0x7f257c152af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2570006770 tx=0x7f2570008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570009580 con 0x7f257c0acee0 2020-11-16T22:14:14.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700096e0 con 0x7f257c0acee0 2020-11-16T22:14:14.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c140ad0 con 0x7f257c0acee0 2020-11-16T22:14:14.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570009a60 con 0x7f257c0acee0 2020-11-16T22:14:14.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.847+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c116f90 con 0x7f257c0acee0 2020-11-16T22:14:14.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f257c0b3160 con 0x7f257c0acee0 2020-11-16T22:14:14.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2570013480 con 0x7f257c0acee0 2020-11-16T22:14:14.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2554055060 0x7f2554057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f257006e2e0 con 0x7f257c0acee0 2020-11-16T22:14:14.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f257affd700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2554055060 0x7f2554057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x7f255405c5a0 0x7f255405e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f255405f010 con 0x7f255405c5a0 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2570070950 con 0x7f257c0acee0 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f257bfff700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x7f255405c5a0 0x7f255405e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f257affd700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2554055060 0x7f2554057520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f257c0ba190 tx=0x7f256c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f257bfff700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x7f255405c5a0 0x7f255405e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:14.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.851+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== osd.0 v2:172.21.15.185:6808/15202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f255405f010 con 0x7f255405c5a0 2020-11-16T22:14:14.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 --> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f257c05d200 con 0x7f255405c5a0 2020-11-16T22:14:14.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f255b7fe700 1 -- 172.21.15.185:0/3479199281 <== osd.0 v2:172.21.15.185:6808/15202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f257c05d200 con 0x7f255405c5a0 2020-11-16T22:14:14.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x7f255405c5a0 msgr2=0x7f255405e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x7f255405c5a0 0x7f255405e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:14.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2554055060 msgr2=0x7f2554057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2554055060 0x7f2554057520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f257c0ba190 tx=0x7f256c015000).stop 2020-11-16T22:14:14.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 msgr2=0x7f257c152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 --2- 172.21.15.185:0/3479199281 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f257c0acee0 0x7f257c152af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2570006770 tx=0x7f2570008040).stop 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f257bfff700 1 -- 172.21.15.185:0/3479199281 reap_dead start 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 shutdown_connections 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 >> 172.21.15.185:0/3479199281 conn(0x7f257c067860 msgr2=0x7f257c141230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 shutdown_connections 2020-11-16T22:14:14.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:14.871+0000 7f2580ef1700 1 -- 172.21.15.185:0/3479199281 wait complete. 2020-11-16T22:14:14.872 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:14.873 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-16T22:14:14.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.007+0000 7f7e35b11700 1 Processor -- start 2020-11-16T22:14:14.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.007+0000 7f7e35b11700 1 -- start start 2020-11-16T22:14:14.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7e300b3090 0x7f7e300b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e301453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e301458a0 0x7f7e30145d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e30060ef0 con 0x7f7e301458a0 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e30062090 con 0x7f7e300b3090 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300ad640 con 0x7f7e300b3b80 2020-11-16T22:14:14.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7e300b3090 0x7f7e300b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e301453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7e300b3090 0x7f7e300b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56034/0 (socket says 172.21.15.185:56034) 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 -- 172.21.15.185:0/2627107064 learned_addr learned my addr 172.21.15.185:0/2627107064 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35310700 1 --2- 172.21.15.185:0/2627107064 >> v2:172.21.15.96:3300/0 conn(0x7f7e301458a0 0x7f7e30145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 -- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 msgr2=0x7f7e301453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e301453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 -- 172.21.15.185:0/2627107064 >> v2:172.21.15.96:3300/0 conn(0x7f7e301458a0 msgr2=0x7f7e30145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2627107064 >> v2:172.21.15.96:3300/0 conn(0x7f7e301458a0 0x7f7e30145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 -- 172.21.15.185:0/2627107064 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300ad8e0 con 0x7f7e300b3090 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e300b1840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f7e18006fd0 tx=0x7f7e18006eb0).ready entity=mon.1 client_cookie=f37e84f3a09ed4c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e2f7fe700 1 -- 172.21.15.185:0/2627107064 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180097d0 con 0x7f7e300b3090 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e2f7fe700 1 -- 172.21.15.185:0/2627107064 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18009930 con 0x7f7e300b3090 2020-11-16T22:14:15.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e2f7fe700 1 -- 172.21.15.185:0/2627107064 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18009cb0 con 0x7f7e300b3090 2020-11-16T22:14:15.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 msgr2=0x7f7e300b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e300b1840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f7e18006fd0 tx=0x7f7e18006eb0).stop 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- 172.21.15.185:0/2627107064 shutdown_connections 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2627107064 >> v2:172.21.15.96:3300/0 conn(0x7f7e301458a0 0x7f7e30145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e300b1840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2627107064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e301453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- 172.21.15.185:0/2627107064 >> 172.21.15.185:0/2627107064 conn(0x7f7e30060130 msgr2=0x7f7e30060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- 172.21.15.185:0/2627107064 shutdown_connections 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.011+0000 7f7e35b11700 1 -- 172.21.15.185:0/2627107064 wait complete. 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 Processor -- start 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- start start 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e30153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e30067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e301458a0 0x7f7e3013c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30143f90 con 0x7f7e301458a0 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e301534d0 con 0x7f7e300b3090 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30154330 con 0x7f7e300b3b80 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e34b0f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e30153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e30067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e34b0f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e30153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56040/0 (socket says 172.21.15.185:56040) 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e30067310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49486/0 (socket says 172.21.15.185:49486) 2020-11-16T22:14:15.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e34b0f700 1 -- 172.21.15.185:0/2101344016 learned_addr learned my addr 172.21.15.185:0/2101344016 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35310700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e301458a0 0x7f7e3013c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 -- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 msgr2=0x7f7e30153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e30153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 -- 172.21.15.185:0/2101344016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e301458a0 msgr2=0x7f7e3013c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e301458a0 0x7f7e3013c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e301452f0 con 0x7f7e300b3b80 2020-11-16T22:14:15.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35310700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e301458a0 0x7f7e3013c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e300b3090 0x7f7e30153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2ffff700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e30067310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7e2000eba0 tx=0x7f7e20006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000b2c0 con 0x7f7e300b3b80 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2000c8e0 con 0x7f7e300b3b80 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e30066320 con 0x7f7e300b3b80 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000cc80 con 0x7f7e300b3b80 2020-11-16T22:14:15.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e300665e0 con 0x7f7e300b3b80 2020-11-16T22:14:15.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.015+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e3005d200 con 0x7f7e300b3b80 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7e2001c460 con 0x7f7e300b3b80 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7e0c055060 0x7f7e0c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7e2006dfd0 con 0x7f7e300b3b80 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7e0c055060 0x7f7e0c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x7f7e0c05c5a0 0x7f7e0c05e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e0c05f010 con 0x7f7e0c05c5a0 2020-11-16T22:14:15.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e20073500 con 0x7f7e300b3b80 2020-11-16T22:14:15.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e35310700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x7f7e0c05c5a0 0x7f7e0c05e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e34b0f700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7e0c055060 0x7f7e0c057520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7e1800beb0 tx=0x7f7e18015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e35310700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x7f7e0c05c5a0 0x7f7e0c05e9c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.019+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== osd.1 v2:172.21.15.185:6812/15205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f7e0c05f010 con 0x7f7e0c05c5a0 2020-11-16T22:14:15.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.039+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e30153610 con 0x7f7e0c05c5a0 2020-11-16T22:14:15.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e2cff9700 1 -- 172.21.15.185:0/2101344016 <== osd.1 v2:172.21.15.185:6812/15205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7e30153610 con 0x7f7e0c05c5a0 2020-11-16T22:14:15.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x7f7e0c05c5a0 msgr2=0x7f7e0c05e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x7f7e0c05c5a0 0x7f7e0c05e9c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7e0c055060 msgr2=0x7f7e0c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7e0c055060 0x7f7e0c057520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7e1800beb0 tx=0x7f7e18015000).stop 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 msgr2=0x7f7e30067310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 --2- 172.21.15.185:0/2101344016 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e300b3b80 0x7f7e30067310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7e2000eba0 tx=0x7f7e20006df0).stop 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35310700 1 -- 172.21.15.185:0/2101344016 reap_dead start 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 shutdown_connections 2020-11-16T22:14:15.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 >> 172.21.15.185:0/2101344016 conn(0x7f7e30060130 msgr2=0x7f7e300b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 shutdown_connections 2020-11-16T22:14:15.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.043+0000 7f7e35b11700 1 -- 172.21.15.185:0/2101344016 wait complete. 2020-11-16T22:14:15.050 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:15.050 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-16T22:14:15.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 Processor -- start 2020-11-16T22:14:15.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 -- start start 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f24780b2450 0x7f24780674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24780b3b70 con 0x7f24780670c0 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2478067990 con 0x7f247813f9b0 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247ec48700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2478144500 con 0x7f24780b2450 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247e447700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f247813f9b0 0x7f247813fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49492/0 (socket says 172.21.15.185:49492) 2020-11-16T22:14:15.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247e447700 1 -- 172.21.15.185:0/3183119111 learned_addr learned my addr 172.21.15.185:0/3183119111 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:15.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247d445700 1 --2- 172.21.15.185:0/3183119111 >> v2:172.21.15.96:3300/0 conn(0x7f24780b2450 0x7f24780674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588325346 0 0) 0x7f2478067990 con 0x7f247813f9b0 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2460003240 con 0x7f247813f9b0 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828931612 0 0) 0x7f24780b3b70 con 0x7f24780670c0 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2478067990 con 0x7f24780670c0 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247d445700 1 -- 172.21.15.185:0/3183119111 >> v1:172.21.15.185:6789/0 conn(0x7f247813f9b0 legacy=0x7f247813fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247d445700 1 -- 172.21.15.185:0/3183119111 >> v1:172.21.15.96:6789/0 conn(0x7f24780670c0 legacy=0x7f2478068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247d445700 1 -- 172.21.15.185:0/3183119111 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24781447a0 con 0x7f24780b2450 2020-11-16T22:14:15.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247d445700 1 --2- 172.21.15.185:0/3183119111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24780b2450 0x7f24780674a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f246800b9a0 tx=0x7f2468006df0).ready entity=mon.0 client_cookie=fad0e725e6af1665 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468009680 con 0x7f24780b2450 2020-11-16T22:14:15.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24680097e0 con 0x7f24780b2450 2020-11-16T22:14:15.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.183+0000 7f247cc44700 1 -- 172.21.15.185:0/3183119111 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468009b60 con 0x7f24780b2450 2020-11-16T22:14:15.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- 172.21.15.185:0/3183119111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24780b2450 msgr2=0x7f24780674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 --2- 172.21.15.185:0/3183119111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24780b2450 0x7f24780674a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f246800b9a0 tx=0x7f2468006df0).stop 2020-11-16T22:14:15.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- 172.21.15.185:0/3183119111 shutdown_connections 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 --2- 172.21.15.185:0/3183119111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24780b2450 0x7f24780674a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- 172.21.15.185:0/3183119111 >> 172.21.15.185:0/3183119111 conn(0x7f2478060130 msgr2=0x7f2478060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- 172.21.15.185:0/3183119111 shutdown_connections 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- 172.21.15.185:0/3183119111 wait complete. 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 Processor -- start 2020-11-16T22:14:15.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- start start 2020-11-16T22:14:15.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24780670c0 0x7f2478142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247813f9b0 0x7f24780aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24780aefd0 0x7f247813ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478136a50 con 0x7f247813f9b0 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247813a770 con 0x7f24780aefd0 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247ec48700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247813a9d0 con 0x7f24780670c0 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247e447700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24780aefd0 0x7f247813ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247dc46700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24780670c0 0x7f2478142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247dc46700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24780670c0 0x7f2478142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49502/0 (socket says 172.21.15.185:49502) 2020-11-16T22:14:15.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247dc46700 1 -- 172.21.15.185:0/2497707055 learned_addr learned my addr 172.21.15.185:0/2497707055 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.187+0000 7f247d445700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247813f9b0 0x7f24780aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 -- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24780670c0 msgr2=0x7f2478142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24780670c0 0x7f2478142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 -- 172.21.15.185:0/2497707055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247813f9b0 msgr2=0x7f24780aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247813f9b0 0x7f24780aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24780afe90 con 0x7f24780aefd0 2020-11-16T22:14:15.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247e447700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24780aefd0 0x7f247813ee50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f247000c6e0 tx=0x7f2470007940).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700072e0 con 0x7f24780aefd0 2020-11-16T22:14:15.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470007440 con 0x7f24780aefd0 2020-11-16T22:14:15.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24780b0130 con 0x7f24780aefd0 2020-11-16T22:14:15.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470005800 con 0x7f24780aefd0 2020-11-16T22:14:15.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247813c6e0 con 0x7f24780aefd0 2020-11-16T22:14:15.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f247805d200 con 0x7f24780aefd0 2020-11-16T22:14:15.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f247001a060 con 0x7f24780aefd0 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2454055060 0x7f2454057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f247dc46700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2454055060 0x7f2454057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.191+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f247006d810 con 0x7f24780aefd0 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f2465ffb700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x7f245405c5a0 0x7f245405e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f245405f010 con 0x7f245405c5a0 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247006db00 con 0x7f24780aefd0 2020-11-16T22:14:15.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f247dc46700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2454055060 0x7f2454057520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2474007fd0 tx=0x7f2474007e10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f247d445700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x7f245405c5a0 0x7f245405e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f247d445700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x7f245405c5a0 0x7f245405e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.195+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== osd.2 v2:172.21.15.185:6800/15193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f245405f010 con 0x7f245405c5a0 2020-11-16T22:14:15.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.215+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2478061ed0 con 0x7f245405c5a0 2020-11-16T22:14:15.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f2465ffb700 1 -- 172.21.15.185:0/2497707055 <== osd.2 v2:172.21.15.185:6800/15193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2478061ed0 con 0x7f245405c5a0 2020-11-16T22:14:15.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x7f245405c5a0 msgr2=0x7f245405e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x7f245405c5a0 0x7f245405e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2454055060 msgr2=0x7f2454057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2454055060 0x7f2454057520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2474007fd0 tx=0x7f2474007e10).stop 2020-11-16T22:14:15.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24780aefd0 msgr2=0x7f247813ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 --2- 172.21.15.185:0/2497707055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24780aefd0 0x7f247813ee50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f247000c6e0 tx=0x7f2470007940).stop 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247e447700 1 -- 172.21.15.185:0/2497707055 reap_dead start 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 shutdown_connections 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 >> 172.21.15.185:0/2497707055 conn(0x7f2478060130 msgr2=0x7f24780acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 shutdown_connections 2020-11-16T22:14:15.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.219+0000 7f247ec48700 1 -- 172.21.15.185:0/2497707055 wait complete. 2020-11-16T22:14:15.218 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:15.219 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2020-11-16T22:14:15.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.351+0000 7fe199ee4700 1 Processor -- start 2020-11-16T22:14:15.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.351+0000 7fe199ee4700 1 -- start start 2020-11-16T22:14:15.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 0x7fe194141050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe194141620 con 0x7fe19413e440 2020-11-16T22:14:15.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1941418c0 con 0x7fe1940b2af0 2020-11-16T22:14:15.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe194141b60 con 0x7fe19413e820 2020-11-16T22:14:15.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe193fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe1940b2af0 0x7fe1941446f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49504/0 (socket says 172.21.15.185:49504) 2020-11-16T22:14:15.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe193fff700 1 -- 172.21.15.185:0/3715415542 learned_addr learned my addr 172.21.15.185:0/3715415542 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:15.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe192ffd700 1 --2- 172.21.15.185:0/3715415542 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 0x7fe194141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380255945 0 0) 0x7fe1941418c0 con 0x7fe1940b2af0 2020-11-16T22:14:15.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178003240 con 0x7fe1940b2af0 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647720523 0 0) 0x7fe194141620 con 0x7fe19413e440 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1941418c0 con 0x7fe19413e440 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe192ffd700 1 -- 172.21.15.185:0/3715415542 >> v1:172.21.15.185:6789/0 conn(0x7fe1940b2af0 legacy=0x7fe1941446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe192ffd700 1 -- 172.21.15.185:0/3715415542 >> v1:172.21.15.96:6789/0 conn(0x7fe19413e440 legacy=0x7fe1940b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe192ffd700 1 -- 172.21.15.185:0/3715415542 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940b3ea0 con 0x7fe19413e820 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe192ffd700 1 --2- 172.21.15.185:0/3715415542 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 0x7fe194141050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe184009cc0 tx=0x7fe1840091d0).ready entity=mon.2 client_cookie=31c6d4c12258f925 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400ba20 con 0x7fe19413e820 2020-11-16T22:14:15.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18400bb80 con 0x7fe19413e820 2020-11-16T22:14:15.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe1927fc700 1 -- 172.21.15.185:0/3715415542 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184011400 con 0x7fe19413e820 2020-11-16T22:14:15.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- 172.21.15.185:0/3715415542 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 msgr2=0x7fe194141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 --2- 172.21.15.185:0/3715415542 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 0x7fe194141050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe184009cc0 tx=0x7fe1840091d0).stop 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- 172.21.15.185:0/3715415542 shutdown_connections 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 --2- 172.21.15.185:0/3715415542 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e820 0x7fe194141050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- 172.21.15.185:0/3715415542 >> 172.21.15.185:0/3715415542 conn(0x7fe1940604a0 msgr2=0x7fe1940608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- 172.21.15.185:0/3715415542 shutdown_connections 2020-11-16T22:14:15.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.355+0000 7fe199ee4700 1 -- 172.21.15.185:0/3715415542 wait complete. 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 Processor -- start 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- start start 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1940b2af0 0x7fe1940acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e440 0x7fe19409a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 0x7fe19409ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940ab860 con 0x7fe1940b2af0 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19409b7a0 con 0x7fe19413e820 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940ad600 con 0x7fe19413e440 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 0x7fe19409ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe192ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e440 0x7fe19409a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 0x7fe19409ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56076/0 (socket says 172.21.15.185:56076) 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 -- 172.21.15.185:0/1361676745 learned_addr learned my addr 172.21.15.185:0/1361676745 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe1937fe700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1940b2af0 0x7fe1940acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 -- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e440 msgr2=0x7fe19409a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e440 0x7fe19409a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 -- 172.21.15.185:0/1361676745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1940b2af0 msgr2=0x7fe1940acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1940b2af0 0x7fe1940acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19409b500 con 0x7fe19413e820 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe192ffd700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe19413e440 0x7fe19409a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:15.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe193fff700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 0x7fe19409ac60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe18c00c6e0 tx=0x7fe18c007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0072e0 con 0x7fe19413e820 2020-11-16T22:14:15.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c007440 con 0x7fe19413e820 2020-11-16T22:14:15.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe194099650 con 0x7fe19413e820 2020-11-16T22:14:15.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c005800 con 0x7fe19413e820 2020-11-16T22:14:15.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.359+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940ad3d0 con 0x7fe19413e820 2020-11-16T22:14:15.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe19405d4e0 con 0x7fe19413e820 2020-11-16T22:14:15.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe18c010070 con 0x7fe19413e820 2020-11-16T22:14:15.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe164055090 0x7fe164057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe18c06d430 con 0x7fe19413e820 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe1937fe700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe164055090 0x7fe164057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x7fe16405c5d0 0x7fe16405e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16405f040 con 0x7fe16405c5d0 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c06f990 con 0x7fe19413e820 2020-11-16T22:14:15.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe192ffd700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x7fe16405c5d0 0x7fe16405e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe1937fe700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe164055090 0x7fe164057550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe17c007f90 tx=0x7fe17c007990).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe192ffd700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x7fe16405c5d0 0x7fe16405e9f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.363+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fe16405f040 con 0x7fe16405c5d0 2020-11-16T22:14:15.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1940b2220 con 0x7fe16405c5d0 2020-11-16T22:14:15.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe18b7fe700 1 -- 172.21.15.185:0/1361676745 <== osd.3 v2:172.21.15.185:6824/15203 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1940b2220 con 0x7fe16405c5d0 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x7fe16405c5d0 msgr2=0x7fe16405e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x7fe16405c5d0 0x7fe16405e9f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe164055090 msgr2=0x7fe164057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe164055090 0x7fe164057550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe17c007f90 tx=0x7fe17c007990).stop 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 msgr2=0x7fe19409ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 --2- 172.21.15.185:0/1361676745 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe19413e820 0x7fe19409ac60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe18c00c6e0 tx=0x7fe18c007a20).stop 2020-11-16T22:14:15.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe193fff700 1 -- 172.21.15.185:0/1361676745 reap_dead start 2020-11-16T22:14:15.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 shutdown_connections 2020-11-16T22:14:15.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 >> 172.21.15.185:0/1361676745 conn(0x7fe1940604a0 msgr2=0x7fe194065e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 shutdown_connections 2020-11-16T22:14:15.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.383+0000 7fe199ee4700 1 -- 172.21.15.185:0/1361676745 wait complete. 2020-11-16T22:14:15.390 INFO:teuthology.orchestra.run.smithi185.stdout:55834574855 2020-11-16T22:14:15.390 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2020-11-16T22:14:15.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 Processor -- start 2020-11-16T22:14:15.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 -- start start 2020-11-16T22:14:15.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7de00acc10 0x7f7de01432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de00abf90 con 0x7f7de00669a0 2020-11-16T22:14:15.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de00ac230 con 0x7f7de0061e60 2020-11-16T22:14:15.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de72a3700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de00ac4d0 con 0x7f7de00acc10 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de6aa2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7de0061e60 0x7f7de01437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49522/0 (socket says 172.21.15.185:49522) 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de6aa2700 1 -- 172.21.15.185:0/3477741053 learned_addr learned my addr 172.21.15.185:0/3477741053 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de5aa0700 1 --2- 172.21.15.185:0/3477741053 >> v2:172.21.15.96:3300/0 conn(0x7f7de00acc10 0x7f7de01432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176196481 0 0) 0x7f7de00ac230 con 0x7f7de0061e60 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc0003240 con 0x7f7de0061e60 2020-11-16T22:14:15.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156331950 0 0) 0x7f7de00abf90 con 0x7f7de00669a0 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de00ac230 con 0x7f7de00669a0 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de5aa0700 1 -- 172.21.15.185:0/3477741053 >> v1:172.21.15.185:6789/0 conn(0x7f7de0061e60 legacy=0x7f7de01437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de5aa0700 1 -- 172.21.15.185:0/3477741053 >> v1:172.21.15.96:6789/0 conn(0x7f7de00669a0 legacy=0x7f7de0066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de5aa0700 1 -- 172.21.15.185:0/3477741053 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de00ac770 con 0x7f7de00acc10 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de5aa0700 1 --2- 172.21.15.185:0/3477741053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00acc10 0x7f7de01432e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7dc800b9a0 tx=0x7f7dc8006df0).ready entity=mon.0 client_cookie=a6c34ec4a5c60a8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8009680 con 0x7f7de00acc10 2020-11-16T22:14:15.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc80097e0 con 0x7f7de00acc10 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.531+0000 7f7de529f700 1 -- 172.21.15.185:0/3477741053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8009b60 con 0x7f7de00acc10 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/3477741053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00acc10 msgr2=0x7f7de01432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 --2- 172.21.15.185:0/3477741053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00acc10 0x7f7de01432e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7dc800b9a0 tx=0x7f7dc8006df0).stop 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/3477741053 shutdown_connections 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 --2- 172.21.15.185:0/3477741053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00acc10 0x7f7de01432e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/3477741053 >> 172.21.15.185:0/3477741053 conn(0x7f7de0060130 msgr2=0x7f7de0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/3477741053 shutdown_connections 2020-11-16T22:14:15.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/3477741053 wait complete. 2020-11-16T22:14:15.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 Processor -- start 2020-11-16T22:14:15.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- start start 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7de0061e60 0x7f7de0153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 0x7f7de01572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00ad980 0x7f7de0143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ae760 con 0x7f7de00ad980 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0153960 con 0x7f7de00669a0 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ae4c0 con 0x7f7de0061e60 2020-11-16T22:14:15.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 0x7f7de01572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 0x7f7de01572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56088/0 (socket says 172.21.15.185:56088) 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 -- 172.21.15.185:0/2758409718 learned_addr learned my addr 172.21.15.185:0/2758409718 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de62a1700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7de0061e60 0x7f7de0153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de6aa2700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00ad980 0x7f7de0143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 -- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7de0061e60 msgr2=0x7f7de0153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7de0061e60 0x7f7de0153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 -- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00ad980 msgr2=0x7f7de0143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00ad980 0x7f7de0143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de0156b90 con 0x7f7de00669a0 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de6aa2700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de00ad980 0x7f7de0143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de5aa0700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 0x7f7de01572d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7de00619f0 tx=0x7f7dc8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8011d70 con 0x7f7de00669a0 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc801d450 con 0x7f7de00669a0 2020-11-16T22:14:15.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc801d810 con 0x7f7de00669a0 2020-11-16T22:14:15.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de0156e30 con 0x7f7de00669a0 2020-11-16T22:14:15.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.535+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de0149c70 con 0x7f7de00669a0 2020-11-16T22:14:15.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7de005d200 con 0x7f7de00669a0 2020-11-16T22:14:15.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7dc8024030 con 0x7f7de00669a0 2020-11-16T22:14:15.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dbc055060 0x7f7dbc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7dc806f1c0 con 0x7f7de00669a0 2020-11-16T22:14:15.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7de62a1700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dbc055060 0x7f7dbc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x7f7dbc05c520 0x7f7dbc05e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dbc05ef90 con 0x7f7dbc05c520 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc8071830 con 0x7f7de00669a0 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7de62a1700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dbc055060 0x7f7dbc057520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7dd0000f40 tx=0x7f7dd0007d80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7de6aa2700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x7f7dbc05c520 0x7f7dbc05e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.539+0000 7f7de6aa2700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x7f7dbc05c520 0x7f7dbc05e940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.543+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== osd.4 v2:172.21.15.96:6800/15251 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f7dbc05ef90 con 0x7f7dbc05c520 2020-11-16T22:14:15.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 --> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7de0157010 con 0x7f7dbc05c520 2020-11-16T22:14:15.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7dde7fc700 1 -- 172.21.15.185:0/2758409718 <== osd.4 v2:172.21.15.96:6800/15251 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7de0157010 con 0x7f7dbc05c520 2020-11-16T22:14:15.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x7f7dbc05c520 msgr2=0x7f7dbc05e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x7f7dbc05c520 0x7f7dbc05e940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dbc055060 msgr2=0x7f7dbc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dbc055060 0x7f7dbc057520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7dd0000f40 tx=0x7f7dd0007d80).stop 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 msgr2=0x7f7de01572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 --2- 172.21.15.185:0/2758409718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7de00669a0 0x7f7de01572d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7de00619f0 tx=0x7f7dc8015000).stop 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de6aa2700 1 -- 172.21.15.185:0/2758409718 reap_dead start 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 shutdown_connections 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 >> 172.21.15.185:0/2758409718 conn(0x7f7de0060130 msgr2=0x7f7de0142630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 shutdown_connections 2020-11-16T22:14:15.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.563+0000 7f7de72a3700 1 -- 172.21.15.185:0/2758409718 wait complete. 2020-11-16T22:14:15.565 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:15.565 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2020-11-16T22:14:15.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 Processor -- start 2020-11-16T22:14:15.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 -- start start 2020-11-16T22:14:15.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe96c0acc10 0x7fe96c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe96c061e60 0x7fe96c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0abf90 con 0x7fe96c0669a0 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0ac230 con 0x7fe96c061e60 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe973361700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0ac4d0 con 0x7fe96c0acc10 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe96c0acc10 0x7fe96c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe96c0acc10 0x7fe96c1432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56100/0 (socket says 172.21.15.185:56100) 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 -- 172.21.15.185:0/1822673573 learned_addr learned my addr 172.21.15.185:0/1822673573 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe972b60700 1 --2- 172.21.15.185:0/1822673573 >> v2:172.21.15.96:3300/0 conn(0x7fe96c061e60 0x7fe96c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 -- 172.21.15.185:0/1822673573 >> v2:172.21.15.96:3300/0 conn(0x7fe96c061e60 msgr2=0x7fe96c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 --2- 172.21.15.185:0/1822673573 >> v2:172.21.15.96:3300/0 conn(0x7fe96c061e60 0x7fe96c1437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 -- 172.21.15.185:0/1822673573 >> v1:172.21.15.96:6789/0 conn(0x7fe96c0669a0 legacy=0x7fe96c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 -- 172.21.15.185:0/1822673573 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0ac770 con 0x7fe96c0acc10 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe971b5e700 1 --2- 172.21.15.185:0/1822673573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c0acc10 0x7fe96c1432e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe96c067490 tx=0x7fe954009290).ready entity=mon.1 client_cookie=c88bec62126bd887 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe97135d700 1 -- 172.21.15.185:0/1822673573 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400bbc0 con 0x7fe96c0acc10 2020-11-16T22:14:15.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe97135d700 1 -- 172.21.15.185:0/1822673573 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400bd20 con 0x7fe96c0acc10 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.703+0000 7fe97135d700 1 -- 172.21.15.185:0/1822673573 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954011610 con 0x7fe96c0acc10 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- 172.21.15.185:0/1822673573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c0acc10 msgr2=0x7fe96c1432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- 172.21.15.185:0/1822673573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c0acc10 0x7fe96c1432e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe96c067490 tx=0x7fe954009290).stop 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- 172.21.15.185:0/1822673573 shutdown_connections 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- 172.21.15.185:0/1822673573 >> v2:172.21.15.96:3300/0 conn(0x7fe96c061e60 0x7fe96c1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- 172.21.15.185:0/1822673573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c0acc10 0x7fe96c1432e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- 172.21.15.185:0/1822673573 >> 172.21.15.185:0/1822673573 conn(0x7fe96c060130 msgr2=0x7fe96c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- 172.21.15.185:0/1822673573 shutdown_connections 2020-11-16T22:14:15.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- 172.21.15.185:0/1822673573 wait complete. 2020-11-16T22:14:15.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 Processor -- start 2020-11-16T22:14:15.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- start start 2020-11-16T22:14:15.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 0x7fe96c115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe96c0669a0 0x7fe96c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 0x7fe96c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c1537b0 con 0x7fe96c0669a0 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0b0410 con 0x7fe96c061e60 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe973361700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0b06b0 con 0x7fe96c0b5170 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe97235f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 0x7fe96c115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe972b60700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 0x7fe96c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe972b60700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 0x7fe96c0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49552/0 (socket says 172.21.15.185:49552) 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe97235f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 0x7fe96c115450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56106/0 (socket says 172.21.15.185:56106) 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe972b60700 1 -- 172.21.15.185:0/2300768609 learned_addr learned my addr 172.21.15.185:0/2300768609 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.707+0000 7fe971b5e700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe96c0669a0 0x7fe96c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 -- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 msgr2=0x7fe96c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 0x7fe96c0aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 -- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe96c0669a0 msgr2=0x7fe96c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe96c0669a0 0x7fe96c0b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c1539f0 con 0x7fe96c061e60 2020-11-16T22:14:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe972b60700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe96c0b5170 0x7fe96c0aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe97235f700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 0x7fe96c115450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe95c00c6e0 tx=0x7fe95c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00f470 con 0x7fe96c061e60 2020-11-16T22:14:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c00f5d0 con 0x7fe96c061e60 2020-11-16T22:14:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00f970 con 0x7fe96c061e60 2020-11-16T22:14:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c117590 con 0x7fe96c061e60 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c0ae860 con 0x7fe96c061e60 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe96c05d200 con 0x7fe96c061e60 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.711+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe95c019030 con 0x7fe96c061e60 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe948055060 0x7fe948057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe95c06cfe0 con 0x7fe96c061e60 2020-11-16T22:14:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe971b5e700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe948055060 0x7fe948057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x7fe94805c5a0 0x7fe94805e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe94805f010 con 0x7fe94805c5a0 2020-11-16T22:14:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe95c06f560 con 0x7fe96c061e60 2020-11-16T22:14:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe971b5e700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe948055060 0x7fe948057520 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe9540090f0 tx=0x7fe954016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe972b60700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x7fe94805c5a0 0x7fe94805e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe972b60700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x7fe94805c5a0 0x7fe94805e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.715+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fe94805f010 con 0x7fe94805c5a0 2020-11-16T22:14:15.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe96c0b3930 con 0x7fe94805c5a0 2020-11-16T22:14:15.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe96a7fc700 1 -- 172.21.15.185:0/2300768609 <== osd.5 v2:172.21.15.96:6809/15269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe96c0b3930 con 0x7fe94805c5a0 2020-11-16T22:14:15.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x7fe94805c5a0 msgr2=0x7fe94805e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x7fe94805c5a0 0x7fe94805e9c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe948055060 msgr2=0x7fe948057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe948055060 0x7fe948057520 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fe9540090f0 tx=0x7fe954016000).stop 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 msgr2=0x7fe96c115450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 --2- 172.21.15.185:0/2300768609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe96c061e60 0x7fe96c115450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe95c00c6e0 tx=0x7fe95c007b50).stop 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe972b60700 1 -- 172.21.15.185:0/2300768609 reap_dead start 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 shutdown_connections 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 >> 172.21.15.185:0/2300768609 conn(0x7fe96c060130 msgr2=0x7fe96c0674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 shutdown_connections 2020-11-16T22:14:15.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.735+0000 7fe973361700 1 -- 172.21.15.185:0/2300768609 wait complete. 2020-11-16T22:14:15.737 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:15.737 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2020-11-16T22:14:15.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 Processor -- start 2020-11-16T22:14:15.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 -- start start 2020-11-16T22:14:15.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc0bc0b17e0 0x7fc0bc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc098cc0 con 0x7fc0bc0b2160 2020-11-16T22:14:15.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc0b3950 con 0x7fc0bc066240 2020-11-16T22:14:15.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0c0c8d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc14c4f0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0bad9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc0bc066240 0x7fc0bc065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49554/0 (socket says 172.21.15.185:49554) 2020-11-16T22:14:15.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0bad9d700 1 -- 172.21.15.185:0/421839245 learned_addr learned my addr 172.21.15.185:0/421839245 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:15.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.875+0000 7fc0ba59c700 1 --2- 172.21.15.185:0/421839245 >> v2:172.21.15.96:3300/0 conn(0x7fc0bc0b17e0 0x7fc0bc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0b959a700 1 -- 172.21.15.185:0/421839245 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416202716 0 0) 0x7fc0bc0b3950 con 0x7fc0bc066240 2020-11-16T22:14:15.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0b959a700 1 -- 172.21.15.185:0/421839245 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003240 con 0x7fc0bc066240 2020-11-16T22:14:15.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0ba59c700 1 -- 172.21.15.185:0/421839245 >> v1:172.21.15.185:6789/0 conn(0x7fc0bc066240 legacy=0x7fc0bc065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0ba59c700 1 -- 172.21.15.185:0/421839245 >> v1:172.21.15.96:6789/0 conn(0x7fc0bc0b2160 legacy=0x7fc0bc14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0ba59c700 1 -- 172.21.15.185:0/421839245 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0b3bf0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0ba59c700 1 --2- 172.21.15.185:0/421839245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc067fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc0ac00b9a0 tx=0x7fc0ac006df0).ready entity=mon.0 client_cookie=bc94b7c11d07f110 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0b959a700 1 -- 172.21.15.185:0/421839245 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0097b0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0b959a700 1 -- 172.21.15.185:0/421839245 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac009910 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0b959a700 1 -- 172.21.15.185:0/421839245 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac009c90 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/421839245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc0b17e0 msgr2=0x7fc0bc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 --2- 172.21.15.185:0/421839245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc067fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc0ac00b9a0 tx=0x7fc0ac006df0).stop 2020-11-16T22:14:15.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/421839245 shutdown_connections 2020-11-16T22:14:15.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 --2- 172.21.15.185:0/421839245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc067fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/421839245 >> 172.21.15.185:0/421839245 conn(0x7fc0bc0604a0 msgr2=0x7fc0bc0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/421839245 shutdown_connections 2020-11-16T22:14:15.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/421839245 wait complete. 2020-11-16T22:14:15.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 Processor -- start 2020-11-16T22:14:15.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.879+0000 7fc0c0c8d700 1 -- start start 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc066240 0x7fc0bc158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc08ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0bc0b2160 0x7fc0bc095550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc158a20 con 0x7fc0bc066240 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0acfd0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0ad270 con 0x7fc0bc0b2160 2020-11-16T22:14:15.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc08ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc08ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56122/0 (socket says 172.21.15.185:56122) 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 -- 172.21.15.185:0/260970626 learned_addr learned my addr 172.21.15.185:0/260970626 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0bad9d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0bc0b2160 0x7fc0bc095550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0ba59c700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc066240 0x7fc0bc158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 -- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0bc0b2160 msgr2=0x7fc0bc095550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0bc0b2160 0x7fc0bc095550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 -- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc066240 msgr2=0x7fc0bc158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0bc066240 0x7fc0bc158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0ad510 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b9d9b700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc08ea20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc0a400c6e0 tx=0x7fc0a4007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4007020 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc0ad7b0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4007180 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40064f0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc14c2d0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.883+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0bc05d4e0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc0a4010070 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc098055060 0x7fc098057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc0a406da30 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0ba59c700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc098055060 0x7fc098057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x7fc09805c520 0x7fc09805e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:15.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc09805ef90 con 0x7fc09805c520 2020-11-16T22:14:15.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0a4001ea0 con 0x7fc0bc0b17e0 2020-11-16T22:14:15.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0ba59c700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc098055060 0x7fc098057520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc0ac006e30 tx=0x7fc0ac015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0bad9d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x7fc09805c520 0x7fc09805e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:15.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.887+0000 7fc0bad9d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x7fc09805c520 0x7fc09805e940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:15.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.891+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== osd.6 v2:172.21.15.96:6808/15253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fc09805ef90 con 0x7fc09805c520 2020-11-16T22:14:15.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 --> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0bc0ae570 con 0x7fc09805c520 2020-11-16T22:14:15.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0b27fc700 1 -- 172.21.15.185:0/260970626 <== osd.6 v2:172.21.15.96:6808/15253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0b4006e20 con 0x7fc09805c520 2020-11-16T22:14:15.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x7fc09805c520 msgr2=0x7fc09805e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x7fc09805c520 0x7fc09805e940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:15.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc098055060 msgr2=0x7fc098057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc098055060 0x7fc098057520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc0ac006e30 tx=0x7fc0ac015000).stop 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 msgr2=0x7fc0bc08ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 --2- 172.21.15.185:0/260970626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0bc0b17e0 0x7fc0bc08ea20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc0a400c6e0 tx=0x7fc0a4007c50).stop 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0bad9d700 1 -- 172.21.15.185:0/260970626 reap_dead start 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 shutdown_connections 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 >> 172.21.15.185:0/260970626 conn(0x7fc0bc0604a0 msgr2=0x7fc0bc14e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:15.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 shutdown_connections 2020-11-16T22:14:15.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:15.911+0000 7fc0c0c8d700 1 -- 172.21.15.185:0/260970626 wait complete. 2020-11-16T22:14:15.912 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:15.912 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2020-11-16T22:14:16.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.047+0000 7f4153fff700 1 Processor -- start 2020-11-16T22:14:16.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.047+0000 7f4153fff700 1 -- start start 2020-11-16T22:14:16.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 0x7f41540b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4154066ad0 con 0x7f415413e720 2020-11-16T22:14:16.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4154140d10 con 0x7f41540b7470 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4154144d90 con 0x7f41540b2920 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4152ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 0x7f41540b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49576/0 (socket says 172.21.15.185:49576) 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 -- 172.21.15.185:0/1847506492 learned_addr learned my addr 172.21.15.185:0/1847506492 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4152ffd700 1 --2- 172.21.15.185:0/1847506492 >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 0x7f41540b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56130/0 (socket says 172.21.15.185:56130) 2020-11-16T22:14:16.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 -- 172.21.15.185:0/1847506492 >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 msgr2=0x7f41540b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 --2- 172.21.15.185:0/1847506492 >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 0x7f41540b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 -- 172.21.15.185:0/1847506492 >> v1:172.21.15.96:6789/0 conn(0x7f415413e720 legacy=0x7f4154141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 -- 172.21.15.185:0/1847506492 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154144ff0 con 0x7f41540b2920 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f414a7fc700 1 --2- 172.21.15.185:0/1847506492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f413c006fd0 tx=0x7f413c006eb0).ready entity=mon.2 client_cookie=aff8dcd6b6f45b32 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f41527fc700 1 -- 172.21.15.185:0/1847506492 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0098a0 con 0x7f41540b2920 2020-11-16T22:14:16.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f41527fc700 1 -- 172.21.15.185:0/1847506492 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c009a00 con 0x7f41540b2920 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f41527fc700 1 -- 172.21.15.185:0/1847506492 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c009d80 con 0x7f41540b2920 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- 172.21.15.185:0/1847506492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 msgr2=0x7f415413e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 --2- 172.21.15.185:0/1847506492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f413c006fd0 tx=0x7f413c006eb0).stop 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- 172.21.15.185:0/1847506492 shutdown_connections 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 --2- 172.21.15.185:0/1847506492 >> v2:172.21.15.185:3300/0 conn(0x7f41540b7470 0x7f41540b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 --2- 172.21.15.185:0/1847506492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b2920 0x7f415413e230 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- 172.21.15.185:0/1847506492 >> 172.21.15.185:0/1847506492 conn(0x7f41540637b0 msgr2=0x7f41540b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- 172.21.15.185:0/1847506492 shutdown_connections 2020-11-16T22:14:16.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.051+0000 7f4153fff700 1 -- 172.21.15.185:0/1847506492 wait complete. 2020-11-16T22:14:16.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 Processor -- start 2020-11-16T22:14:16.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 -- start start 2020-11-16T22:14:16.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41540b2920 0x7f4154153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 0x7f415413f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 0x7f415413b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4154153ef0 con 0x7f41540b2920 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4154154190 con 0x7f415413e720 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4153fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41541483e0 con 0x7f41540b7470 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f414a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 0x7f415413f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f41537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 0x7f415413b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f4152ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41540b2920 0x7f4154153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f41537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 0x7f415413b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56140/0 (socket says 172.21.15.185:56140) 2020-11-16T22:14:16.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f414a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 0x7f415413f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49582/0 (socket says 172.21.15.185:49582) 2020-11-16T22:14:16.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.055+0000 7f41537fe700 1 -- 172.21.15.185:0/3985382460 learned_addr learned my addr 172.21.15.185:0/3985382460 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 -- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 msgr2=0x7f415413f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 0x7f415413f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 -- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41540b2920 msgr2=0x7f4154153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41540b2920 0x7f4154153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154153c00 con 0x7f415413e720 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414a7fc700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f41540b7470 0x7f415413f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f41537fe700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 0x7f415413b5d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f414c00c710 tx=0x7f414c007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c013410 con 0x7f415413e720 2020-11-16T22:14:16.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c013570 con 0x7f415413e720 2020-11-16T22:14:16.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0138f0 con 0x7f415413e720 2020-11-16T22:14:16.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415413fa70 con 0x7f415413e720 2020-11-16T22:14:16.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4154140400 con 0x7f415413e720 2020-11-16T22:14:16.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f415405d200 con 0x7f415413e720 2020-11-16T22:14:16.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f414c010030 con 0x7f415413e720 2020-11-16T22:14:16.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4130055060 0x7f4130057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f414c06e1d0 con 0x7f415413e720 2020-11-16T22:14:16.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f4152ffd700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4130055060 0x7f4130057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x7f413005c5a0 0x7f413005e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f413005f010 con 0x7f413005c5a0 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f414c06e5d0 con 0x7f415413e720 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.059+0000 7f4152ffd700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4130055060 0x7f4130057520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f414400bae0 tx=0x7f414400b470).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.063+0000 7f414a7fc700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x7f413005c5a0 0x7f413005e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.063+0000 7f414a7fc700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x7f413005c5a0 0x7f413005e9c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.063+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== osd.7 v2:172.21.15.96:6811/15252 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f413005f010 con 0x7f413005c5a0 2020-11-16T22:14:16.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4154066f70 con 0x7f413005c5a0 2020-11-16T22:14:16.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f414b7fe700 1 -- 172.21.15.185:0/3985382460 <== osd.7 v2:172.21.15.96:6811/15252 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4154066f70 con 0x7f413005c5a0 2020-11-16T22:14:16.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x7f413005c5a0 msgr2=0x7f413005e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x7f413005c5a0 0x7f413005e9c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4130055060 msgr2=0x7f4130057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4130055060 0x7f4130057520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f414400bae0 tx=0x7f414400b470).stop 2020-11-16T22:14:16.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 msgr2=0x7f415413b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f4153fff700 1 --2- 172.21.15.185:0/3985382460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f415413e720 0x7f415413b5d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f414c00c710 tx=0x7f414c007b30).stop 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.083+0000 7f41537fe700 1 -- 172.21.15.185:0/3985382460 reap_dead start 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.087+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 shutdown_connections 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.087+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 >> 172.21.15.185:0/3985382460 conn(0x7f41540637b0 msgr2=0x7f415413df10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.087+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 shutdown_connections 2020-11-16T22:14:16.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.087+0000 7f4153fff700 1 -- 172.21.15.185:0/3985382460 wait complete. 2020-11-16T22:14:16.086 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:16.086 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T22:14:16.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.219+0000 7f7b4ffff700 1 Processor -- start 2020-11-16T22:14:16.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 -- start start 2020-11-16T22:14:16.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7b500ad3a0 0x7f7b50147560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 0x7f7b50144310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b500b70b0 con 0x7f7b500b74c0 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b500b6b70 con 0x7f7b500b6430 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4ffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b500b6e10 con 0x7f7b500ad3a0 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4effd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7b500b74c0 0x7f7b500adf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49584/0 (socket says 172.21.15.185:49584) 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4effd700 1 -- 172.21.15.185:0/1363263175 learned_addr learned my addr 172.21.15.185:0/1363263175 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/1363263175 >> v2:172.21.15.185:3300/0 conn(0x7f7b500ad3a0 0x7f7b50147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4f7fe700 1 --2- 172.21.15.185:0/1363263175 >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 0x7f7b50144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4dffb700 1 -- 172.21.15.185:0/1363263175 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200857013 0 0) 0x7f7b500b70b0 con 0x7f7b500b74c0 2020-11-16T22:14:16.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4dffb700 1 -- 172.21.15.185:0/1363263175 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b30003240 con 0x7f7b500b74c0 2020-11-16T22:14:16.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 -- 172.21.15.185:0/1363263175 >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 msgr2=0x7f7b50144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/1363263175 >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 0x7f7b50144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 -- 172.21.15.185:0/1363263175 >> v1:172.21.15.185:6789/0 conn(0x7f7b500b74c0 legacy=0x7f7b500adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 -- 172.21.15.185:0/1363263175 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500adaa0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4f7fe700 1 --2- 172.21.15.185:0/1363263175 >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 0x7f7b50144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/1363263175 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500ad3a0 0x7f7b50147560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7b50060330 tx=0x7f7b38006df0).ready entity=mon.1 client_cookie=d9ec220ad3b0e834 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4dffb700 1 -- 172.21.15.185:0/1363263175 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b380096d0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4dffb700 1 -- 172.21.15.185:0/1363263175 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b38009830 con 0x7f7b500ad3a0 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.223+0000 7f7b4dffb700 1 -- 172.21.15.185:0/1363263175 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38009bb0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- 172.21.15.185:0/1363263175 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500ad3a0 msgr2=0x7f7b50147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/1363263175 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500ad3a0 0x7f7b50147560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7b50060330 tx=0x7f7b38006df0).stop 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- 172.21.15.185:0/1363263175 shutdown_connections 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/1363263175 >> v2:172.21.15.96:3300/0 conn(0x7f7b500b6430 0x7f7b50144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/1363263175 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500ad3a0 0x7f7b50147560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- 172.21.15.185:0/1363263175 >> 172.21.15.185:0/1363263175 conn(0x7f7b50067860 msgr2=0x7f7b50067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- 172.21.15.185:0/1363263175 shutdown_connections 2020-11-16T22:14:16.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- 172.21.15.185:0/1363263175 wait complete. 2020-11-16T22:14:16.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 Processor -- start 2020-11-16T22:14:16.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.227+0000 7f7b4ffff700 1 -- start start 2020-11-16T22:14:16.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500b6430 0x7f7b500b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b500b74c0 0x7f7b50153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500b3010 con 0x7f7b500b74c0 2020-11-16T22:14:16.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500b4380 con 0x7f7b500b6430 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500b4620 con 0x7f7b500ad3a0 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49596/0 (socket says 172.21.15.185:49596) 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 -- 172.21.15.185:0/2729109032 learned_addr learned my addr 172.21.15.185:0/2729109032 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500b6430 0x7f7b500b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4f7fe700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b500b74c0 0x7f7b50153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 -- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500b6430 msgr2=0x7f7b500b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500b6430 0x7f7b500b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 -- 172.21.15.185:0/2729109032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b500b74c0 msgr2=0x7f7b50153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b500b74c0 0x7f7b50153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 -- 172.21.15.185:0/2729109032 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500b48c0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4effd700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7b4800cae0 tx=0x7f7b48007cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48007e70 con 0x7f7b500ad3a0 2020-11-16T22:14:16.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b48007320 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480076a0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b500b2c90 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b50153500 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b500b2e80 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.231+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7b48010030 con 0x7f7b500ad3a0 2020-11-16T22:14:16.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.235+0000 7f7b46ffd700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 0x7f7b2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.235+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b4806dde0 con 0x7f7b500ad3a0 2020-11-16T22:14:16.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.235+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 0x7f7b2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.235+0000 7f7b4e7fc700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 0x7f7b2c057520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f7b3800b150 tx=0x7f7b38015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.239+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7b48044020 con 0x7f7b500ad3a0 2020-11-16T22:14:16.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.387+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7b5005d200 con 0x7f7b500ad3a0 2020-11-16T22:14:16.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.387+0000 7f7b46ffd700 1 -- 172.21.15.185:0/2729109032 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b48047640 con 0x7f7b500ad3a0 2020-11-16T22:14:16.376 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:16.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 msgr2=0x7f7b2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 0x7f7b2c057520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f7b3800b150 tx=0x7f7b38015000).stop 2020-11-16T22:14:16.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 msgr2=0x7f7b500b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7b4800cae0 tx=0x7f7b48007cb0).stop 2020-11-16T22:14:16.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 shutdown_connections 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b2c055060 0x7f7b2c057520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b500b74c0 0x7f7b50153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b500ad3a0 0x7f7b500b37b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 --2- 172.21.15.185:0/2729109032 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b500b6430 0x7f7b500b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 >> 172.21.15.185:0/2729109032 conn(0x7f7b50067860 msgr2=0x7f7b500ba070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 shutdown_connections 2020-11-16T22:14:16.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.391+0000 7f7b4ffff700 1 -- 172.21.15.185:0/2729109032 wait complete. 2020-11-16T22:14:16.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2020-11-16T22:14:16.391 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-16T22:14:16.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 Processor -- start 2020-11-16T22:14:16.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 -- start start 2020-11-16T22:14:16.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f100b4d40 0x7f7f10068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7f101418d0 0x7f7f1013de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100b7e10 con 0x7f7f100675b0 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100b7670 con 0x7f7f101418d0 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f161b2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100b78d0 con 0x7f7f100b4d40 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f100b4d40 0x7f7f10068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f100b4d40 0x7f7f10068460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56162/0 (socket says 172.21.15.185:56162) 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 -- 172.21.15.185:0/3764268936 learned_addr learned my addr 172.21.15.185:0/3764268936 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f159b1700 1 --2- 172.21.15.185:0/3764268936 >> v2:172.21.15.96:3300/0 conn(0x7f7f101418d0 0x7f7f1013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 -- 172.21.15.185:0/3764268936 >> v2:172.21.15.96:3300/0 conn(0x7f7f101418d0 msgr2=0x7f7f1013de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:16.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 --2- 172.21.15.185:0/3764268936 >> v2:172.21.15.96:3300/0 conn(0x7f7f101418d0 0x7f7f1013de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 -- 172.21.15.185:0/3764268936 >> v1:172.21.15.96:6789/0 conn(0x7f7f100675b0 legacy=0x7f7f100b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 -- 172.21.15.185:0/3764268936 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100b7b70 con 0x7f7f100b4d40 2020-11-16T22:14:16.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.527+0000 7f7f149af700 1 --2- 172.21.15.185:0/3764268936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10068460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7f10065f60 tx=0x7f7f04006eb0).ready entity=mon.1 client_cookie=4eed3471b26894dd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f03fff700 1 -- 172.21.15.185:0/3764268936 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04009810 con 0x7f7f100b4d40 2020-11-16T22:14:16.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f03fff700 1 -- 172.21.15.185:0/3764268936 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04009970 con 0x7f7f100b4d40 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f03fff700 1 -- 172.21.15.185:0/3764268936 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04009cf0 con 0x7f7f100b4d40 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- 172.21.15.185:0/3764268936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 msgr2=0x7f7f10068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- 172.21.15.185:0/3764268936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10068460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7f10065f60 tx=0x7f7f04006eb0).stop 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- 172.21.15.185:0/3764268936 shutdown_connections 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- 172.21.15.185:0/3764268936 >> v2:172.21.15.96:3300/0 conn(0x7f7f101418d0 0x7f7f1013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- 172.21.15.185:0/3764268936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10068460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- 172.21.15.185:0/3764268936 >> 172.21.15.185:0/3764268936 conn(0x7f7f1005ba80 msgr2=0x7f7f1005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- 172.21.15.185:0/3764268936 shutdown_connections 2020-11-16T22:14:16.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- 172.21.15.185:0/3764268936 wait complete. 2020-11-16T22:14:16.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 Processor -- start 2020-11-16T22:14:16.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- start start 2020-11-16T22:14:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 0x7f7f1013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101418d0 0x7f7f10144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100afe50 con 0x7f7f101418d0 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100af910 con 0x7f7f100b4d40 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f161b2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100afbb0 con 0x7f7f100675b0 2020-11-16T22:14:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f149af700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 0x7f7f1013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f149af700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56168/0 (socket says 172.21.15.185:56168) 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f149af700 1 -- 172.21.15.185:0/1670949889 learned_addr learned my addr 172.21.15.185:0/1670949889 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f159b1700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101418d0 0x7f7f10144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 -- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 msgr2=0x7f7f10144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 -- 172.21.15.185:0/1670949889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101418d0 msgr2=0x7f7f10144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101418d0 0x7f7f10144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.531+0000 7f7f151b0700 1 -- 172.21.15.185:0/1670949889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1013e260 con 0x7f7f100675b0 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f151b0700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 0x7f7f1013dd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7f0c00c740 tx=0x7f7f0c007b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c007020 con 0x7f7f100675b0 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f101450e0 con 0x7f7f100675b0 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1013fbd0 con 0x7f7f100675b0 2020-11-16T22:14:16.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c007180 con 0x7f7f100675b0 2020-11-16T22:14:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0064f0 con 0x7f7f100675b0 2020-11-16T22:14:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f100b1650 con 0x7f7f100675b0 2020-11-16T22:14:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7f0c010030 con 0x7f7f100675b0 2020-11-16T22:14:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 0x7f7ee8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f0c06d620 con 0x7f7f100675b0 2020-11-16T22:14:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.535+0000 7f7f149af700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 0x7f7ee8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.539+0000 7f7f149af700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 0x7f7ee8057520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7f100b0bb0 tx=0x7f7f04015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.543+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7f0c043770 con 0x7f7f100675b0 2020-11-16T22:14:16.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f1005d200 con 0x7f7f100675b0 2020-11-16T22:14:16.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f017fa700 1 -- 172.21.15.185:0/1670949889 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f0c046d90 con 0x7f7f100675b0 2020-11-16T22:14:16.682 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:16.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 msgr2=0x7f7ee8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 0x7f7ee8057520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7f100b0bb0 tx=0x7f7f04015000).stop 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 msgr2=0x7f7f1013dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 0x7f7f1013dd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7f0c00c740 tx=0x7f7f0c007b00).stop 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 shutdown_connections 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ee8055060 0x7f7ee8057520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f101418d0 0x7f7f10144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f100675b0 0x7f7f1013dd70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 --2- 172.21.15.185:0/1670949889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f100b4d40 0x7f7f10144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 >> 172.21.15.185:0/1670949889 conn(0x7f7f1005ba80 msgr2=0x7f7f100b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 shutdown_connections 2020-11-16T22:14:16.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.695+0000 7f7f161b2700 1 -- 172.21.15.185:0/1670949889 wait complete. 2020-11-16T22:14:16.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2020-11-16T22:14:16.697 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:14:16.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 Processor -- start 2020-11-16T22:14:16.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 -- start start 2020-11-16T22:14:16.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2404061e60 0x7f24041437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24040abf90 con 0x7f24040669a0 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24040ac230 con 0x7f2404061e60 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2409b4c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040ac4d0 con 0x7f24040acc10 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2403fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f240934b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2404061e60 0x7f24041437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2403fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49618/0 (socket says 172.21.15.185:49618) 2020-11-16T22:14:16.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.831+0000 7f2403fff700 1 -- 172.21.15.185:0/3287498887 learned_addr learned my addr 172.21.15.185:0/3287498887 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2403fff700 1 -- 172.21.15.185:0/3287498887 >> v2:172.21.15.185:3300/0 conn(0x7f2404061e60 msgr2=0x7f24041437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:16.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2403fff700 1 --2- 172.21.15.185:0/3287498887 >> v2:172.21.15.185:3300/0 conn(0x7f2404061e60 0x7f24041437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2403fff700 1 -- 172.21.15.185:0/3287498887 >> v1:172.21.15.185:6789/0 conn(0x7f24040669a0 legacy=0x7f2404066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:16.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2403fff700 1 -- 172.21.15.185:0/3287498887 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24040ac770 con 0x7f24040acc10 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2403fff700 1 --2- 172.21.15.185:0/3287498887 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f23f8006fd0 tx=0x7f23f8006eb0).ready entity=mon.2 client_cookie=fafe0c888dde533 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f24037fe700 1 -- 172.21.15.185:0/3287498887 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80097d0 con 0x7f24040acc10 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f24037fe700 1 -- 172.21.15.185:0/3287498887 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8009930 con 0x7f24040acc10 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f24037fe700 1 -- 172.21.15.185:0/3287498887 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8009cb0 con 0x7f24040acc10 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- 172.21.15.185:0/3287498887 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 msgr2=0x7f24041432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3287498887 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f23f8006fd0 tx=0x7f23f8006eb0).stop 2020-11-16T22:14:16.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- 172.21.15.185:0/3287498887 shutdown_connections 2020-11-16T22:14:16.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3287498887 >> v2:172.21.15.185:3300/0 conn(0x7f2404061e60 0x7f24041437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3287498887 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f24040acc10 0x7f24041432e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- 172.21.15.185:0/3287498887 >> 172.21.15.185:0/3287498887 conn(0x7f2404060130 msgr2=0x7f2404060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- 172.21.15.185:0/3287498887 shutdown_connections 2020-11-16T22:14:16.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- 172.21.15.185:0/3287498887 wait complete. 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 Processor -- start 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- start start 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 0x7f240413fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24040b0470 0x7f2404141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2404140d90 con 0x7f24040b0470 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040b6750 con 0x7f24040669a0 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.835+0000 7f2409b4c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040b64b0 con 0x7f2404061e60 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49624/0 (socket says 172.21.15.185:49624) 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 -- 172.21.15.185:0/3040384997 learned_addr learned my addr 172.21.15.185:0/3040384997 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:16.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2403fff700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 0x7f240413fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f240934b700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24040b0470 0x7f2404141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 -- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 msgr2=0x7f240413fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 0x7f240413fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 -- 172.21.15.185:0/3040384997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24040b0470 msgr2=0x7f2404141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24040b0470 0x7f2404141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 -- 172.21.15.185:0/3040384997 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24041543c0 con 0x7f2404061e60 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2403fff700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 0x7f240413fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:16.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2408b4a700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f23f400c760 tx=0x7f23f4007c30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400e490 con 0x7f2404061e60 2020-11-16T22:14:16.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f400e610 con 0x7f2404061e60 2020-11-16T22:14:16.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2404154660 con 0x7f2404061e60 2020-11-16T22:14:16.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400e9d0 con 0x7f2404061e60 2020-11-16T22:14:16.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2404142300 con 0x7f2404061e60 2020-11-16T22:14:16.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24041547f0 con 0x7f2404061e60 2020-11-16T22:14:16.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f23f4010030 con 0x7f2404061e60 2020-11-16T22:14:16.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 0x7f23dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:16.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f23f406db80 con 0x7f2404061e60 2020-11-16T22:14:16.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2403fff700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 0x7f23dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:16.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.839+0000 7f2403fff700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 0x7f23dc057520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f23f800beb0 tx=0x7f23f8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:16.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.847+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f24041547f0 con 0x7f2404061e60 2020-11-16T22:14:16.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24041547f0 con 0x7f2404061e60 2020-11-16T22:14:16.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2400ff9700 1 -- 172.21.15.185:0/3040384997 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23f4043de0 con 0x7f2404061e60 2020-11-16T22:14:16.986 INFO:teuthology.orchestra.run.smithi185.stdout:55834574853 2020-11-16T22:14:16.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 msgr2=0x7f23dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 0x7f23dc057520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f23f800beb0 tx=0x7f23f8015000).stop 2020-11-16T22:14:16.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 msgr2=0x7f24040aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:16.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f23f400c760 tx=0x7f23f4007c30).stop 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 shutdown_connections 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f23dc055060 0x7f23dc057520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24040b0470 0x7f2404141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2404061e60 0x7f24040aed30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 --2- 172.21.15.185:0/3040384997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f24040669a0 0x7f240413fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 >> 172.21.15.185:0/3040384997 conn(0x7f2404060130 msgr2=0x7f24040b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:16.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 shutdown_connections 2020-11-16T22:14:16.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:16.999+0000 7f2409b4c700 1 -- 172.21.15.185:0/3040384997 wait complete. 2020-11-16T22:14:17.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2020-11-16T22:14:18.001 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T22:14:18.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 Processor -- start 2020-11-16T22:14:18.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 -- start start 2020-11-16T22:14:18.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c5c0b3140 0x7f8c5c1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c5c144dc0 0x7f8c5c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c065cf0 0x7f8c5c0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5c0663b0 con 0x7f8c5c144dc0 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5c140fd0 con 0x7f8c5c0b3140 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6216d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c141270 con 0x7f8c5c065cf0 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c5c0b3140 0x7f8c5c1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c5c0b3140 0x7f8c5c1419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56188/0 (socket says 172.21.15.185:56188) 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 -- 172.21.15.185:0/686883908 learned_addr learned my addr 172.21.15.185:0/686883908 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:18.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6196c700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c065cf0 0x7f8c5c0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6096a700 1 --2- 172.21.15.185:0/686883908 >> v2:172.21.15.96:3300/0 conn(0x7f8c5c144dc0 0x7f8c5c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 -- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c065cf0 msgr2=0x7f8c5c0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c065cf0 0x7f8c5c0b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 -- 172.21.15.185:0/686883908 >> v2:172.21.15.96:3300/0 conn(0x7f8c5c144dc0 msgr2=0x7f8c5c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 --2- 172.21.15.185:0/686883908 >> v2:172.21.15.96:3300/0 conn(0x7f8c5c144dc0 0x7f8c5c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 -- 172.21.15.185:0/686883908 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c141510 con 0x7f8c5c0b3140 2020-11-16T22:14:18.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c6116b700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c0b3140 0x7f8c5c1419b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8c5c0b7510 tx=0x7f8c58006eb0).ready entity=mon.1 client_cookie=cfa338a78b2f7c64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c4bfff700 1 -- 172.21.15.185:0/686883908 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009720 con 0x7f8c5c0b3140 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c4bfff700 1 -- 172.21.15.185:0/686883908 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58009880 con 0x7f8c5c0b3140 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.151+0000 7f8c4bfff700 1 -- 172.21.15.185:0/686883908 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009c00 con 0x7f8c5c0b3140 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c0b3140 msgr2=0x7f8c5c1419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c0b3140 0x7f8c5c1419b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f8c5c0b7510 tx=0x7f8c58006eb0).stop 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- 172.21.15.185:0/686883908 shutdown_connections 2020-11-16T22:14:18.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c065cf0 0x7f8c5c0b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- 172.21.15.185:0/686883908 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c0b3140 0x7f8c5c1419b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- 172.21.15.185:0/686883908 >> v2:172.21.15.96:3300/0 conn(0x7f8c5c144dc0 0x7f8c5c0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- 172.21.15.185:0/686883908 >> 172.21.15.185:0/686883908 conn(0x7f8c5c0666d0 msgr2=0x7f8c5c066ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:18.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- 172.21.15.185:0/686883908 shutdown_connections 2020-11-16T22:14:18.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- 172.21.15.185:0/686883908 wait complete. 2020-11-16T22:14:18.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 Processor -- start 2020-11-16T22:14:18.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- start start 2020-11-16T22:14:18.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 0x7f8c5c13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0ac5f0 con 0x7f8c5c065cf0 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0ac890 con 0x7f8c5c138980 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6216d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0acb30 con 0x7f8c5c144dc0 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6096a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6096a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49640/0 (socket says 172.21.15.185:49640) 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56198/0 (socket says 172.21.15.185:56198) 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6096a700 1 -- 172.21.15.185:0/1914333879 learned_addr learned my addr 172.21.15.185:0/1914333879 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6116b700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 0x7f8c5c13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 -- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 msgr2=0x7f8c5c138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 -- 172.21.15.185:0/1914333879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 msgr2=0x7f8c5c13ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 0x7f8c5c13ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 -- 172.21.15.185:0/1914333879 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0acdd0 con 0x7f8c5c138980 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6116b700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 0x7f8c5c13ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6096a700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.155+0000 7f8c6196c700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f8c5400ed20 tx=0x7f8c540091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400eee0 con 0x7f8c5c138980 2020-11-16T22:14:18.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5400cbb0 con 0x7f8c5c138980 2020-11-16T22:14:18.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c13e2e0 con 0x7f8c5c138980 2020-11-16T22:14:18.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540123f0 con 0x7f8c5c138980 2020-11-16T22:14:18.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c1376c0 con 0x7f8c5c138980 2020-11-16T22:14:18.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8c54022070 con 0x7f8c5c138980 2020-11-16T22:14:18.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 0x7f8c34057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c05d200 con 0x7f8c5c138980 2020-11-16T22:14:18.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c5406e450 con 0x7f8c5c138980 2020-11-16T22:14:18.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c6116b700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 0x7f8c34057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.159+0000 7f8c6116b700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 0x7f8c34057550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8c5c067aa0 tx=0x7f8c58015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.167+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8c540480e0 con 0x7f8c5c138980 2020-11-16T22:14:18.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.319+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c5c0b2490 con 0x7f8c5c138980 2020-11-16T22:14:18.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.319+0000 7f8c497fa700 1 -- 172.21.15.185:0/1914333879 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c54016070 con 0x7f8c5c138980 2020-11-16T22:14:18.308 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:18.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 msgr2=0x7f8c34057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 0x7f8c34057550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8c5c067aa0 tx=0x7f8c58015000).stop 2020-11-16T22:14:18.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 msgr2=0x7f8c5c1371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f8c5400ed20 tx=0x7f8c540091d0).stop 2020-11-16T22:14:18.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 shutdown_connections 2020-11-16T22:14:18.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c34055090 0x7f8c34057550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c5c138980 0x7f8c5c1371d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c065cf0 0x7f8c5c13ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 --2- 172.21.15.185:0/1914333879 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c5c144dc0 0x7f8c5c138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 >> 172.21.15.185:0/1914333879 conn(0x7f8c5c0666d0 msgr2=0x7f8c5c066ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 shutdown_connections 2020-11-16T22:14:18.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.323+0000 7f8c6216d700 1 -- 172.21.15.185:0/1914333879 wait complete. 2020-11-16T22:14:18.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2020-11-16T22:14:18.323 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2020-11-16T22:14:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.455+0000 7fa0d3a27700 1 Processor -- start 2020-11-16T22:14:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.455+0000 7fa0d3a27700 1 -- start start 2020-11-16T22:14:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09d260 0x7fa0cc0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc1444d0 con 0x7fa0cc0b5b30 2020-11-16T22:14:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc144770 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc144a10 con 0x7fa0cc09d260 2020-11-16T22:14:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d1fc4700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa0cc0b6b30 0x7fa0cc09a070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49644/0 (socket says 172.21.15.185:49644) 2020-11-16T22:14:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d1fc4700 1 -- 172.21.15.185:0/1825486787 learned_addr learned my addr 172.21.15.185:0/1825486787 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/1825486787 >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09d260 0x7fa0cc0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518354155 0 0) 0x7fa0cc144770 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b8003240 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1998015847 0 0) 0x7fa0b8003240 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09d260 msgr2=0x7fa0cc0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:18.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 --2- 172.21.15.185:0/1825486787 >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09d260 0x7fa0cc0b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 >> v1:172.21.15.96:6789/0 conn(0x7fa0cc0b5b30 legacy=0x7fa0cc1437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc144cb0 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fa0c4001860 con 0x7fa0cc0b6b30 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 >> v1:172.21.15.185:6789/0 conn(0x7fa0cc0b6b30 legacy=0x7fa0cc09a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0b8003730 0x7fa0b8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 0x7fa0b8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b8004e50 0x7fa0b800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc144a10 con 0x7fa0b8004e50 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8004260 con 0x7fa0b8003730 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b800bc20 con 0x7fa0b80044e0 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 0x7fa0b8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d17c3700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0b8003730 0x7fa0b8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d1fc4700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b8004e50 0x7fa0b800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0b8003730 msgr2=0x7fa0b8003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0b8003730 0x7fa0b8003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/1825486787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b8004e50 msgr2=0x7fa0b800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b8004e50 0x7fa0b800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/1825486787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8003240 con 0x7fa0b80044e0 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d1fc4700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b8004e50 0x7fa0b800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d1fc4700 1 -- 172.21.15.185:0/1825486787 reap_dead start 2020-11-16T22:14:18.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 0x7fa0b8004960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa0c800bc30 tx=0x7fa0c8006df0).ready entity=mon.2 client_cookie=fcb1d1c909c45470 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800b300 con 0x7fa0b80044e0 2020-11-16T22:14:18.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800ea90 con 0x7fa0b80044e0 2020-11-16T22:14:18.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0c3fff700 1 -- 172.21.15.185:0/1825486787 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800ee10 con 0x7fa0b80044e0 2020-11-16T22:14:18.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 msgr2=0x7fa0b8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 0x7fa0b8004960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa0c800bc30 tx=0x7fa0c8006df0).stop 2020-11-16T22:14:18.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- 172.21.15.185:0/1825486787 shutdown_connections 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/1825486787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0b80044e0 0x7fa0b8004960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- 172.21.15.185:0/1825486787 >> 172.21.15.185:0/1825486787 conn(0x7fa0cc068330 msgr2=0x7fa0cc0ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- 172.21.15.185:0/1825486787 shutdown_connections 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.459+0000 7fa0d3a27700 1 -- 172.21.15.185:0/1825486787 wait complete. 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 Processor -- start 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- start start 2020-11-16T22:14:18.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 0x7fa0cc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0cc146f00 0x7fa0cc148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1486c0 con 0x7fa0b80044e0 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1459a0 con 0x7fa0cc146f00 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc145e70 con 0x7fa0cc153680 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49660/0 (socket says 172.21.15.185:49660) 2020-11-16T22:14:18.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/687794053 learned_addr learned my addr 172.21.15.185:0/687794053 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:18.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d1fc4700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0cc146f00 0x7fa0cc148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d17c3700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 0x7fa0cc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0cc146f00 msgr2=0x7fa0cc148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0cc146f00 0x7fa0cc148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/687794053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 msgr2=0x7fa0cc153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 0x7fa0cc153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 -- 172.21.15.185:0/687794053 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc146110 con 0x7fa0cc153680 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d17c3700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 0x7fa0cc153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d0fc2700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fa0b800bd60 tx=0x7fa0c8019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801d410 con 0x7fa0cc153680 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c801d570 con 0x7fa0cc153680 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc145620 con 0x7fa0cc153680 2020-11-16T22:14:18.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801d700 con 0x7fa0cc153680 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.463+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc0b4610 con 0x7fa0cc153680 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc05d4e0 con 0x7fa0cc153680 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa0c802d070 con 0x7fa0cc153680 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0c17fa700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0c806fc60 con 0x7fa0cc153680 2020-11-16T22:14:18.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0d17c3700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.467+0000 7fa0d17c3700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 0x7fa0a4057520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa0bc000f20 tx=0x7fa0bc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:18.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.471+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa0c8045e60 con 0x7fa0cc153680 2020-11-16T22:14:18.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa0cc145810 con 0x7fa0cc153680 2020-11-16T22:14:18.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0c17fa700 1 -- 172.21.15.185:0/687794053 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0c802b070 con 0x7fa0cc153680 2020-11-16T22:14:18.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 msgr2=0x7fa0a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 0x7fa0a4057520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa0bc000f20 tx=0x7fa0bc009040).stop 2020-11-16T22:14:18.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 msgr2=0x7fa0cc146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:18.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fa0b800bd60 tx=0x7fa0c8019000).stop 2020-11-16T22:14:18.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 shutdown_connections 2020-11-16T22:14:18.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0a4055060 0x7fa0a4057520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0cc146f00 0x7fa0cc148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b80044e0 0x7fa0cc153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 --2- 172.21.15.185:0/687794053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0cc153680 0x7fa0cc146a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:18.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 >> 172.21.15.185:0/687794053 conn(0x7fa0cc068330 msgr2=0x7fa0cc0ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:18.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 shutdown_connections 2020-11-16T22:14:18.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:18.623+0000 7fa0d3a27700 1 -- 172.21.15.185:0/687794053 wait complete. 2020-11-16T22:14:18.867 INFO:teuthology.orchestra.run.smithi185.stdout:55834574855 2020-11-16T22:14:18.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2020-11-16T22:14:18.868 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-16T22:14:18.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 Processor -- start 2020-11-16T22:14:18.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- start start 2020-11-16T22:14:18.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 0x7f895c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f895c060ab0 0x7f895c0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c060f40 con 0x7f895c0b7510 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c1420f0 con 0x7f895c060130 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c142390 con 0x7f895c060ab0 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f895c060ab0 0x7f895c0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f895c060ab0 0x7f895c0673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56226/0 (socket says 172.21.15.185:56226) 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 -- 172.21.15.185:0/943269264 learned_addr learned my addr 172.21.15.185:0/943269264 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:18.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895affd700 1 --2- 172.21.15.185:0/943269264 >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 0x7f895c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 -- 172.21.15.185:0/943269264 >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 msgr2=0x7f895c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 --2- 172.21.15.185:0/943269264 >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 0x7f895c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 -- 172.21.15.185:0/943269264 >> v1:172.21.15.96:6789/0 conn(0x7f895c0b7510 legacy=0x7f895c13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 -- 172.21.15.185:0/943269264 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c142630 con 0x7f895c060ab0 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895affd700 1 --2- 172.21.15.185:0/943269264 >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 0x7f895c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:19.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895bfff700 1 --2- 172.21.15.185:0/943269264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060ab0 0x7f895c0673d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f895c0605c0 tx=0x7f8954009290).ready entity=mon.1 client_cookie=468ef6e78db079e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895a7fc700 1 -- 172.21.15.185:0/943269264 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895400bbc0 con 0x7f895c060ab0 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895a7fc700 1 -- 172.21.15.185:0/943269264 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895400bd20 con 0x7f895c060ab0 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f895a7fc700 1 -- 172.21.15.185:0/943269264 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954011610 con 0x7f895c060ab0 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- 172.21.15.185:0/943269264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060ab0 msgr2=0x7f895c0673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 --2- 172.21.15.185:0/943269264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060ab0 0x7f895c0673d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f895c0605c0 tx=0x7f8954009290).stop 2020-11-16T22:14:19.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- 172.21.15.185:0/943269264 shutdown_connections 2020-11-16T22:14:19.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 --2- 172.21.15.185:0/943269264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060ab0 0x7f895c0673d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 --2- 172.21.15.185:0/943269264 >> v2:172.21.15.96:3300/0 conn(0x7f895c060130 0x7f895c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- 172.21.15.185:0/943269264 >> 172.21.15.185:0/943269264 conn(0x7f895c05ba80 msgr2=0x7f895c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:19.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- 172.21.15.185:0/943269264 shutdown_connections 2020-11-16T22:14:19.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.011+0000 7f8960f5b700 1 -- 172.21.15.185:0/943269264 wait complete. 2020-11-16T22:14:19.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 Processor -- start 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- start start 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c060ab0 0x7f895c13e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b7510 0x7f895c13e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c13d0e0 con 0x7f895c060ab0 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c13dbf0 con 0x7f895c060130 2020-11-16T22:14:19.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c13de90 con 0x7f895c0b7510 2020-11-16T22:14:19.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56230/0 (socket says 172.21.15.185:56230) 2020-11-16T22:14:19.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 -- 172.21.15.185:0/2879655227 learned_addr learned my addr 172.21.15.185:0/2879655227 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:19.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895bfff700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b7510 0x7f895c13e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895affd700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c060ab0 0x7f895c13e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 -- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b7510 msgr2=0x7f895c13e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b7510 0x7f895c13e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 -- 172.21.15.185:0/2879655227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c060ab0 msgr2=0x7f895c13e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c060ab0 0x7f895c13e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 -- 172.21.15.185:0/2879655227 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0acd80 con 0x7f895c060130 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f895b7fe700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f894c00c6e0 tx=0x7f894c007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c007020 con 0x7f895c060130 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c007180 con 0x7f895c060130 2020-11-16T22:14:19.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c141250 con 0x7f895c060130 2020-11-16T22:14:19.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0064f0 con 0x7f895c060130 2020-11-16T22:14:19.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.015+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0b1930 con 0x7f895c060130 2020-11-16T22:14:19.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c05d200 con 0x7f895c060130 2020-11-16T22:14:19.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f894c010070 con 0x7f895c060130 2020-11-16T22:14:19.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f89537fe700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f894c06d050 con 0x7f895c060130 2020-11-16T22:14:19.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f895affd700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.019+0000 7f895affd700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f895c0b55c0 tx=0x7f8944008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.023+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f894c0432b0 con 0x7f895c060130 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.175+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f895c0b3970 con 0x7f895c060130 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.175+0000 7f89537fe700 1 -- 172.21.15.185:0/2879655227 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f894c0468d0 con 0x7f895c060130 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 msgr2=0x7f8938057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f895c0b55c0 tx=0x7f8944008040).stop 2020-11-16T22:14:19.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 msgr2=0x7f895c0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f894c00c6e0 tx=0x7f894c007c50).stop 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 shutdown_connections 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c060130 0x7f895c0ac890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b7510 0x7f895c13e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 --2- 172.21.15.185:0/2879655227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c060ab0 0x7f895c13e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 >> 172.21.15.185:0/2879655227 conn(0x7f895c05ba80 msgr2=0x7f895c067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:19.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 shutdown_connections 2020-11-16T22:14:19.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.179+0000 7f8960f5b700 1 -- 172.21.15.185:0/2879655227 wait complete. 2020-11-16T22:14:19.217 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:19.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2020-11-16T22:14:19.218 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2020-11-16T22:14:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 Processor -- start 2020-11-16T22:14:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- start start 2020-11-16T22:14:19.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 0x7f8f80147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f80061e60 0x7f8f80147a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800abf90 con 0x7f8f80061e60 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800ac230 con 0x7f8f800669a0 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800ac4d0 con 0x7f8f800acc10 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f86102700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 0x7f8f80147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f86102700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56236/0 (socket says 172.21.15.185:56236) 2020-11-16T22:14:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f86102700 1 -- 172.21.15.185:0/4121911916 learned_addr learned my addr 172.21.15.185:0/4121911916 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f86903700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.96:3300/0 conn(0x7f8f80061e60 0x7f8f80147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 -- 172.21.15.185:0/4121911916 >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 msgr2=0x7f8f80066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:19.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 -- 172.21.15.185:0/4121911916 >> v2:172.21.15.96:3300/0 conn(0x7f8f80061e60 msgr2=0x7f8f80147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:19.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.96:3300/0 conn(0x7f8f80061e60 0x7f8f80147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 -- 172.21.15.185:0/4121911916 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800ac770 con 0x7f8f800acc10 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f86102700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85901700 1 --2- 172.21.15.185:0/4121911916 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 0x7f8f80147550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f800b2ec0 tx=0x7f8f70006df0).ready entity=mon.2 client_cookie=c791e64c5908d32a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85100700 1 -- 172.21.15.185:0/4121911916 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70009640 con 0x7f8f800acc10 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85100700 1 -- 172.21.15.185:0/4121911916 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f700097a0 con 0x7f8f800acc10 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f85100700 1 -- 172.21.15.185:0/4121911916 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70009b20 con 0x7f8f800acc10 2020-11-16T22:14:19.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- 172.21.15.185:0/4121911916 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 msgr2=0x7f8f80147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- 172.21.15.185:0/4121911916 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 0x7f8f80147550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f800b2ec0 tx=0x7f8f70006df0).stop 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- 172.21.15.185:0/4121911916 shutdown_connections 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.96:3300/0 conn(0x7f8f80061e60 0x7f8f80147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- 172.21.15.185:0/4121911916 >> v2:172.21.15.185:3300/0 conn(0x7f8f800669a0 0x7f8f80066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 --2- 172.21.15.185:0/4121911916 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800acc10 0x7f8f80147550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- 172.21.15.185:0/4121911916 >> 172.21.15.185:0/4121911916 conn(0x7f8f80060130 msgr2=0x7f8f80060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:19.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- 172.21.15.185:0/4121911916 shutdown_connections 2020-11-16T22:14:19.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.359+0000 7f8f87104700 1 -- 172.21.15.185:0/4121911916 wait complete. 2020-11-16T22:14:19.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 Processor -- start 2020-11-16T22:14:19.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- start start 2020-11-16T22:14:19.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 0x7f8f80140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 0x7f8f80146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f80067880 0x7f8f80147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80113e60 con 0x7f8f80067880 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80145cb0 con 0x7f8f80061e60 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80147800 con 0x7f8f800669a0 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 0x7f8f80146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f86102700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 0x7f8f80140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f86102700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 0x7f8f80140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56242/0 (socket says 172.21.15.185:56242) 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f86102700 1 -- 172.21.15.185:0/2323385690 learned_addr learned my addr 172.21.15.185:0/2323385690 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f86903700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f80067880 0x7f8f80147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 -- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 msgr2=0x7f8f80140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 0x7f8f80140640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 -- 172.21.15.185:0/2323385690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f80067880 msgr2=0x7f8f80147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f80067880 0x7f8f80147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 -- 172.21.15.185:0/2323385690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80154370 con 0x7f8f800669a0 2020-11-16T22:14:19.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f85901700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 0x7f8f80146e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f8f70012000 tx=0x7f8f70015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70011b40 con 0x7f8f800669a0 2020-11-16T22:14:19.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70011ca0 con 0x7f8f800669a0 2020-11-16T22:14:19.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f80154610 con 0x7f8f800669a0 2020-11-16T22:14:19.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7001e670 con 0x7f8f800669a0 2020-11-16T22:14:19.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.363+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f801459b0 con 0x7f8f800669a0 2020-11-16T22:14:19.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8f70025030 con 0x7f8f800669a0 2020-11-16T22:14:19.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8005d200 con 0x7f8f800669a0 2020-11-16T22:14:19.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f7e7fc700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 0x7f8f58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8f7006f140 con 0x7f8f800669a0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f86102700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 0x7f8f58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.367+0000 7f8f86102700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 0x7f8f58057520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8f78009cf0 tx=0x7f8f78009290).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.371+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8f700453a0 con 0x7f8f800669a0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8f80067270 con 0x7f8f800669a0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f7e7fc700 1 -- 172.21.15.185:0/2323385690 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f70023020 con 0x7f8f800669a0 2020-11-16T22:14:19.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 msgr2=0x7f8f58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 0x7f8f58057520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8f78009cf0 tx=0x7f8f78009290).stop 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 msgr2=0x7f8f80146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 0x7f8f80146e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f8f70012000 tx=0x7f8f70015000).stop 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 shutdown_connections 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f58055060 0x7f8f58057520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f80061e60 0x7f8f80140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f80067880 0x7f8f80147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 --2- 172.21.15.185:0/2323385690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f800669a0 0x7f8f80146e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 >> 172.21.15.185:0/2323385690 conn(0x7f8f80060130 msgr2=0x7f8f80142610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 shutdown_connections 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.523+0000 7f8f87104700 1 -- 172.21.15.185:0/2323385690 wait complete. 2020-11-16T22:14:19.563 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:19.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2020-11-16T22:14:19.564 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2020-11-16T22:14:19.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 Processor -- start 2020-11-16T22:14:19.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 -- start start 2020-11-16T22:14:19.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa5ec0669a0 0x7fa5ec066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0abf90 con 0x7fa5ec061e60 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0ac230 con 0x7fa5ec0669a0 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0ac4d0 con 0x7fa5ec0acc10 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49694/0 (socket says 172.21.15.185:49694) 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 -- 172.21.15.185:0/2253496122 learned_addr learned my addr 172.21.15.185:0/2253496122 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:19.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f278e700 1 --1- 172.21.15.185:0/2253496122 >> v1:172.21.15.185:6789/0 conn(0x7fa5ec061e60 0x7fa5ec147a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49690/0 (socket says 172.21.15.185:49690) 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f1f8d700 1 --2- 172.21.15.185:0/2253496122 >> v2:172.21.15.96:3300/0 conn(0x7fa5ec0669a0 0x7fa5ec066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 -- 172.21.15.185:0/2253496122 >> v2:172.21.15.96:3300/0 conn(0x7fa5ec0669a0 msgr2=0x7fa5ec066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 --2- 172.21.15.185:0/2253496122 >> v2:172.21.15.96:3300/0 conn(0x7fa5ec0669a0 0x7fa5ec066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 -- 172.21.15.185:0/2253496122 >> v1:172.21.15.185:6789/0 conn(0x7fa5ec061e60 legacy=0x7fa5ec147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f0f8b700 1 -- 172.21.15.185:0/2253496122 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381772915 0 0) 0x7fa5ec0abf90 con 0x7fa5ec061e60 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 -- 172.21.15.185:0/2253496122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0ac770 con 0x7fa5ec0acc10 2020-11-16T22:14:19.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f178c700 1 --2- 172.21.15.185:0/2253496122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa5dc006fd0 tx=0x7fa5dc006eb0).ready entity=mon.2 client_cookie=b3cb44fb7d1168d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f0f8b700 1 -- 172.21.15.185:0/2253496122 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0098d0 con 0x7fa5ec0acc10 2020-11-16T22:14:19.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f0f8b700 1 -- 172.21.15.185:0/2253496122 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc009a30 con 0x7fa5ec0acc10 2020-11-16T22:14:19.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f0f8b700 1 -- 172.21.15.185:0/2253496122 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc009db0 con 0x7fa5ec0acc10 2020-11-16T22:14:19.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/2253496122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 msgr2=0x7fa5ec147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.703+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/2253496122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa5dc006fd0 tx=0x7fa5dc006eb0).stop 2020-11-16T22:14:19.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/2253496122 shutdown_connections 2020-11-16T22:14:19.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/2253496122 >> v2:172.21.15.96:3300/0 conn(0x7fa5ec0669a0 0x7fa5ec066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/2253496122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0acc10 0x7fa5ec147550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/2253496122 >> 172.21.15.185:0/2253496122 conn(0x7fa5ec060130 msgr2=0x7fa5ec060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/2253496122 shutdown_connections 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/2253496122 wait complete. 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 Processor -- start 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- start start 2020-11-16T22:14:19.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ec061e60 0x7fa5ec147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ec0669a0 0x7fa5ec1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1574d0 con 0x7fa5ec0669a0 2020-11-16T22:14:19.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec157740 con 0x7fa5ec061e60 2020-11-16T22:14:19.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0b0190 con 0x7fa5ec0b0630 2020-11-16T22:14:19.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49702/0 (socket says 172.21.15.185:49702) 2020-11-16T22:14:19.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 -- 172.21.15.185:0/1163287981 learned_addr learned my addr 172.21.15.185:0/1163287981 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:19.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f1f8d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ec061e60 0x7fa5ec147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f178c700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ec0669a0 0x7fa5ec1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 -- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ec061e60 msgr2=0x7fa5ec147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ec061e60 0x7fa5ec147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 -- 172.21.15.185:0/1163287981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ec0669a0 msgr2=0x7fa5ec1402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ec0669a0 0x7fa5ec1402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 -- 172.21.15.185:0/1163287981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec1407e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f278e700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa5e400cab0 tx=0x7fa5e4007ce0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005ac0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4005c20 con 0x7fa5ec0b0630 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec1579e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4007110 con 0x7fa5ec0b0630 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.707+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec147690 con 0x7fa5ec0b0630 2020-11-16T22:14:19.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec1478e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa5e4010030 con 0x7fa5ec0b0630 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5d9ffb700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 0x7fa5c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5e406dcb0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5f1f8d700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 0x7fa5c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.711+0000 7fa5f1f8d700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 0x7fa5c4057520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa5e800b6a0 tx=0x7fa5e8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:19.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.719+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa5ec1478e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.871+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa5ec1478e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.871+0000 7fa5d9ffb700 1 -- 172.21.15.185:0/1163287981 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5ec1478e0 con 0x7fa5ec0b0630 2020-11-16T22:14:19.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 msgr2=0x7fa5c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:19.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 0x7fa5c4057520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa5e800b6a0 tx=0x7fa5e8008040).stop 2020-11-16T22:14:20.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 msgr2=0x7fa5ec1535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa5e400cab0 tx=0x7fa5e4007ce0).stop 2020-11-16T22:14:20.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 shutdown_connections 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5c4055060 0x7fa5c4057520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5ec061e60 0x7fa5ec147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5ec0b0630 0x7fa5ec1535a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 --2- 172.21.15.185:0/1163287981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5ec0669a0 0x7fa5ec1402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 >> 172.21.15.185:0/1163287981 conn(0x7fa5ec060130 msgr2=0x7fa5ec1430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 shutdown_connections 2020-11-16T22:14:20.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:19.875+0000 7fa5f2f8f700 1 -- 172.21.15.185:0/1163287981 wait complete. 2020-11-16T22:14:20.112 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:20.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2020-11-16T22:14:20.112 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2020-11-16T22:14:20.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 Processor -- start 2020-11-16T22:14:20.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 -- start start 2020-11-16T22:14:20.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f95800b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 0x7f9580141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9580066ad0 con 0x7f95800b7470 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9580140d10 con 0x7f9580141100 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9580144d90 con 0x7f95800b2920 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f95800b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f95800b7470 0x7f958013e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49704/0 (socket says 172.21.15.185:49704) 2020-11-16T22:14:20.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957e59c700 1 -- 172.21.15.185:0/1700722142 learned_addr learned my addr 172.21.15.185:0/1700722142 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:20.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957d59a700 1 -- 172.21.15.185:0/1700722142 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702619849 0 0) 0x7f9580066ad0 con 0x7f95800b7470 2020-11-16T22:14:20.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 -- 172.21.15.185:0/1700722142 >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 msgr2=0x7f9580141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:20.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957ed9d700 1 --2- 172.21.15.185:0/1700722142 >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 0x7f9580141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 --2- 172.21.15.185:0/1700722142 >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 0x7f9580141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 -- 172.21.15.185:0/1700722142 >> v1:172.21.15.185:6789/0 conn(0x7f95800b7470 legacy=0x7f958013e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 -- 172.21.15.185:0/1700722142 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580144ff0 con 0x7f95800b2920 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957ed9d700 1 --2- 172.21.15.185:0/1700722142 >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 0x7f9580141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957dd9b700 1 --2- 172.21.15.185:0/1700722142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f95800b7850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f956800b9a0 tx=0x7f9568006df0).ready entity=mon.2 client_cookie=e542bbe36b72f45 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957d59a700 1 -- 172.21.15.185:0/1700722142 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568009840 con 0x7f95800b2920 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957d59a700 1 -- 172.21.15.185:0/1700722142 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95680099a0 con 0x7f95800b2920 2020-11-16T22:14:20.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.259+0000 7f957d59a700 1 -- 172.21.15.185:0/1700722142 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568009d20 con 0x7f95800b2920 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 -- 172.21.15.185:0/1700722142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 msgr2=0x7f95800b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 --2- 172.21.15.185:0/1700722142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f95800b7850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f956800b9a0 tx=0x7f9568006df0).stop 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 -- 172.21.15.185:0/1700722142 shutdown_connections 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 --2- 172.21.15.185:0/1700722142 >> v2:172.21.15.96:3300/0 conn(0x7f9580141100 0x7f9580141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 --2- 172.21.15.185:0/1700722142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f95800b7850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 -- 172.21.15.185:0/1700722142 >> 172.21.15.185:0/1700722142 conn(0x7f95800637b0 msgr2=0x7f95800b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:20.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 -- 172.21.15.185:0/1700722142 shutdown_connections 2020-11-16T22:14:20.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 -- 172.21.15.185:0/1700722142 wait complete. 2020-11-16T22:14:20.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.263+0000 7f957f59e700 1 Processor -- start 2020-11-16T22:14:20.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- start start 2020-11-16T22:14:20.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f9580150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9580141100 0x7f9580153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801501b0 con 0x7f9580141100 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9580141e40 con 0x7f95800b7470 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95801503e0 con 0x7f95800b2920 2020-11-16T22:14:20.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f9580150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56274/0 (socket says 172.21.15.185:56274) 2020-11-16T22:14:20.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 -- 172.21.15.185:0/1152135971 learned_addr learned my addr 172.21.15.185:0/1152135971 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:20.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957ed9d700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9580141100 0x7f9580153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 -- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 msgr2=0x7f9580150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f9580150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 -- 172.21.15.185:0/1152135971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9580141100 msgr2=0x7f9580153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9580141100 0x7f9580153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 -- 172.21.15.185:0/1152135971 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580150680 con 0x7f95800b7470 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957e59c700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f9580150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957dd9b700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f956800bf10 tx=0x7f9568015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801e560 con 0x7f95800b7470 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956801e6c0 con 0x7f95800b7470 2020-11-16T22:14:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801ea40 con 0x7f95800b7470 2020-11-16T22:14:20.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9580067c80 con 0x7f95800b7470 2020-11-16T22:14:20.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95800adba0 con 0x7f95800b7470 2020-11-16T22:14:20.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.267+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958005d200 con 0x7f95800b7470 2020-11-16T22:14:20.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.271+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f956801a040 con 0x7f95800b7470 2020-11-16T22:14:20.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.271+0000 7f95767fc700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 0x7f955c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.271+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9568071a70 con 0x7f95800b7470 2020-11-16T22:14:20.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.271+0000 7f957e59c700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 0x7f955c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.271+0000 7f957e59c700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 0x7f955c057520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f95700011a0 tx=0x7f9570007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.275+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9568047bc0 con 0x7f95800b7470 2020-11-16T22:14:20.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.423+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f958005c6d0 con 0x7f95800b7470 2020-11-16T22:14:20.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.423+0000 7f95767fc700 1 -- 172.21.15.185:0/1152135971 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f956804b1e0 con 0x7f95800b7470 2020-11-16T22:14:20.412 INFO:teuthology.orchestra.run.smithi185.stdout:55834574854 2020-11-16T22:14:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 msgr2=0x7f955c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 0x7f955c057520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f95700011a0 tx=0x7f9570007a00).stop 2020-11-16T22:14:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 msgr2=0x7f9580067790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f956800bf10 tx=0x7f9568015000).stop 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 shutdown_connections 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f955c055060 0x7f955c057520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9580141100 0x7f9580153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95800b2920 0x7f9580150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 --2- 172.21.15.185:0/1152135971 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95800b7470 0x7f9580067790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 >> 172.21.15.185:0/1152135971 conn(0x7f95800637b0 msgr2=0x7f958013cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:20.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 shutdown_connections 2020-11-16T22:14:20.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.427+0000 7f957f59e700 1 -- 172.21.15.185:0/1152135971 wait complete. 2020-11-16T22:14:20.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2020-11-16T22:14:20.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T22:14:20.426 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:14:20.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 Processor -- start 2020-11-16T22:14:20.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 -- start start 2020-11-16T22:14:20.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 0x7f23580b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 0x7f23580acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2358147f90 con 0x7f23580b64b0 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23580af020 con 0x7f23580b6aa0 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235e108700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b4810 con 0x7f23580b72e0 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d907700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 0x7f23580acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 0x7f23580b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235c905700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f23580b64b0 0x7f2358144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49718/0 (socket says 172.21.15.185:49718) 2020-11-16T22:14:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235c905700 1 -- 172.21.15.185:0/4186639019 learned_addr learned my addr 172.21.15.185:0/4186639019 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 -- 172.21.15.185:0/4186639019 >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 msgr2=0x7f23580acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 --2- 172.21.15.185:0/4186639019 >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 0x7f23580acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 -- 172.21.15.185:0/4186639019 >> v1:172.21.15.185:6789/0 conn(0x7f23580b64b0 legacy=0x7f2358144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 -- 172.21.15.185:0/4186639019 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358148460 con 0x7f23580b72e0 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d907700 1 --2- 172.21.15.185:0/4186639019 >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 0x7f23580acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:20.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f235d106700 1 --2- 172.21.15.185:0/4186639019 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 0x7f23580b76c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f234c00b990 tx=0x7f234c006d90).ready entity=mon.2 client_cookie=7826222e3e89079 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.559+0000 7f234bfff700 1 -- 172.21.15.185:0/4186639019 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c006030 con 0x7f23580b72e0 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f234bfff700 1 -- 172.21.15.185:0/4186639019 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c006190 con 0x7f23580b72e0 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f234bfff700 1 -- 172.21.15.185:0/4186639019 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c006510 con 0x7f23580b72e0 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- 172.21.15.185:0/4186639019 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 msgr2=0x7f23580b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- 172.21.15.185:0/4186639019 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 0x7f23580b76c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f234c00b990 tx=0x7f234c006d90).stop 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- 172.21.15.185:0/4186639019 shutdown_connections 2020-11-16T22:14:20.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- 172.21.15.185:0/4186639019 >> v2:172.21.15.185:3300/0 conn(0x7f23580b6aa0 0x7f23580acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- 172.21.15.185:0/4186639019 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b72e0 0x7f23580b76c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- 172.21.15.185:0/4186639019 >> 172.21.15.185:0/4186639019 conn(0x7f2358067860 msgr2=0x7f2358067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:20.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- 172.21.15.185:0/4186639019 shutdown_connections 2020-11-16T22:14:20.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- 172.21.15.185:0/4186639019 wait complete. 2020-11-16T22:14:20.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 Processor -- start 2020-11-16T22:14:20.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- start start 2020-11-16T22:14:20.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 0x7f2358114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23580b6aa0 0x7f23580b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 0x7f2358156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b3540 con 0x7f23580b6aa0 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358114b00 con 0x7f23580b72e0 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.563+0000 7f235e108700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b3770 con 0x7f23580b64b0 2020-11-16T22:14:20.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d106700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 0x7f2358114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d106700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 0x7f2358114f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49730/0 (socket says 172.21.15.185:49730) 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d106700 1 -- 172.21.15.185:0/1721030765 learned_addr learned my addr 172.21.15.185:0/1721030765 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 0x7f2358156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235c905700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23580b6aa0 0x7f23580b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 -- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 msgr2=0x7f2358114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 0x7f2358114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 -- 172.21.15.185:0/1721030765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23580b6aa0 msgr2=0x7f23580b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23580b6aa0 0x7f23580b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 -- 172.21.15.185:0/1721030765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580b3a10 con 0x7f23580b72e0 2020-11-16T22:14:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235d907700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 0x7f2358156db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f23580b6fc0 tx=0x7f23500091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000cbc0 con 0x7f23580b72e0 2020-11-16T22:14:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235000cd20 con 0x7f23580b72e0 2020-11-16T22:14:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350012620 con 0x7f23580b72e0 2020-11-16T22:14:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23580b2510 con 0x7f23580b72e0 2020-11-16T22:14:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23580b0f60 con 0x7f23580b72e0 2020-11-16T22:14:20.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235805d200 con 0x7f23580b72e0 2020-11-16T22:14:20.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.567+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2350012a10 con 0x7f23580b72e0 2020-11-16T22:14:20.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.571+0000 7f23497fa700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 0x7f2330057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:20.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.571+0000 7f235d106700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 0x7f2330057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:20.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.571+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f235006e7a0 con 0x7f23580b72e0 2020-11-16T22:14:20.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.571+0000 7f235d106700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 0x7f2330057520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f234c006dd0 tx=0x7f234c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:20.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.575+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2350044a00 con 0x7f23580b72e0 2020-11-16T22:14:20.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.723+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 --> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23580603a0 con 0x7f2330055060 2020-11-16T22:14:20.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.727+0000 7f23497fa700 1 -- 172.21.15.185:0/1721030765 <== mgr.4106 v2:172.21.15.185:6832/15035 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82177 (secure 0 0 0) 0x7f23580603a0 con 0x7f2330055060 2020-11-16T22:14:20.716 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:20.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 msgr2=0x7f2330057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 0x7f2330057520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f234c006dd0 tx=0x7f234c015000).stop 2020-11-16T22:14:20.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 msgr2=0x7f2358156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:20.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 0x7f2358156db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f23580b6fc0 tx=0x7f23500091d0).stop 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 shutdown_connections 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2330055060 0x7f2330057520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23580b72e0 0x7f2358156db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23580b64b0 0x7f2358114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 --2- 172.21.15.185:0/1721030765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23580b6aa0 0x7f23580b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 >> 172.21.15.185:0/1721030765 conn(0x7f2358067860 msgr2=0x7f23581453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:20.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 shutdown_connections 2020-11-16T22:14:20.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:20.731+0000 7f235e108700 1 -- 172.21.15.185:0/1721030765 wait complete. 2020-11-16T22:14:20.720 INFO:teuthology.orchestra.run.smithi185.stderr:dumped all 2020-11-16T22:14:20.737 INFO:teuthology.orchestra.run.smithi185.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2020-11-16T22:14:19.048993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40928,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":38912,"kb_avail":754933792,"statfs":{"total":773094113280,"available":773052203008,"internally_reserved":0,"allocated":1998848,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39845888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.499135"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579437+0000","last_change":"2020-11-16T22:14:07.566898+0000","last_active":"2020-11-16T22:14:07.579437+0000","last_peered":"2020-11-16T22:14:07.579437+0000","last_clean":"2020-11-16T22:14:07.579437+0000","last_became_active":"2020-11-16T22:14:07.565584+0000","last_became_peered":"2020-11-16T22:14:07.565584+0000","last_unstale":"2020-11-16T22:14:07.579437+0000","last_undegraded":"2020-11-16T22:14:07.579437+0000","last_fullsized":"2020-11-16T22:14:07.579437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.552159+0000","last_change":"2020-11-16T22:14:00.778100+0000","last_active":"2020-11-16T22:14:07.552159+0000","last_peered":"2020-11-16T22:14:07.552159+0000","last_clean":"2020-11-16T22:14:07.552159+0000","last_became_active":"2020-11-16T22:14:00.776950+0000","last_became_peered":"2020-11-16T22:14:00.776950+0000","last_unstale":"2020-11-16T22:14:07.552159+0000","last_undegraded":"2020-11-16T22:14:07.552159+0000","last_fullsized":"2020-11-16T22:14:07.552159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"22'10","reported_seq":"20","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.610735+0000","last_change":"2020-11-16T22:14:07.190947+0000","last_active":"2020-11-16T22:14:10.610735+0000","last_peered":"2020-11-16T22:14:10.610735+0000","last_clean":"2020-11-16T22:14:10.610735+0000","last_became_active":"2020-11-16T22:14:07.190501+0000","last_became_peered":"2020-11-16T22:14:07.190501+0000","last_unstale":"2020-11-16T22:14:10.610735+0000","last_undegraded":"2020-11-16T22:14:10.610735+0000","last_fullsized":"2020-11-16T22:14:10.610735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579881+0000","last_change":"2020-11-16T22:14:07.573161+0000","last_active":"2020-11-16T22:14:07.579881+0000","last_peered":"2020-11-16T22:14:07.579881+0000","last_clean":"2020-11-16T22:14:07.579881+0000","last_became_active":"2020-11-16T22:14:07.572739+0000","last_became_peered":"2020-11-16T22:14:07.572739+0000","last_unstale":"2020-11-16T22:14:07.579881+0000","last_undegraded":"2020-11-16T22:14:07.579881+0000","last_fullsized":"2020-11-16T22:14:07.579881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.556778+0000","last_change":"2020-11-16T22:14:00.775255+0000","last_active":"2020-11-16T22:14:07.556778+0000","last_peered":"2020-11-16T22:14:07.556778+0000","last_clean":"2020-11-16T22:14:07.556778+0000","last_became_active":"2020-11-16T22:14:00.774540+0000","last_became_peered":"2020-11-16T22:14:00.774540+0000","last_unstale":"2020-11-16T22:14:07.556778+0000","last_undegraded":"2020-11-16T22:14:07.556778+0000","last_fullsized":"2020-11-16T22:14:07.556778+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"22'18","reported_seq":"28","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.600278+0000","last_change":"2020-11-16T22:14:07.193600+0000","last_active":"2020-11-16T22:14:10.600278+0000","last_peered":"2020-11-16T22:14:10.600278+0000","last_clean":"2020-11-16T22:14:10.600278+0000","last_became_active":"2020-11-16T22:14:07.193214+0000","last_became_peered":"2020-11-16T22:14:07.193214+0000","last_unstale":"2020-11-16T22:14:10.600278+0000","last_undegraded":"2020-11-16T22:14:10.600278+0000","last_fullsized":"2020-11-16T22:14:10.600278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579988+0000","last_change":"2020-11-16T22:14:07.572671+0000","last_active":"2020-11-16T22:14:07.579988+0000","last_peered":"2020-11-16T22:14:07.579988+0000","last_clean":"2020-11-16T22:14:07.579988+0000","last_became_active":"2020-11-16T22:14:07.571659+0000","last_became_peered":"2020-11-16T22:14:07.571659+0000","last_unstale":"2020-11-16T22:14:07.579988+0000","last_undegraded":"2020-11-16T22:14:07.579988+0000","last_fullsized":"2020-11-16T22:14:07.579988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557158+0000","last_change":"2020-11-16T22:13:59.943744+0000","last_active":"2020-11-16T22:14:07.557158+0000","last_peered":"2020-11-16T22:14:07.557158+0000","last_clean":"2020-11-16T22:14:07.557158+0000","last_became_active":"2020-11-16T22:13:59.942771+0000","last_became_peered":"2020-11-16T22:13:59.942771+0000","last_unstale":"2020-11-16T22:14:07.557158+0000","last_undegraded":"2020-11-16T22:14:07.557158+0000","last_fullsized":"2020-11-16T22:14:07.557158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"22'21","reported_seq":"31","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.611811+0000","last_change":"2020-11-16T22:14:07.194335+0000","last_active":"2020-11-16T22:14:10.611811+0000","last_peered":"2020-11-16T22:14:10.611811+0000","last_clean":"2020-11-16T22:14:10.611811+0000","last_became_active":"2020-11-16T22:14:07.193814+0000","last_became_peered":"2020-11-16T22:14:07.193814+0000","last_unstale":"2020-11-16T22:14:10.611811+0000","last_undegraded":"2020-11-16T22:14:10.611811+0000","last_fullsized":"2020-11-16T22:14:10.611811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.571074+0000","last_change":"2020-11-16T22:14:07.566129+0000","last_active":"2020-11-16T22:14:07.571074+0000","last_peered":"2020-11-16T22:14:07.571074+0000","last_clean":"2020-11-16T22:14:07.571074+0000","last_became_active":"2020-11-16T22:14:07.565783+0000","last_became_peered":"2020-11-16T22:14:07.565783+0000","last_unstale":"2020-11-16T22:14:07.571074+0000","last_undegraded":"2020-11-16T22:14:07.571074+0000","last_fullsized":"2020-11-16T22:14:07.571074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557441+0000","last_change":"2020-11-16T22:14:00.734870+0000","last_active":"2020-11-16T22:14:07.557441+0000","last_peered":"2020-11-16T22:14:07.557441+0000","last_clean":"2020-11-16T22:14:07.557441+0000","last_became_active":"2020-11-16T22:14:00.733576+0000","last_became_peered":"2020-11-16T22:14:00.733576+0000","last_unstale":"2020-11-16T22:14:07.557441+0000","last_undegraded":"2020-11-16T22:14:07.557441+0000","last_fullsized":"2020-11-16T22:14:07.557441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"15","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.552384+0000","last_change":"2020-11-16T22:13:55.098170+0000","last_active":"2020-11-16T22:14:07.552384+0000","last_peered":"2020-11-16T22:14:07.552384+0000","last_clean":"2020-11-16T22:14:07.552384+0000","last_became_active":"2020-11-16T22:13:55.097158+0000","last_became_peered":"2020-11-16T22:13:55.097158+0000","last_unstale":"2020-11-16T22:14:07.552384+0000","last_undegraded":"2020-11-16T22:14:07.552384+0000","last_fullsized":"2020-11-16T22:14:07.552384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:52.038457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:52.038457+0000","last_clean_scrub_stamp":"2020-11-16T22:13:52.038457+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.609149+0000","last_change":"2020-11-16T22:14:07.196200+0000","last_active":"2020-11-16T22:14:10.609149+0000","last_peered":"2020-11-16T22:14:10.609149+0000","last_clean":"2020-11-16T22:14:10.609149+0000","last_became_active":"2020-11-16T22:14:07.195690+0000","last_became_peered":"2020-11-16T22:14:07.195690+0000","last_unstale":"2020-11-16T22:14:10.609149+0000","last_undegraded":"2020-11-16T22:14:10.609149+0000","last_fullsized":"2020-11-16T22:14:10.609149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.568990+0000","last_change":"2020-11-16T22:14:07.566447+0000","last_active":"2020-11-16T22:14:07.568990+0000","last_peered":"2020-11-16T22:14:07.568990+0000","last_clean":"2020-11-16T22:14:07.568990+0000","last_became_active":"2020-11-16T22:14:07.565200+0000","last_became_peered":"2020-11-16T22:14:07.565200+0000","last_unstale":"2020-11-16T22:14:07.568990+0000","last_undegraded":"2020-11-16T22:14:07.568990+0000","last_fullsized":"2020-11-16T22:14:07.568990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557551+0000","last_change":"2020-11-16T22:13:59.947294+0000","last_active":"2020-11-16T22:14:07.557551+0000","last_peered":"2020-11-16T22:14:07.557551+0000","last_clean":"2020-11-16T22:14:07.557551+0000","last_became_active":"2020-11-16T22:13:59.946791+0000","last_became_peered":"2020-11-16T22:13:59.946791+0000","last_unstale":"2020-11-16T22:14:07.557551+0000","last_undegraded":"2020-11-16T22:14:07.557551+0000","last_fullsized":"2020-11-16T22:14:07.557551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"22'12","reported_seq":"22","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.613425+0000","last_change":"2020-11-16T22:14:07.197117+0000","last_active":"2020-11-16T22:14:10.613425+0000","last_peered":"2020-11-16T22:14:10.613425+0000","last_clean":"2020-11-16T22:14:10.613425+0000","last_became_active":"2020-11-16T22:14:07.196668+0000","last_became_peered":"2020-11-16T22:14:07.196668+0000","last_unstale":"2020-11-16T22:14:10.613425+0000","last_undegraded":"2020-11-16T22:14:10.613425+0000","last_fullsized":"2020-11-16T22:14:10.613425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579638+0000","last_change":"2020-11-16T22:14:07.572538+0000","last_active":"2020-11-16T22:14:07.579638+0000","last_peered":"2020-11-16T22:14:07.579638+0000","last_clean":"2020-11-16T22:14:07.579638+0000","last_became_active":"2020-11-16T22:14:07.571320+0000","last_became_peered":"2020-11-16T22:14:07.571320+0000","last_unstale":"2020-11-16T22:14:07.579638+0000","last_undegraded":"2020-11-16T22:14:07.579638+0000","last_fullsized":"2020-11-16T22:14:07.579638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557560+0000","last_change":"2020-11-16T22:13:59.946679+0000","last_active":"2020-11-16T22:14:07.557560+0000","last_peered":"2020-11-16T22:14:07.557560+0000","last_clean":"2020-11-16T22:14:07.557560+0000","last_became_active":"2020-11-16T22:13:59.946248+0000","last_became_peered":"2020-11-16T22:13:59.946248+0000","last_unstale":"2020-11-16T22:14:07.557560+0000","last_undegraded":"2020-11-16T22:14:07.557560+0000","last_fullsized":"2020-11-16T22:14:07.557560+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"22'13","reported_seq":"23","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.609944+0000","last_change":"2020-11-16T22:14:07.194904+0000","last_active":"2020-11-16T22:14:10.609944+0000","last_peered":"2020-11-16T22:14:10.609944+0000","last_clean":"2020-11-16T22:14:10.609944+0000","last_became_active":"2020-11-16T22:14:07.194443+0000","last_became_peered":"2020-11-16T22:14:07.194443+0000","last_unstale":"2020-11-16T22:14:10.609944+0000","last_undegraded":"2020-11-16T22:14:10.609944+0000","last_fullsized":"2020-11-16T22:14:10.609944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579498+0000","last_change":"2020-11-16T22:14:07.574655+0000","last_active":"2020-11-16T22:14:07.579498+0000","last_peered":"2020-11-16T22:14:07.579498+0000","last_clean":"2020-11-16T22:14:07.579498+0000","last_became_active":"2020-11-16T22:14:07.574038+0000","last_became_peered":"2020-11-16T22:14:07.574038+0000","last_unstale":"2020-11-16T22:14:07.579498+0000","last_undegraded":"2020-11-16T22:14:07.579498+0000","last_fullsized":"2020-11-16T22:14:07.579498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'17","reported_seq":"27","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.613164+0000","last_change":"2020-11-16T22:14:07.193233+0000","last_active":"2020-11-16T22:14:10.613164+0000","last_peered":"2020-11-16T22:14:10.613164+0000","last_clean":"2020-11-16T22:14:10.613164+0000","last_became_active":"2020-11-16T22:14:07.192772+0000","last_became_peered":"2020-11-16T22:14:07.192772+0000","last_unstale":"2020-11-16T22:14:10.613164+0000","last_undegraded":"2020-11-16T22:14:10.613164+0000","last_fullsized":"2020-11-16T22:14:10.613164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.562278+0000","last_change":"2020-11-16T22:13:59.946652+0000","last_active":"2020-11-16T22:14:07.562278+0000","last_peered":"2020-11-16T22:14:07.562278+0000","last_clean":"2020-11-16T22:14:07.562278+0000","last_became_active":"2020-11-16T22:13:59.946109+0000","last_became_peered":"2020-11-16T22:13:59.946109+0000","last_unstale":"2020-11-16T22:14:07.562278+0000","last_undegraded":"2020-11-16T22:14:07.562278+0000","last_fullsized":"2020-11-16T22:14:07.562278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.563643+0000","last_change":"2020-11-16T22:14:07.562465+0000","last_active":"2020-11-16T22:14:07.563643+0000","last_peered":"2020-11-16T22:14:07.563643+0000","last_clean":"2020-11-16T22:14:07.563643+0000","last_became_active":"2020-11-16T22:14:07.562029+0000","last_became_peered":"2020-11-16T22:14:07.562029+0000","last_unstale":"2020-11-16T22:14:07.563643+0000","last_undegraded":"2020-11-16T22:14:07.563643+0000","last_fullsized":"2020-11-16T22:14:07.563643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557775+0000","last_change":"2020-11-16T22:13:59.944154+0000","last_active":"2020-11-16T22:14:07.557775+0000","last_peered":"2020-11-16T22:14:07.557775+0000","last_clean":"2020-11-16T22:14:07.557775+0000","last_became_active":"2020-11-16T22:13:59.943659+0000","last_became_peered":"2020-11-16T22:13:59.943659+0000","last_unstale":"2020-11-16T22:14:07.557775+0000","last_undegraded":"2020-11-16T22:14:07.557775+0000","last_fullsized":"2020-11-16T22:14:07.557775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'9","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.608412+0000","last_change":"2020-11-16T22:14:07.196926+0000","last_active":"2020-11-16T22:14:10.608412+0000","last_peered":"2020-11-16T22:14:10.608412+0000","last_clean":"2020-11-16T22:14:10.608412+0000","last_became_active":"2020-11-16T22:14:07.196552+0000","last_became_peered":"2020-11-16T22:14:07.196552+0000","last_unstale":"2020-11-16T22:14:10.608412+0000","last_undegraded":"2020-11-16T22:14:10.608412+0000","last_fullsized":"2020-11-16T22:14:10.608412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":9924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":86097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3530000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.254}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5630000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6640000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5216,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366624,"statfs":{"total":96636764160,"available":96631422976,"internally_reserved":0,"allocated":286720,"data_stored":90193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9490000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.827}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6070000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5489999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5032,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366808,"statfs":{"total":96636764160,"available":96631611392,"internally_reserved":0,"allocated":229376,"data_stored":85985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2029999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1040000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":86627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9740000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4580000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2959999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5140,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366700,"statfs":{"total":96636764160,"available":96631500800,"internally_reserved":0,"allocated":274432,"data_stored":88082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2810000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.206}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5096,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366744,"statfs":{"total":96636764160,"available":96631545856,"internally_reserved":0,"allocated":229376,"data_stored":85985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1869999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.742}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.919}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.073}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":241664,"data_stored":86650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7189999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":253952,"data_stored":88185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3239999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.52}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0830000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9670000000000001}]}]}]}} 2020-11-16T22:14:21.759 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T22:14:21.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 Processor -- start 2020-11-16T22:14:21.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 -- start start 2020-11-16T22:14:21.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 0x7f5e30066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:21.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e300acc10 0x7f5e30147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300abf90 con 0x7f5e30061e60 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300ac230 con 0x7f5e300acc10 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e35527700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ac4d0 con 0x7f5e300669a0 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 0x7f5e30066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 0x7f5e30066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49740/0 (socket says 172.21.15.185:49740) 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e300acc10 0x7f5e30147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:21.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2ffff700 1 -- 172.21.15.185:0/2388539910 learned_addr learned my addr 172.21.15.185:0/2388539910 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2388539910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 msgr2=0x7f5e30066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 --2- 172.21.15.185:0/2388539910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 0x7f5e30066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2388539910 >> v1:172.21.15.96:6789/0 conn(0x7f5e30061e60 legacy=0x7f5e30147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2388539910 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300ac770 con 0x7f5e300acc10 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2effd700 1 -- 172.21.15.185:0/2388539910 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120788488 0 0) 0x7f5e300abf90 con 0x7f5e30061e60 2020-11-16T22:14:21.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.903+0000 7f5e2f7fe700 1 --2- 172.21.15.185:0/2388539910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e300acc10 0x7f5e30147550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5e18009c90 tx=0x7f5e180091d0).ready entity=mon.0 client_cookie=b0a290ebbf296a98 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:22.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2effd700 1 -- 172.21.15.185:0/2388539910 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800ba70 con 0x7f5e300acc10 2020-11-16T22:14:22.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2effd700 1 -- 172.21.15.185:0/2388539910 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1800bbd0 con 0x7f5e300acc10 2020-11-16T22:14:22.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2effd700 1 -- 172.21.15.185:0/2388539910 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18011400 con 0x7f5e300acc10 2020-11-16T22:14:22.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- 172.21.15.185:0/2388539910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e300acc10 msgr2=0x7f5e30147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- 172.21.15.185:0/2388539910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e300acc10 0x7f5e30147550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5e18009c90 tx=0x7f5e180091d0).stop 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- 172.21.15.185:0/2388539910 shutdown_connections 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- 172.21.15.185:0/2388539910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300669a0 0x7f5e30066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- 172.21.15.185:0/2388539910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e300acc10 0x7f5e30147550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- 172.21.15.185:0/2388539910 >> 172.21.15.185:0/2388539910 conn(0x7f5e30060130 msgr2=0x7f5e30060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- 172.21.15.185:0/2388539910 shutdown_connections 2020-11-16T22:14:22.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- 172.21.15.185:0/2388539910 wait complete. 2020-11-16T22:14:22.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 Processor -- start 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- start start 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e30061e60 0x7f5e30114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300aec40 0x7f5e30142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ae370 con 0x7f5e30061e60 2020-11-16T22:14:22.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30114c60 con 0x7f5e300669a0 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e35527700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300af440 con 0x7f5e300aec40 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56302/0 (socket says 172.21.15.185:56302) 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2681829945 learned_addr learned my addr 172.21.15.185:0/2681829945 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e2ffff700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e30061e60 0x7f5e30114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:22.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.907+0000 7f5e34d26700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300aec40 0x7f5e30142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300aec40 msgr2=0x7f5e30142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300aec40 0x7f5e30142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2681829945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e30061e60 msgr2=0x7f5e30114770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e30061e60 0x7f5e30114770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 -- 172.21.15.185:0/2681829945 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300af6e0 con 0x7f5e300669a0 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2f7fe700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5e1800b770 tx=0x7f5e18016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:22.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1801c5f0 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1801c750 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1801cad0 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e301142d0 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e30153980 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5e18017070 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300af260 con 0x7f5e300669a0 2020-11-16T22:14:22.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 0x7f5e00057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e1806ff00 con 0x7f5e300669a0 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2ffff700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 0x7f5e00057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.911+0000 7f5e2ffff700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 0x7f5e00057520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5e28006a10 tx=0x7f5e28008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:21.923+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5e18046160 con 0x7f5e300669a0 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.067+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 --> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e300af870 con 0x7f5e00055060 2020-11-16T22:14:22.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.075+0000 7f5e27fff700 1 -- 172.21.15.185:0/2681829945 <== mgr.4106 v2:172.21.15.185:6832/15035 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82101 (secure 0 0 0) 0x7f5e300af870 con 0x7f5e00055060 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 msgr2=0x7f5e00057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 0x7f5e00057520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5e28006a10 tx=0x7f5e28008040).stop 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 msgr2=0x7f5e30153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5e1800b770 tx=0x7f5e18016000).stop 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 shutdown_connections 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e00055060 0x7f5e00057520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e300aec40 0x7f5e30142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e30061e60 0x7f5e30114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 --2- 172.21.15.185:0/2681829945 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e300669a0 0x7f5e30153490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 >> 172.21.15.185:0/2681829945 conn(0x7f5e30060130 msgr2=0x7f5e301430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 shutdown_connections 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:22.079+0000 7f5e35527700 1 -- 172.21.15.185:0/2681829945 wait complete. 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stderr:dumped all 2020-11-16T22:14:22.110 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:22.111 INFO:teuthology.orchestra.run.smithi185.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2020-11-16T22:14:21.049543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4962,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40992,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":38976,"kb_avail":754933728,"statfs":{"total":773094113280,"available":773052137472,"internally_reserved":0,"allocated":1998848,"data_stored":697804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39911424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003594"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579437+0000","last_change":"2020-11-16T22:14:07.566898+0000","last_active":"2020-11-16T22:14:07.579437+0000","last_peered":"2020-11-16T22:14:07.579437+0000","last_clean":"2020-11-16T22:14:07.579437+0000","last_became_active":"2020-11-16T22:14:07.565584+0000","last_became_peered":"2020-11-16T22:14:07.565584+0000","last_unstale":"2020-11-16T22:14:07.579437+0000","last_undegraded":"2020-11-16T22:14:07.579437+0000","last_fullsized":"2020-11-16T22:14:07.579437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.552159+0000","last_change":"2020-11-16T22:14:00.778100+0000","last_active":"2020-11-16T22:14:07.552159+0000","last_peered":"2020-11-16T22:14:07.552159+0000","last_clean":"2020-11-16T22:14:07.552159+0000","last_became_active":"2020-11-16T22:14:00.776950+0000","last_became_peered":"2020-11-16T22:14:00.776950+0000","last_unstale":"2020-11-16T22:14:07.552159+0000","last_undegraded":"2020-11-16T22:14:07.552159+0000","last_fullsized":"2020-11-16T22:14:07.552159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"22'10","reported_seq":"20","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.610735+0000","last_change":"2020-11-16T22:14:07.190947+0000","last_active":"2020-11-16T22:14:10.610735+0000","last_peered":"2020-11-16T22:14:10.610735+0000","last_clean":"2020-11-16T22:14:10.610735+0000","last_became_active":"2020-11-16T22:14:07.190501+0000","last_became_peered":"2020-11-16T22:14:07.190501+0000","last_unstale":"2020-11-16T22:14:10.610735+0000","last_undegraded":"2020-11-16T22:14:10.610735+0000","last_fullsized":"2020-11-16T22:14:10.610735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":735,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579881+0000","last_change":"2020-11-16T22:14:07.573161+0000","last_active":"2020-11-16T22:14:07.579881+0000","last_peered":"2020-11-16T22:14:07.579881+0000","last_clean":"2020-11-16T22:14:07.579881+0000","last_became_active":"2020-11-16T22:14:07.572739+0000","last_became_peered":"2020-11-16T22:14:07.572739+0000","last_unstale":"2020-11-16T22:14:07.579881+0000","last_undegraded":"2020-11-16T22:14:07.579881+0000","last_fullsized":"2020-11-16T22:14:07.579881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.556778+0000","last_change":"2020-11-16T22:14:00.775255+0000","last_active":"2020-11-16T22:14:07.556778+0000","last_peered":"2020-11-16T22:14:07.556778+0000","last_clean":"2020-11-16T22:14:07.556778+0000","last_became_active":"2020-11-16T22:14:00.774540+0000","last_became_peered":"2020-11-16T22:14:00.774540+0000","last_unstale":"2020-11-16T22:14:07.556778+0000","last_undegraded":"2020-11-16T22:14:07.556778+0000","last_fullsized":"2020-11-16T22:14:07.556778+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"22'18","reported_seq":"28","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.600278+0000","last_change":"2020-11-16T22:14:07.193600+0000","last_active":"2020-11-16T22:14:10.600278+0000","last_peered":"2020-11-16T22:14:10.600278+0000","last_clean":"2020-11-16T22:14:10.600278+0000","last_became_active":"2020-11-16T22:14:07.193214+0000","last_became_peered":"2020-11-16T22:14:07.193214+0000","last_unstale":"2020-11-16T22:14:10.600278+0000","last_undegraded":"2020-11-16T22:14:10.600278+0000","last_fullsized":"2020-11-16T22:14:10.600278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":642,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579988+0000","last_change":"2020-11-16T22:14:07.572671+0000","last_active":"2020-11-16T22:14:07.579988+0000","last_peered":"2020-11-16T22:14:07.579988+0000","last_clean":"2020-11-16T22:14:07.579988+0000","last_became_active":"2020-11-16T22:14:07.571659+0000","last_became_peered":"2020-11-16T22:14:07.571659+0000","last_unstale":"2020-11-16T22:14:07.579988+0000","last_undegraded":"2020-11-16T22:14:07.579988+0000","last_fullsized":"2020-11-16T22:14:07.579988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557158+0000","last_change":"2020-11-16T22:13:59.943744+0000","last_active":"2020-11-16T22:14:07.557158+0000","last_peered":"2020-11-16T22:14:07.557158+0000","last_clean":"2020-11-16T22:14:07.557158+0000","last_became_active":"2020-11-16T22:13:59.942771+0000","last_became_peered":"2020-11-16T22:13:59.942771+0000","last_unstale":"2020-11-16T22:14:07.557158+0000","last_undegraded":"2020-11-16T22:14:07.557158+0000","last_fullsized":"2020-11-16T22:14:07.557158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"22'21","reported_seq":"31","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.611811+0000","last_change":"2020-11-16T22:14:07.194335+0000","last_active":"2020-11-16T22:14:10.611811+0000","last_peered":"2020-11-16T22:14:10.611811+0000","last_clean":"2020-11-16T22:14:10.611811+0000","last_became_active":"2020-11-16T22:14:07.193814+0000","last_became_peered":"2020-11-16T22:14:07.193814+0000","last_unstale":"2020-11-16T22:14:10.611811+0000","last_undegraded":"2020-11-16T22:14:10.611811+0000","last_fullsized":"2020-11-16T22:14:10.611811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":112,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.571074+0000","last_change":"2020-11-16T22:14:07.566129+0000","last_active":"2020-11-16T22:14:07.571074+0000","last_peered":"2020-11-16T22:14:07.571074+0000","last_clean":"2020-11-16T22:14:07.571074+0000","last_became_active":"2020-11-16T22:14:07.565783+0000","last_became_peered":"2020-11-16T22:14:07.565783+0000","last_unstale":"2020-11-16T22:14:07.571074+0000","last_undegraded":"2020-11-16T22:14:07.571074+0000","last_fullsized":"2020-11-16T22:14:07.571074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557441+0000","last_change":"2020-11-16T22:14:00.734870+0000","last_active":"2020-11-16T22:14:07.557441+0000","last_peered":"2020-11-16T22:14:07.557441+0000","last_clean":"2020-11-16T22:14:07.557441+0000","last_became_active":"2020-11-16T22:14:00.733576+0000","last_became_peered":"2020-11-16T22:14:00.733576+0000","last_unstale":"2020-11-16T22:14:07.557441+0000","last_undegraded":"2020-11-16T22:14:07.557441+0000","last_fullsized":"2020-11-16T22:14:07.557441+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"15","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.552384+0000","last_change":"2020-11-16T22:13:55.098170+0000","last_active":"2020-11-16T22:14:07.552384+0000","last_peered":"2020-11-16T22:14:07.552384+0000","last_clean":"2020-11-16T22:14:07.552384+0000","last_became_active":"2020-11-16T22:13:55.097158+0000","last_became_peered":"2020-11-16T22:13:55.097158+0000","last_unstale":"2020-11-16T22:14:07.552384+0000","last_undegraded":"2020-11-16T22:14:07.552384+0000","last_fullsized":"2020-11-16T22:14:07.552384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:52.038457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:52.038457+0000","last_clean_scrub_stamp":"2020-11-16T22:13:52.038457+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.609149+0000","last_change":"2020-11-16T22:14:07.196200+0000","last_active":"2020-11-16T22:14:10.609149+0000","last_peered":"2020-11-16T22:14:10.609149+0000","last_clean":"2020-11-16T22:14:10.609149+0000","last_became_active":"2020-11-16T22:14:07.195690+0000","last_became_peered":"2020-11-16T22:14:07.195690+0000","last_unstale":"2020-11-16T22:14:10.609149+0000","last_undegraded":"2020-11-16T22:14:10.609149+0000","last_fullsized":"2020-11-16T22:14:10.609149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.568990+0000","last_change":"2020-11-16T22:14:07.566447+0000","last_active":"2020-11-16T22:14:07.568990+0000","last_peered":"2020-11-16T22:14:07.568990+0000","last_clean":"2020-11-16T22:14:07.568990+0000","last_became_active":"2020-11-16T22:14:07.565200+0000","last_became_peered":"2020-11-16T22:14:07.565200+0000","last_unstale":"2020-11-16T22:14:07.568990+0000","last_undegraded":"2020-11-16T22:14:07.568990+0000","last_fullsized":"2020-11-16T22:14:07.568990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557551+0000","last_change":"2020-11-16T22:13:59.947294+0000","last_active":"2020-11-16T22:14:07.557551+0000","last_peered":"2020-11-16T22:14:07.557551+0000","last_clean":"2020-11-16T22:14:07.557551+0000","last_became_active":"2020-11-16T22:13:59.946791+0000","last_became_peered":"2020-11-16T22:13:59.946791+0000","last_unstale":"2020-11-16T22:14:07.557551+0000","last_undegraded":"2020-11-16T22:14:07.557551+0000","last_fullsized":"2020-11-16T22:14:07.557551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"22'12","reported_seq":"22","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.613425+0000","last_change":"2020-11-16T22:14:07.197117+0000","last_active":"2020-11-16T22:14:10.613425+0000","last_peered":"2020-11-16T22:14:10.613425+0000","last_clean":"2020-11-16T22:14:10.613425+0000","last_became_active":"2020-11-16T22:14:07.196668+0000","last_became_peered":"2020-11-16T22:14:07.196668+0000","last_unstale":"2020-11-16T22:14:10.613425+0000","last_undegraded":"2020-11-16T22:14:10.613425+0000","last_fullsized":"2020-11-16T22:14:10.613425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":720,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579638+0000","last_change":"2020-11-16T22:14:07.572538+0000","last_active":"2020-11-16T22:14:07.579638+0000","last_peered":"2020-11-16T22:14:07.579638+0000","last_clean":"2020-11-16T22:14:07.579638+0000","last_became_active":"2020-11-16T22:14:07.571320+0000","last_became_peered":"2020-11-16T22:14:07.571320+0000","last_unstale":"2020-11-16T22:14:07.579638+0000","last_undegraded":"2020-11-16T22:14:07.579638+0000","last_fullsized":"2020-11-16T22:14:07.579638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557560+0000","last_change":"2020-11-16T22:13:59.946679+0000","last_active":"2020-11-16T22:14:07.557560+0000","last_peered":"2020-11-16T22:14:07.557560+0000","last_clean":"2020-11-16T22:14:07.557560+0000","last_became_active":"2020-11-16T22:13:59.946248+0000","last_became_peered":"2020-11-16T22:13:59.946248+0000","last_unstale":"2020-11-16T22:14:07.557560+0000","last_undegraded":"2020-11-16T22:14:07.557560+0000","last_fullsized":"2020-11-16T22:14:07.557560+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"22'13","reported_seq":"23","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.609944+0000","last_change":"2020-11-16T22:14:07.194904+0000","last_active":"2020-11-16T22:14:10.609944+0000","last_peered":"2020-11-16T22:14:10.609944+0000","last_clean":"2020-11-16T22:14:10.609944+0000","last_became_active":"2020-11-16T22:14:07.194443+0000","last_became_peered":"2020-11-16T22:14:07.194443+0000","last_unstale":"2020-11-16T22:14:10.609944+0000","last_undegraded":"2020-11-16T22:14:10.609944+0000","last_fullsized":"2020-11-16T22:14:10.609944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2088,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.579498+0000","last_change":"2020-11-16T22:14:07.574655+0000","last_active":"2020-11-16T22:14:07.579498+0000","last_peered":"2020-11-16T22:14:07.579498+0000","last_clean":"2020-11-16T22:14:07.579498+0000","last_became_active":"2020-11-16T22:14:07.574038+0000","last_became_peered":"2020-11-16T22:14:07.574038+0000","last_unstale":"2020-11-16T22:14:07.579498+0000","last_undegraded":"2020-11-16T22:14:07.579498+0000","last_fullsized":"2020-11-16T22:14:07.579498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'17","reported_seq":"27","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.613164+0000","last_change":"2020-11-16T22:14:07.193233+0000","last_active":"2020-11-16T22:14:10.613164+0000","last_peered":"2020-11-16T22:14:10.613164+0000","last_clean":"2020-11-16T22:14:10.613164+0000","last_became_active":"2020-11-16T22:14:07.192772+0000","last_became_peered":"2020-11-16T22:14:07.192772+0000","last_unstale":"2020-11-16T22:14:10.613164+0000","last_undegraded":"2020-11-16T22:14:10.613164+0000","last_fullsized":"2020-11-16T22:14:10.613164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":665,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.562278+0000","last_change":"2020-11-16T22:13:59.946652+0000","last_active":"2020-11-16T22:14:07.562278+0000","last_peered":"2020-11-16T22:14:07.562278+0000","last_clean":"2020-11-16T22:14:07.562278+0000","last_became_active":"2020-11-16T22:13:59.946109+0000","last_became_peered":"2020-11-16T22:13:59.946109+0000","last_unstale":"2020-11-16T22:14:07.562278+0000","last_undegraded":"2020-11-16T22:14:07.562278+0000","last_fullsized":"2020-11-16T22:14:07.562278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.563643+0000","last_change":"2020-11-16T22:14:07.562465+0000","last_active":"2020-11-16T22:14:07.563643+0000","last_peered":"2020-11-16T22:14:07.563643+0000","last_clean":"2020-11-16T22:14:07.563643+0000","last_became_active":"2020-11-16T22:14:07.562029+0000","last_became_peered":"2020-11-16T22:14:07.562029+0000","last_unstale":"2020-11-16T22:14:07.563643+0000","last_undegraded":"2020-11-16T22:14:07.563643+0000","last_fullsized":"2020-11-16T22:14:07.563643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:07.173670+0000","last_clean_scrub_stamp":"2020-11-16T22:14:07.173670+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-11-16T22:14:07.557775+0000","last_change":"2020-11-16T22:13:59.944154+0000","last_active":"2020-11-16T22:14:07.557775+0000","last_peered":"2020-11-16T22:14:07.557775+0000","last_clean":"2020-11-16T22:14:07.557775+0000","last_became_active":"2020-11-16T22:13:59.943659+0000","last_became_peered":"2020-11-16T22:13:59.943659+0000","last_unstale":"2020-11-16T22:14:07.557775+0000","last_undegraded":"2020-11-16T22:14:07.557775+0000","last_fullsized":"2020-11-16T22:14:07.557775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:13:58.919778+0000","last_clean_scrub_stamp":"2020-11-16T22:13:58.919778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'9","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-16T22:14:10.608412+0000","last_change":"2020-11-16T22:14:07.196926+0000","last_active":"2020-11-16T22:14:10.608412+0000","last_peered":"2020-11-16T22:14:10.608412+0000","last_clean":"2020-11-16T22:14:10.608412+0000","last_became_active":"2020-11-16T22:14:07.196552+0000","last_became_peered":"2020-11-16T22:14:07.196552+0000","last_unstale":"2020-11-16T22:14:10.608412+0000","last_undegraded":"2020-11-16T22:14:10.608412+0000","last_fullsized":"2020-11-16T22:14:10.608412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T22:14:06.164001+0000","last_clean_scrub_stamp":"2020-11-16T22:14:06.164001+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":4962,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":9924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":86097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.931}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3730000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.25}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5216,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366624,"statfs":{"total":96636764160,"available":96631422976,"internally_reserved":0,"allocated":286720,"data_stored":90193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.728}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.488}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5032,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366808,"statfs":{"total":96636764160,"available":96631611392,"internally_reserved":0,"allocated":229376,"data_stored":85985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2029999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1040000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":86627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.917}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5590000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6219999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.903}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5204,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366636,"statfs":{"total":96636764160,"available":96631435264,"internally_reserved":0,"allocated":274432,"data_stored":88082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2810000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.206}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5096,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366744,"statfs":{"total":96636764160,"available":96631545856,"internally_reserved":0,"allocated":229376,"data_stored":85985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2189999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3889999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.444}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0979999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":241664,"data_stored":86650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7189999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":253952,"data_stored":88185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.177}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8319999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2360000000000002}]}]}]}} 2020-11-16T22:14:23.335 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T22:14:23.336 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-16T22:14:23.336 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-16T22:14:23.336 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-16T22:14:23.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 Processor -- start 2020-11-16T22:14:23.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- start start 2020-11-16T22:14:23.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7640b1fa0 0x7fd764147970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7640b2a30 con 0x7fd7640680a0 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7640b24f0 con 0x7fd7640b1fa0 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640b2790 con 0x7fd7640b0f70 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49756/0 (socket says 172.21.15.185:49756) 2020-11-16T22:14:23.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 -- 172.21.15.185:0/2187103419 learned_addr learned my addr 172.21.15.185:0/2187103419 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:23.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd762ffd700 1 --2- 172.21.15.185:0/2187103419 >> v2:172.21.15.96:3300/0 conn(0x7fd7640b1fa0 0x7fd764147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 -- 172.21.15.185:0/2187103419 >> v2:172.21.15.96:3300/0 conn(0x7fd7640b1fa0 msgr2=0x7fd764147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:23.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 --2- 172.21.15.185:0/2187103419 >> v2:172.21.15.96:3300/0 conn(0x7fd7640b1fa0 0x7fd764147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 -- 172.21.15.185:0/2187103419 >> v1:172.21.15.96:6789/0 conn(0x7fd7640680a0 legacy=0x7fd76414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 -- 172.21.15.185:0/2187103419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640b6420 con 0x7fd7640b0f70 2020-11-16T22:14:23.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd763fff700 1 --2- 172.21.15.185:0/2187103419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd75c006fd0 tx=0x7fd75c006eb0).ready entity=mon.2 client_cookie=867f69041393bca9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd7627fc700 1 -- 172.21.15.185:0/2187103419 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c009810 con 0x7fd7640b0f70 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd7627fc700 1 -- 172.21.15.185:0/2187103419 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c009970 con 0x7fd7640b0f70 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd7627fc700 1 -- 172.21.15.185:0/2187103419 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c009cf0 con 0x7fd7640b0f70 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- 172.21.15.185:0/2187103419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 msgr2=0x7fd764147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2187103419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd75c006fd0 tx=0x7fd75c006eb0).stop 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- 172.21.15.185:0/2187103419 shutdown_connections 2020-11-16T22:14:23.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2187103419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b0f70 0x7fd764147e60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2187103419 >> v2:172.21.15.96:3300/0 conn(0x7fd7640b1fa0 0x7fd764147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- 172.21.15.185:0/2187103419 >> 172.21.15.185:0/2187103419 conn(0x7fd7640657c0 msgr2=0x7fd764065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:23.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.679+0000 7fd768fd8700 1 -- 172.21.15.185:0/2187103419 shutdown_connections 2020-11-16T22:14:23.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.683+0000 7fd768fd8700 1 -- 172.21.15.185:0/2187103419 wait complete. 2020-11-16T22:14:23.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.683+0000 7fd768fd8700 1 Processor -- start 2020-11-16T22:14:23.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.683+0000 7fd768fd8700 1 -- start start 2020-11-16T22:14:23.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 0x7fd7641432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7640b0f70 0x7fd76414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd764141cc0 con 0x7fd7640680a0 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd764141f60 con 0x7fd7640b0f70 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd764142200 con 0x7fd7640b1fa0 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd762ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7640b0f70 0x7fd76414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49762/0 (socket says 172.21.15.185:49762) 2020-11-16T22:14:23.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 -- 172.21.15.185:0/2739534613 learned_addr learned my addr 172.21.15.185:0/2739534613 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:23.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd7637fe700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 0x7fd7641432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 -- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7640b0f70 msgr2=0x7fd76414d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7640b0f70 0x7fd76414d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 -- 172.21.15.185:0/2739534613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 msgr2=0x7fd7641432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 0x7fd7641432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 -- 172.21.15.185:0/2739534613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7641424a0 con 0x7fd7640b1fa0 2020-11-16T22:14:23.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd7637fe700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 0x7fd7641432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd763fff700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd75c0069f0 tx=0x7fd75c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c009810 con 0x7fd7640b1fa0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c006760 con 0x7fd7640b1fa0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd764142940 con 0x7fd7640b1fa0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c01d800 con 0x7fd7640b1fa0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd764143970 con 0x7fd7640b1fa0 2020-11-16T22:14:23.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd764142630 con 0x7fd7640b1fa0 2020-11-16T22:14:23.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.687+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd75c025030 con 0x7fd7640b1fa0 2020-11-16T22:14:23.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.691+0000 7fd75b7fe700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 0x7fd740057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:23.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.691+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd75c06f5d0 con 0x7fd7640b1fa0 2020-11-16T22:14:23.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.691+0000 7fd7637fe700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 0x7fd740057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:23.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.691+0000 7fd7637fe700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 0x7fd740057520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd754007fd0 tx=0x7fd754009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:23.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.695+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd75c0457d0 con 0x7fd7640b1fa0 2020-11-16T22:14:23.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.903+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd76405d200 con 0x7fd7640b1fa0 2020-11-16T22:14:23.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd75b7fe700 1 -- 172.21.15.185:0/2739534613 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fd75c023020 con 0x7fd7640b1fa0 2020-11-16T22:14:23.893 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:23.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 msgr2=0x7fd740057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 0x7fd740057520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd754007fd0 tx=0x7fd754009040).stop 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 msgr2=0x7fd764149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd75c0069f0 tx=0x7fd75c015000).stop 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 shutdown_connections 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd740055060 0x7fd740057520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7640b1fa0 0x7fd764149c60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7640680a0 0x7fd7641432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 --2- 172.21.15.185:0/2739534613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7640b0f70 0x7fd76414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 >> 172.21.15.185:0/2739534613 conn(0x7fd7640657c0 msgr2=0x7fd7640609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 shutdown_connections 2020-11-16T22:14:23.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:23.907+0000 7fd768fd8700 1 -- 172.21.15.185:0/2739534613 wait complete. 2020-11-16T22:14:23.907 INFO:teuthology.orchestra.run.smithi185.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-16T22:14:23.907 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-16T22:14:23.908 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 Processor -- start 2020-11-16T22:14:24.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- start start 2020-11-16T22:14:24.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f67b809b270 0x7f67b809cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b809d410 con 0x7f67b8144840 2020-11-16T22:14:24.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b809b010 con 0x7f67b809c900 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b8144cf0 con 0x7f67b809b270 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b67fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f67b809b270 0x7f67b809cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b6ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f67b809c900 0x7f67b81476e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49760/0 (socket says 172.21.15.185:49760) 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b6ffd700 1 -- 172.21.15.185:0/615440535 learned_addr learned my addr 172.21.15.185:0/615440535 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b5ffb700 1 -- 172.21.15.185:0/615440535 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426579126 0 0) 0x7f67b809b010 con 0x7f67b809c900 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b5ffb700 1 -- 172.21.15.185:0/615440535 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6798003240 con 0x7f67b809c900 2020-11-16T22:14:24.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b67fc700 1 -- 172.21.15.185:0/615440535 >> v1:172.21.15.185:6789/0 conn(0x7f67b809c900 legacy=0x7f67b81476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b67fc700 1 -- 172.21.15.185:0/615440535 >> v1:172.21.15.96:6789/0 conn(0x7f67b8144840 legacy=0x7f67b8147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b67fc700 1 -- 172.21.15.185:0/615440535 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8148500 con 0x7f67b809b270 2020-11-16T22:14:24.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b67fc700 1 --2- 172.21.15.185:0/615440535 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b809b270 0x7f67b809cce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f67a000b9a0 tx=0x7f67a0006df0).ready entity=mon.1 client_cookie=53d41f54475aa883 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b5ffb700 1 -- 172.21.15.185:0/615440535 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00097b0 con 0x7f67b809b270 2020-11-16T22:14:24.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b5ffb700 1 -- 172.21.15.185:0/615440535 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0009910 con 0x7f67b809b270 2020-11-16T22:14:24.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67b5ffb700 1 -- 172.21.15.185:0/615440535 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0009c90 con 0x7f67b809b270 2020-11-16T22:14:24.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- 172.21.15.185:0/615440535 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b809b270 msgr2=0x7f67b809cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 --2- 172.21.15.185:0/615440535 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b809b270 0x7f67b809cce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f67a000b9a0 tx=0x7f67a0006df0).stop 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- 172.21.15.185:0/615440535 shutdown_connections 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 --2- 172.21.15.185:0/615440535 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b809b270 0x7f67b809cce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- 172.21.15.185:0/615440535 >> 172.21.15.185:0/615440535 conn(0x7f67b8068330 msgr2=0x7f67b80ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- 172.21.15.185:0/615440535 shutdown_connections 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.047+0000 7f67bd8af700 1 -- 172.21.15.185:0/615440535 wait complete. 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 Processor -- start 2020-11-16T22:14:24.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 -- start start 2020-11-16T22:14:24.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 0x7f67b81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b81533d0 con 0x7f67b809b270 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8153e30 con 0x7f67b8144840 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80b4d70 con 0x7f67b809c900 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56332/0 (socket says 172.21.15.185:56332) 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49774/0 (socket says 172.21.15.185:49774) 2020-11-16T22:14:24.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 -- 172.21.15.185:0/3164838271 learned_addr learned my addr 172.21.15.185:0/3164838271 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b6ffd700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 0x7f67b81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 -- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 msgr2=0x7f67b8152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 -- 172.21.15.185:0/3164838271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 msgr2=0x7f67b81529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 0x7f67b81529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 -- 172.21.15.185:0/3164838271 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80a04b0 con 0x7f67b8144840 2020-11-16T22:14:24.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b6ffd700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 0x7f67b81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b67fc700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67b77fe700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f67a800c740 tx=0x7f67a8007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800e560 con 0x7f67b8144840 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800e6c0 con 0x7f67b8144840 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b81542a0 con 0x7f67b8144840 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.051+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800ea40 con 0x7f67b8144840 2020-11-16T22:14:24.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b809f4c0 con 0x7f67b8144840 2020-11-16T22:14:24.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b805d4e0 con 0x7f67b8144840 2020-11-16T22:14:24.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f67a8019030 con 0x7f67b8144840 2020-11-16T22:14:24.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67aeffd700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 0x7f6794057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67b6ffd700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 0x7f6794057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67a806d0f0 con 0x7f67b8144840 2020-11-16T22:14:24.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.055+0000 7f67b6ffd700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 0x7f6794057520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f67b0005920 tx=0x7f67b0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.063+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f67a8043400 con 0x7f67b8144840 2020-11-16T22:14:24.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.215+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67b80b4f30 con 0x7f67b8144840 2020-11-16T22:14:24.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67aeffd700 1 -- 172.21.15.185:0/3164838271 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f67a80251d0 con 0x7f67b8144840 2020-11-16T22:14:24.206 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:24.206 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:24.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 msgr2=0x7f6794057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 0x7f6794057520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f67b0005920 tx=0x7f67b0009040).stop 2020-11-16T22:14:24.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 msgr2=0x7f67b8156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f67a800c740 tx=0x7f67a8007b30).stop 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 shutdown_connections 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6794055060 0x7f6794057520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67b8144840 0x7f67b8156a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b809b270 0x7f67b81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 --2- 172.21.15.185:0/3164838271 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67b809c900 0x7f67b8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 >> 172.21.15.185:0/3164838271 conn(0x7f67b8068330 msgr2=0x7f67b8062670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 shutdown_connections 2020-11-16T22:14:24.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.219+0000 7f67bd8af700 1 -- 172.21.15.185:0/3164838271 wait complete. 2020-11-16T22:14:24.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:24.222 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:24.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 Processor -- start 2020-11-16T22:14:24.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 -- start start 2020-11-16T22:14:24.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa39c0b4470 0x7fa39c143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c0b7290 con 0x7fa39c0ba800 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c0b4bf0 con 0x7fa39c0b5d70 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39bfff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c0b74c0 con 0x7fa39c0b4470 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa39c0b4470 0x7fa39c143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39affd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa39c0b5d70 0x7fa39c0b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49776/0 (socket says 172.21.15.185:49776) 2020-11-16T22:14:24.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39affd700 1 -- 172.21.15.185:0/4003627528 learned_addr learned my addr 172.21.15.185:0/4003627528 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa39c0b4470 0x7fa39c143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56340/0 (socket says 172.21.15.185:56340) 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 -- 172.21.15.185:0/4003627528 >> v1:172.21.15.185:6789/0 conn(0x7fa39c0b5d70 legacy=0x7fa39c0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa399ffb700 1 -- 172.21.15.185:0/4003627528 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916635057 0 0) 0x7fa39c0b4bf0 con 0x7fa39c0b5d70 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 -- 172.21.15.185:0/4003627528 >> v1:172.21.15.96:6789/0 conn(0x7fa39c0ba800 legacy=0x7fa39c143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 -- 172.21.15.185:0/4003627528 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0b7760 con 0x7fa39c0b4470 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.367+0000 7fa39b7fe700 1 --2- 172.21.15.185:0/4003627528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b4470 0x7fa39c143b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa394009c90 tx=0x7fa3940091d0).ready entity=mon.1 client_cookie=8979482ecd01017f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa399ffb700 1 -- 172.21.15.185:0/4003627528 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400bbc0 con 0x7fa39c0b4470 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa399ffb700 1 -- 172.21.15.185:0/4003627528 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39400bd20 con 0x7fa39c0b4470 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa399ffb700 1 -- 172.21.15.185:0/4003627528 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394011610 con 0x7fa39c0b4470 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- 172.21.15.185:0/4003627528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b4470 msgr2=0x7fa39c143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 --2- 172.21.15.185:0/4003627528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b4470 0x7fa39c143b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa394009c90 tx=0x7fa3940091d0).stop 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- 172.21.15.185:0/4003627528 shutdown_connections 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 --2- 172.21.15.185:0/4003627528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b4470 0x7fa39c143b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- 172.21.15.185:0/4003627528 >> 172.21.15.185:0/4003627528 conn(0x7fa39c05ba80 msgr2=0x7fa39c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- 172.21.15.185:0/4003627528 shutdown_connections 2020-11-16T22:14:24.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- 172.21.15.185:0/4003627528 wait complete. 2020-11-16T22:14:24.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 Processor -- start 2020-11-16T22:14:24.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- start start 2020-11-16T22:14:24.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39c0ba800 0x7fa39c1421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c115b40 con 0x7fa39c0ba800 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0ae980 con 0x7fa39c0b5d70 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0afe10 con 0x7fa39c0b4470 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56346/0 (socket says 172.21.15.185:56346) 2020-11-16T22:14:24.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39a7fc700 1 -- 172.21.15.185:0/2477464394 learned_addr learned my addr 172.21.15.185:0/2477464394 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49786/0 (socket says 172.21.15.185:49786) 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.371+0000 7fa39b7fe700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39c0ba800 0x7fa39c1421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 -- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 msgr2=0x7fa39c115650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 -- 172.21.15.185:0/2477464394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39c0ba800 msgr2=0x7fa39c1421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39c0ba800 0x7fa39c1421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 -- 172.21.15.185:0/2477464394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c157540 con 0x7fa39c0b4470 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39a7fc700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39affd700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa38800c710 tx=0x7fa388007900).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3880072e0 con 0x7fa39c0b4470 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388007440 con 0x7fa39c0b4470 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388005840 con 0x7fa39c0b4470 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c1577e0 con 0x7fa39c0b4470 2020-11-16T22:14:24.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c0af640 con 0x7fa39c0b4470 2020-11-16T22:14:24.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa388010070 con 0x7fa39c0b4470 2020-11-16T22:14:24.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c0af920 con 0x7fa39c0b4470 2020-11-16T22:14:24.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 0x7fa370057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39a7fc700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 0x7fa370057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa38806d290 con 0x7fa39c0b4470 2020-11-16T22:14:24.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.375+0000 7fa39a7fc700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 0x7fa370057520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fa39c0aef60 tx=0x7fa38c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.379+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa3880434f0 con 0x7fa39c0b4470 2020-11-16T22:14:24.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.531+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa39c05d200 con 0x7fa39c0b4470 2020-11-16T22:14:24.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.531+0000 7fa392ffd700 1 -- 172.21.15.185:0/2477464394 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa388046b10 con 0x7fa39c0b4470 2020-11-16T22:14:24.520 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 msgr2=0x7fa370057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 0x7fa370057520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fa39c0aef60 tx=0x7fa38c009040).stop 2020-11-16T22:14:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 msgr2=0x7fa39c115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa38800c710 tx=0x7fa388007900).stop 2020-11-16T22:14:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 shutdown_connections 2020-11-16T22:14:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa370055060 0x7fa370057520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39c0ba800 0x7fa39c1421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa39c0b4470 0x7fa39c115160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 --2- 172.21.15.185:0/2477464394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa39c0b5d70 0x7fa39c115650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 >> 172.21.15.185:0/2477464394 conn(0x7fa39c05ba80 msgr2=0x7fa39c0b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 shutdown_connections 2020-11-16T22:14:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.535+0000 7fa39bfff700 1 -- 172.21.15.185:0/2477464394 wait complete. 2020-11-16T22:14:24.534 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:24.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 Processor -- start 2020-11-16T22:14:24.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 -- start start 2020-11-16T22:14:24.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4ccc0afc60 0x7f4ccc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ccc144860 0x7f4ccc0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc0ad490 con 0x7f4ccc0b7440 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0ad730 con 0x7f4ccc0afc60 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4cd189f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0ad9d0 con 0x7f4ccc144860 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4ccb7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ccc144860 0x7f4ccc0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4ccb7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ccc144860 0x7f4ccc0ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56354/0 (socket says 172.21.15.185:56354) 2020-11-16T22:14:24.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.675+0000 7f4ccb7fe700 1 -- 172.21.15.185:0/2887821769 learned_addr learned my addr 172.21.15.185:0/2887821769 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cca7fc700 1 --2- 172.21.15.185:0/2887821769 >> v2:172.21.15.96:3300/0 conn(0x7f4ccc0afc60 0x7f4ccc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4ccb7fe700 1 -- 172.21.15.185:0/2887821769 >> v2:172.21.15.96:3300/0 conn(0x7f4ccc0afc60 msgr2=0x7f4ccc0b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:24.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4ccb7fe700 1 --2- 172.21.15.185:0/2887821769 >> v2:172.21.15.96:3300/0 conn(0x7f4ccc0afc60 0x7f4ccc0b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4ccb7fe700 1 -- 172.21.15.185:0/2887821769 >> v1:172.21.15.96:6789/0 conn(0x7f4ccc0b7440 legacy=0x7f4ccc13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4ccb7fe700 1 -- 172.21.15.185:0/2887821769 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc13e5b0 con 0x7f4ccc144860 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4ccb7fe700 1 --2- 172.21.15.185:0/2887821769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc0ade70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4ccc0b5290 tx=0x7f4cc4009290).ready entity=mon.1 client_cookie=76ee594dd89a7cc9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cc9ffb700 1 -- 172.21.15.185:0/2887821769 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400bbc0 con 0x7f4ccc144860 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cc9ffb700 1 -- 172.21.15.185:0/2887821769 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400bd20 con 0x7f4ccc144860 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cc9ffb700 1 -- 172.21.15.185:0/2887821769 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4011610 con 0x7f4ccc144860 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 -- 172.21.15.185:0/2887821769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 msgr2=0x7f4ccc0ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 --2- 172.21.15.185:0/2887821769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc0ade70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4ccc0b5290 tx=0x7f4cc4009290).stop 2020-11-16T22:14:24.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 -- 172.21.15.185:0/2887821769 shutdown_connections 2020-11-16T22:14:24.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 --2- 172.21.15.185:0/2887821769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc0ade70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 --2- 172.21.15.185:0/2887821769 >> v2:172.21.15.96:3300/0 conn(0x7f4ccc0afc60 0x7f4ccc0b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 -- 172.21.15.185:0/2887821769 >> 172.21.15.185:0/2887821769 conn(0x7f4ccc067860 msgr2=0x7f4ccc067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 -- 172.21.15.185:0/2887821769 shutdown_connections 2020-11-16T22:14:24.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.679+0000 7f4cd189f700 1 -- 172.21.15.185:0/2887821769 wait complete. 2020-11-16T22:14:24.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 Processor -- start 2020-11-16T22:14:24.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- start start 2020-11-16T22:14:24.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 0x7f4ccc0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0b2950 con 0x7f4ccc0afc60 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0b2b40 con 0x7f4ccc144860 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0b2de0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49802/0 (socket says 172.21.15.185:49802) 2020-11-16T22:14:24.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 -- 172.21.15.185:0/4166676316 learned_addr learned my addr 172.21.15.185:0/4166676316 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4ccb7fe700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4ccaffd700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 0x7f4ccc0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 -- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 msgr2=0x7f4ccc146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 -- 172.21.15.185:0/4166676316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 msgr2=0x7f4ccc0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 0x7f4ccc0b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 -- 172.21.15.185:0/4166676316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc0b0ae0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4ccaffd700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 0x7f4ccc0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cca7fc700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4cbc00ebd0 tx=0x7f4cbc006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00b3a0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc00c9b0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc147350 con 0x7f4ccc0b7440 2020-11-16T22:14:24.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00cd70 con 0x7f4ccc0b7440 2020-11-16T22:14:24.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc0b0f80 con 0x7f4ccc0b7440 2020-11-16T22:14:24.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.683+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc0b2fa0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.687+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4cbc012600 con 0x7f4ccc0b7440 2020-11-16T22:14:24.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.687+0000 7f4cc2ffd700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 0x7f4ca8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.687+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4cbc06e120 con 0x7f4ccc0b7440 2020-11-16T22:14:24.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.687+0000 7f4ccaffd700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 0x7f4ca8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.687+0000 7f4ccaffd700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 0x7f4ca8057520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4ccc0b1fa0 tx=0x7f4cb4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.691+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4cbc044380 con 0x7f4ccc0b7440 2020-11-16T22:14:24.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ccc05d4e0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cc2ffd700 1 -- 172.21.15.185:0/4166676316 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4cbc0479a0 con 0x7f4ccc0b7440 2020-11-16T22:14:24.830 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:24.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 msgr2=0x7f4ca8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 0x7f4ca8057520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4ccc0b1fa0 tx=0x7f4cb4009040).stop 2020-11-16T22:14:24.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 msgr2=0x7f4ccc153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4cbc00ebd0 tx=0x7f4cbc006df0).stop 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 shutdown_connections 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ca8055060 0x7f4ca8057520 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ccc144860 0x7f4ccc146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ccc0afc60 0x7f4ccc0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 --2- 172.21.15.185:0/4166676316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ccc0b7440 0x7f4ccc153ec0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 >> 172.21.15.185:0/4166676316 conn(0x7f4ccc067860 msgr2=0x7f4ccc0ba080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 shutdown_connections 2020-11-16T22:14:24.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.843+0000 7f4cd189f700 1 -- 172.21.15.185:0/4166676316 wait complete. 2020-11-16T22:14:24.844 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:24.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.983+0000 7f5078c6a700 1 Processor -- start 2020-11-16T22:14:24.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.983+0000 7f5078c6a700 1 -- start start 2020-11-16T22:14:24.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50740b34d0 con 0x7f50740b76a0 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50740672d0 con 0x7f5074144c50 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074061070 con 0x7f5074060850 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49808/0 (socket says 172.21.15.185:49808) 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 -- 172.21.15.185:0/1022121896 learned_addr learned my addr 172.21.15.185:0/1022121896 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 -- 172.21.15.185:0/1022121896 >> v1:172.21.15.185:6789/0 conn(0x7f5074144c50 legacy=0x7f50740679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 -- 172.21.15.185:0/1022121896 >> v1:172.21.15.96:6789/0 conn(0x7f50740b76a0 legacy=0x7f507413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:24.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 -- 172.21.15.185:0/1022121896 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50741450e0 con 0x7f5074060850 2020-11-16T22:14:24.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f506bfff700 1 --2- 172.21.15.185:0/1022121896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f506400ba20 tx=0x7f5064006eb0).ready entity=mon.2 client_cookie=52afc190d6dc8254 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5071d9b700 1 -- 172.21.15.185:0/1022121896 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064009890 con 0x7f5074060850 2020-11-16T22:14:24.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5071d9b700 1 -- 172.21.15.185:0/1022121896 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50640099f0 con 0x7f5074060850 2020-11-16T22:14:24.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5071d9b700 1 -- 172.21.15.185:0/1022121896 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064009d70 con 0x7f5074060850 2020-11-16T22:14:24.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- 172.21.15.185:0/1022121896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 msgr2=0x7f50740b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 --2- 172.21.15.185:0/1022121896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f506400ba20 tx=0x7f5064006eb0).stop 2020-11-16T22:14:24.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- 172.21.15.185:0/1022121896 shutdown_connections 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 --2- 172.21.15.185:0/1022121896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5074060850 0x7f50740b7a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- 172.21.15.185:0/1022121896 >> 172.21.15.185:0/1022121896 conn(0x7f5074062d50 msgr2=0x7f5074063130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- 172.21.15.185:0/1022121896 shutdown_connections 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 -- 172.21.15.185:0/1022121896 wait complete. 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.987+0000 7f5078c6a700 1 Processor -- start 2020-11-16T22:14:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- start start 2020-11-16T22:14:24.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 0x7f5074153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 0x7f50741400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507413fca0 con 0x7f5074060850 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074153f90 con 0x7f5074144c50 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740b0710 con 0x7f50740b76a0 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 0x7f50741400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49816/0 (socket says 172.21.15.185:49816) 2020-11-16T22:14:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506bfff700 1 -- 172.21.15.185:0/2082992409 learned_addr learned my addr 172.21.15.185:0/2082992409 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f507259c700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 0x7f5074153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 -- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 msgr2=0x7f50740b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 -- 172.21.15.185:0/2082992409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 msgr2=0x7f5074153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 0x7f5074153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 -- 172.21.15.185:0/2082992409 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740b0970 con 0x7f5074144c50 2020-11-16T22:14:24.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f507259c700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 0x7f5074153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5072d9d700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 0x7f50741400d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f506c00ca60 tx=0x7f506c007de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00a730 con 0x7f5074144c50 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c00a890 con 0x7f5074144c50 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- 172.21.15.185:0/2082992409 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074157b00 con 0x7f5074144c50 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00ac50 con 0x7f5074144c50 2020-11-16T22:14:24.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f5078c6a700 1 -- 172.21.15.185:0/2082992409 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50741405c0 con 0x7f5074144c50 2020-11-16T22:14:24.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.991+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507405d4e0 con 0x7f5074144c50 2020-11-16T22:14:24.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.995+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f506c019030 con 0x7f5074144c50 2020-11-16T22:14:24.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.995+0000 7f506a7fc700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 0x7f5050057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:24.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.995+0000 7f507259c700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 0x7f5050057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:24.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.995+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f506c003030 con 0x7f5074144c50 2020-11-16T22:14:24.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.995+0000 7f507259c700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 0x7f5050057520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f505c0021c0 tx=0x7f505c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:24.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:24.999+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f506c043600 con 0x7f5074144c50 2020-11-16T22:14:25.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f507413fde0 con 0x7f5074144c50 2020-11-16T22:14:25.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f506a7fc700 1 -- 172.21.15.185:0/2082992409 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f506c017020 con 0x7f5074144c50 2020-11-16T22:14:25.139 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:25.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 msgr2=0x7f5050057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 0x7f5050057520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f505c0021c0 tx=0x7f505c009040).stop 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 msgr2=0x7f50741400d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 0x7f50741400d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f506c00ca60 tx=0x7f506c007de0).stop 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 shutdown_connections 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5050055060 0x7f5050057520 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5074144c50 0x7f50741400d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5074060850 0x7f5074153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 --2- 172.21.15.185:0/2082992409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50740b76a0 0x7f50740b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.151+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 >> 172.21.15.185:0/2082992409 conn(0x7f5074062d50 msgr2=0x7f50740b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:25.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.155+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 shutdown_connections 2020-11-16T22:14:25.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.155+0000 7f504bfff700 1 -- 172.21.15.185:0/2082992409 wait complete. 2020-11-16T22:14:25.153 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:25.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 Processor -- start 2020-11-16T22:14:25.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 -- start start 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3b200b3890 0x7f3b200b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3b200adc40 0x7f3b200ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200ad620 con 0x7f3b201410d0 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b20141650 con 0x7f3b200b3890 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b2506f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b201418f0 con 0x7f3b200adc40 2020-11-16T22:14:25.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b1e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3b201410d0 0x7f3b200aca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49818/0 (socket says 172.21.15.185:49818) 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b1e59c700 1 -- 172.21.15.185:0/1051867110 learned_addr learned my addr 172.21.15.185:0/1051867110 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b1f59e700 1 --2- 172.21.15.185:0/1051867110 >> v2:172.21.15.185:3300/0 conn(0x7f3b200adc40 0x7f3b200ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.295+0000 7f3b1ed9d700 1 --2- 172.21.15.185:0/1051867110 >> v2:172.21.15.96:3300/0 conn(0x7f3b200b3890 0x7f3b200b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1dd9b700 1 -- 172.21.15.185:0/1051867110 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348012548 0 0) 0x7f3b200ad620 con 0x7f3b201410d0 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1dd9b700 1 -- 172.21.15.185:0/1051867110 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b0c003240 con 0x7f3b201410d0 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1f59e700 1 -- 172.21.15.185:0/1051867110 >> v2:172.21.15.96:3300/0 conn(0x7f3b200b3890 msgr2=0x7f3b200b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:25.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1f59e700 1 --2- 172.21.15.185:0/1051867110 >> v2:172.21.15.96:3300/0 conn(0x7f3b200b3890 0x7f3b200b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1f59e700 1 -- 172.21.15.185:0/1051867110 >> v1:172.21.15.185:6789/0 conn(0x7f3b201410d0 legacy=0x7f3b200aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1f59e700 1 -- 172.21.15.185:0/1051867110 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20141b90 con 0x7f3b200adc40 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1f59e700 1 --2- 172.21.15.185:0/1051867110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200ad130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3b2013b790 tx=0x7f3b180091d0).ready entity=mon.1 client_cookie=fa72e9a032c87cee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1dd9b700 1 -- 172.21.15.185:0/1051867110 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800b9c0 con 0x7f3b200adc40 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1dd9b700 1 -- 172.21.15.185:0/1051867110 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800bb20 con 0x7f3b200adc40 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b1dd9b700 1 -- 172.21.15.185:0/1051867110 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800bea0 con 0x7f3b200adc40 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 -- 172.21.15.185:0/1051867110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 msgr2=0x7f3b200ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1051867110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200ad130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3b2013b790 tx=0x7f3b180091d0).stop 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 -- 172.21.15.185:0/1051867110 shutdown_connections 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1051867110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200ad130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1051867110 >> v2:172.21.15.96:3300/0 conn(0x7f3b200b3890 0x7f3b200b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 -- 172.21.15.185:0/1051867110 >> 172.21.15.185:0/1051867110 conn(0x7f3b20063f60 msgr2=0x7f3b20064340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 -- 172.21.15.185:0/1051867110 shutdown_connections 2020-11-16T22:14:25.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.299+0000 7f3b2506f700 1 -- 172.21.15.185:0/1051867110 wait complete. 2020-11-16T22:14:25.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 Processor -- start 2020-11-16T22:14:25.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- start start 2020-11-16T22:14:25.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b200b3890 0x7f3b20150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 0x7f3b20150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201501a0 con 0x7f3b200b3890 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2013d120 con 0x7f3b200adc40 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20144a00 con 0x7f3b201410d0 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56384/0 (socket says 172.21.15.185:56384) 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 -- 172.21.15.185:0/1848298373 learned_addr learned my addr 172.21.15.185:0/1848298373 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1f59e700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 0x7f3b20150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1e59c700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b200b3890 0x7f3b20150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 -- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 msgr2=0x7f3b20150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 0x7f3b20150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 -- 172.21.15.185:0/1848298373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b200b3890 msgr2=0x7f3b20150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b200b3890 0x7f3b20150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 -- 172.21.15.185:0/1848298373 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20150390 con 0x7f3b200adc40 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1f59e700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 0x7f3b20150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b1ed9d700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3b1000eba0 tx=0x7f3b10006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000c9b0 con 0x7f3b200adc40 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1000cb10 con 0x7f3b200adc40 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2013ef50 con 0x7f3b200adc40 2020-11-16T22:14:25.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000ce90 con 0x7f3b200adc40 2020-11-16T22:14:25.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2013ce10 con 0x7f3b200adc40 2020-11-16T22:14:25.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b20150520 con 0x7f3b200adc40 2020-11-16T22:14:25.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3b10012600 con 0x7f3b200adc40 2020-11-16T22:14:25.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.303+0000 7f3b06ffd700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 0x7f3afc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.307+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b1006e200 con 0x7f3b200adc40 2020-11-16T22:14:25.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.307+0000 7f3b1e59c700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 0x7f3afc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.307+0000 7f3b1e59c700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 0x7f3afc057520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3b2013df60 tx=0x7f3b14009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.311+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3b10044460 con 0x7f3b200adc40 2020-11-16T22:14:25.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.467+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b2005d4e0 con 0x7f3b200adc40 2020-11-16T22:14:25.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.467+0000 7f3b06ffd700 1 -- 172.21.15.185:0/1848298373 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3b10047a80 con 0x7f3b200adc40 2020-11-16T22:14:25.455 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:25.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 msgr2=0x7f3afc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 0x7f3afc057520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3b2013df60 tx=0x7f3b14009040).stop 2020-11-16T22:14:25.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 msgr2=0x7f3b200b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3b1000eba0 tx=0x7f3b10006df0).stop 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 shutdown_connections 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3afc055060 0x7f3afc057520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b200adc40 0x7f3b200b0910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b201410d0 0x7f3b20150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 --2- 172.21.15.185:0/1848298373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b200b3890 0x7f3b20150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 >> 172.21.15.185:0/1848298373 conn(0x7f3b20063f60 msgr2=0x7f3b200b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:25.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 shutdown_connections 2020-11-16T22:14:25.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.471+0000 7f3b2506f700 1 -- 172.21.15.185:0/1848298373 wait complete. 2020-11-16T22:14:25.470 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:25.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 Processor -- start 2020-11-16T22:14:25.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 -- start start 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 0x7fb7780b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb7780b6aa0 0x7fb7780acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb778147f90 con 0x7fb7780b64b0 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780af020 con 0x7fb7780b6aa0 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.599+0000 7fb77f2c9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780b4810 con 0x7fb7780b72e0 2020-11-16T22:14:25.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77dac6700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb7780b64b0 0x7fb778144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49830/0 (socket says 172.21.15.185:49830) 2020-11-16T22:14:25.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77dac6700 1 -- 172.21.15.185:0/1167558050 learned_addr learned my addr 172.21.15.185:0/1167558050 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:25.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/1167558050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 0x7fb7780b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77eac8700 1 --2- 172.21.15.185:0/1167558050 >> v2:172.21.15.96:3300/0 conn(0x7fb7780b6aa0 0x7fb7780acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629955316 0 0) 0x7fb778147f90 con 0x7fb7780b64b0 2020-11-16T22:14:25.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb764003240 con 0x7fb7780b64b0 2020-11-16T22:14:25.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3700831700 0 0) 0x7fb764003240 con 0x7fb7780b64b0 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 -- 172.21.15.185:0/1167558050 >> v2:172.21.15.96:3300/0 conn(0x7fb7780b6aa0 msgr2=0x7fb7780acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/1167558050 >> v2:172.21.15.96:3300/0 conn(0x7fb7780b6aa0 0x7fb7780acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 -- 172.21.15.185:0/1167558050 >> v1:172.21.15.185:6789/0 conn(0x7fb7780b64b0 legacy=0x7fb778144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 -- 172.21.15.185:0/1167558050 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778148460 con 0x7fb7780b72e0 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/1167558050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 0x7fb7780b76c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb76800b990 tx=0x7fb768006df0).ready entity=mon.2 client_cookie=7056987bc9114352 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774001860 con 0x7fb7780b72e0 2020-11-16T22:14:25.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768011400 con 0x7fb7780b72e0 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77d2c5700 1 -- 172.21.15.185:0/1167558050 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768011780 con 0x7fb7780b72e0 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- 172.21.15.185:0/1167558050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 msgr2=0x7fb7780b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/1167558050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 0x7fb7780b76c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb76800b990 tx=0x7fb768006df0).stop 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- 172.21.15.185:0/1167558050 shutdown_connections 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/1167558050 >> v2:172.21.15.96:3300/0 conn(0x7fb7780b6aa0 0x7fb7780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/1167558050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b72e0 0x7fb7780b76c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- 172.21.15.185:0/1167558050 >> 172.21.15.185:0/1167558050 conn(0x7fb778067860 msgr2=0x7fb778067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:25.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- 172.21.15.185:0/1167558050 shutdown_connections 2020-11-16T22:14:25.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- 172.21.15.185:0/1167558050 wait complete. 2020-11-16T22:14:25.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 Processor -- start 2020-11-16T22:14:25.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- start start 2020-11-16T22:14:25.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b64b0 0x7fb778114e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 0x7fb778156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780b3460 con 0x7fb7780b6aa0 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7781572f0 con 0x7fb7780b72e0 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77f2c9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778157590 con 0x7fb7780b64b0 2020-11-16T22:14:25.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77eac8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.603+0000 7fb77eac8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56400/0 (socket says 172.21.15.185:56400) 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 -- 172.21.15.185:0/4237372236 learned_addr learned my addr 172.21.15.185:0/4237372236 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b64b0 0x7fb778114e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77dac6700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 0x7fb778156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 -- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b64b0 msgr2=0x7fb778114e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b64b0 0x7fb778114e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 -- 172.21.15.185:0/4237372236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 msgr2=0x7fb778156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 0x7fb778156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 -- 172.21.15.185:0/4237372236 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778156f90 con 0x7fb7780b72e0 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77dac6700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 0x7fb778156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77eac8700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb770006a90 tx=0x7fb770006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000b2e0 con 0x7fb7780b72e0 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000c820 con 0x7fb7780b72e0 2020-11-16T22:14:25.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb778113ae0 con 0x7fb7780b72e0 2020-11-16T22:14:25.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000cbc0 con 0x7fb7780b72e0 2020-11-16T22:14:25.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780b31e0 con 0x7fb7780b72e0 2020-11-16T22:14:25.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778157750 con 0x7fb7780b72e0 2020-11-16T22:14:25.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.607+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb77001c460 con 0x7fb7780b72e0 2020-11-16T22:14:25.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.611+0000 7fb76e7fc700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 0x7fb750057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:25.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.611+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb77006e140 con 0x7fb7780b72e0 2020-11-16T22:14:25.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.611+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 0x7fb750057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:25.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.611+0000 7fb77e2c7700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 0x7fb750057520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb768006e30 tx=0x7fb768015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:25.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.615+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb770044230 con 0x7fb7780b72e0 2020-11-16T22:14:25.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb77805d200 con 0x7fb7780b72e0 2020-11-16T22:14:25.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb76e7fc700 1 -- 172.21.15.185:0/4237372236 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2511 (secure 0 0 0) 0x7fb770047850 con 0x7fb7780b72e0 2020-11-16T22:14:25.802 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:25.802 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:11.589550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:25.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 msgr2=0x7fb750057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 0x7fb750057520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb768006e30 tx=0x7fb768015000).stop 2020-11-16T22:14:25.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 msgr2=0x7fb7780b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fb770006a90 tx=0x7fb770006df0).stop 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 shutdown_connections 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb750055060 0x7fb750057520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7780b72e0 0x7fb7780b08d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb7780b64b0 0x7fb778114e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 --2- 172.21.15.185:0/4237372236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7780b6aa0 0x7fb778156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:25.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 >> 172.21.15.185:0/4237372236 conn(0x7fb778067860 msgr2=0x7fb778144580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:25.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 shutdown_connections 2020-11-16T22:14:25.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:25.815+0000 7fb77f2c9700 1 -- 172.21.15.185:0/4237372236 wait complete. 2020-11-16T22:14:25.806 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 10 2020-11-16T22:14:26.120 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:14:07.542849+0000', 'modified': '2020-11-16T22:14:11.589550+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4278, 'mds_1': 4273, 'mds_2': 4217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4217': {'gid': 4217, 'name': 'a', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.185:6835/1427562659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1427562659}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1427562659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4273': {'gid': 4273, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6833/93135634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 93135634}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 93135634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3128886787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3128886787}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3128886787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:14:26.120 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:14:26.120 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:14:26.258 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":4217,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.676742752999999,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":26.235262071000001} 2020-11-16T22:14:26.259 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4217, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 15.676742753, "state": "up:active", "uptime": 26.235262071, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:14:26.259 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:14:26.396 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":4273,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.824519809000002,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":26.362281711000001} 2020-11-16T22:14:26.397 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4273, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 16.824519809, "state": "up:active", "uptime": 26.362281711, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:14:26.397 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:14:26.524 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4278,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.983311298,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":26.474275831} 2020-11-16T22:14:26.524 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4278, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 18.983311298, "state": "up:active", "uptime": 26.474275831, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:14:26.525 INFO:teuthology.run_tasks:Running task kclient... 2020-11-16T22:14:26.544 INFO:tasks.kclient:Mounting kernel clients... 2020-11-16T22:14:26.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.546 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:14:26.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.566 INFO:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2020-11-16T22:14:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2020-11-16T22:14:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:14:26.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.652 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:14:26.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.669 INFO:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2020-11-16T22:14:26.711 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2020-11-16T22:14:26.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T22:14:26.714 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:14:26.715 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:14:26.715 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:14:26.715 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:14:26.715 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:14:26.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-16T22:14:26.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:14:26.716 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:14:26.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.717 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:14:26.732 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:14:26.732 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:14:26.732 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:26.733 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:26.734 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:14:26.735 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:26.735 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-16T22:14:26.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.736 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:26.736 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:26.736 INFO:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2020-11-16T22:14:26.737 INFO:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2020-11-16T22:14:26.737 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2020-11-16T22:14:26.737 INFO:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-16T22:14:26.737 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:26.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.943 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:14:26.974 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:14:26.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:26.977 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:14:27.008 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:14:27.008 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:14:27.008 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:14:27.008 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:14:27.008 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:14:27.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.010 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:27.010 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:27.010 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:14:27.010 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:14:27.011 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:14:27.011 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:27.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.176 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:14:27.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.193 INFO:teuthology.orchestra.run.smithi062:> ip netns list-id 2020-11-16T22:14:27.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.211 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:27.211 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:27.211 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-16T22:14:27.211 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2020-11-16T22:14:27.212 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:27.293 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-11-16T22:14:27.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.294 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:27.294 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:27.295 INFO:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2020-11-16T22:14:27.295 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-16T22:14:27.295 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2020-11-16T22:14:27.295 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2020-11-16T22:14:27.296 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2020-11-16T22:14:27.296 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:27.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:27.481 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:27.481 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:27.481 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:14:27.481 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2020-11-16T22:14:27.482 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:27.569 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:27.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 Processor -- start 2020-11-16T22:14:27.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 -- start start 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ea40b6660 0x7f6ea40ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6ea407f2d0 0x7f6ea407f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea40ac270 con 0x7f6ea40b18d0 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40b2790 con 0x7f6ea40b6660 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea8900700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40ac510 con 0x7f6ea407f2d0 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.711+0000 7f6ea3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6ea407f2d0 0x7f6ea407f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.715+0000 7f6ea3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6ea407f2d0 0x7f6ea407f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56410/0 (socket says 172.21.15.185:56410) 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.715+0000 7f6ea3fff700 1 -- 172.21.15.185:0/3931870073 learned_addr learned my addr 172.21.15.185:0/3931870073 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.715+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920741658 0 0) 0x7f6ea40ac270 con 0x7f6ea40b18d0 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.715+0000 7f6ea37fe700 1 --2- 172.21.15.185:0/3931870073 >> v2:172.21.15.96:3300/0 conn(0x7f6ea40b6660 0x7f6ea40ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e8c003220 con 0x7f6ea40b18d0 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea3fff700 1 -- 172.21.15.185:0/3931870073 >> v2:172.21.15.96:3300/0 conn(0x7f6ea40b6660 msgr2=0x7f6ea40ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea3fff700 1 --2- 172.21.15.185:0/3931870073 >> v2:172.21.15.96:3300/0 conn(0x7f6ea40b6660 0x7f6ea40ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea3fff700 1 -- 172.21.15.185:0/3931870073 >> v1:172.21.15.96:6789/0 conn(0x7f6ea40b18d0 legacy=0x7f6ea4150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2888911557 0 0) 0x7f6e8c003220 con 0x7f6ea40b18d0 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea3fff700 1 -- 172.21.15.185:0/3931870073 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40b1190 con 0x7f6ea407f2d0 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea3fff700 1 --2- 172.21.15.185:0/3931870073 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea407f2d0 0x7f6ea407f760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6e9c0097d0 tx=0x7f6e9c009290).ready entity=mon.1 client_cookie=6c0fbd0bca4b3ee9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980014e0 con 0x7f6ea407f2d0 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c00b800 con 0x7f6ea407f2d0 2020-11-16T22:14:27.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea27fc700 1 -- 172.21.15.185:0/3931870073 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00bb80 con 0x7f6ea407f2d0 2020-11-16T22:14:27.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 -- 172.21.15.185:0/3931870073 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea407f2d0 msgr2=0x7f6ea407f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 --2- 172.21.15.185:0/3931870073 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea407f2d0 0x7f6ea407f760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6e9c0097d0 tx=0x7f6e9c009290).stop 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 -- 172.21.15.185:0/3931870073 shutdown_connections 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 --2- 172.21.15.185:0/3931870073 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea407f2d0 0x7f6ea407f760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 --2- 172.21.15.185:0/3931870073 >> v2:172.21.15.96:3300/0 conn(0x7f6ea40b6660 0x7f6ea40ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 -- 172.21.15.185:0/3931870073 >> 172.21.15.185:0/3931870073 conn(0x7f6ea40655c0 msgr2=0x7f6ea40659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 -- 172.21.15.185:0/3931870073 shutdown_connections 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.751+0000 7f6ea8900700 1 -- 172.21.15.185:0/3931870073 wait complete. 2020-11-16T22:14:27.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 Processor -- start 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- start start 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea407f2d0 0x7f6ea4154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ea40b18d0 0x7f6ea40af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4153310 con 0x7f6ea407f2d0 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41535b0 con 0x7f6ea40b6660 2020-11-16T22:14:27.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41634d0 con 0x7f6ea40b18d0 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56416/0 (socket says 172.21.15.185:56416) 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 -- 172.21.15.185:0/678073401 learned_addr learned my addr 172.21.15.185:0/678073401 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea2ffd700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ea40b18d0 0x7f6ea40af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea37fe700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea407f2d0 0x7f6ea4154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 -- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ea40b18d0 msgr2=0x7f6ea40af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ea40b18d0 0x7f6ea40af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 -- 172.21.15.185:0/678073401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea407f2d0 msgr2=0x7f6ea4154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea407f2d0 0x7f6ea4154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 -- 172.21.15.185:0/678073401 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4153850 con 0x7f6ea40b6660 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea3fff700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6e9c0090f0 tx=0x7f6e9c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c011e40 con 0x7f6ea40b6660 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea40afc70 con 0x7f6ea40b6660 2020-11-16T22:14:27.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea4152a20 con 0x7f6ea40b6660 2020-11-16T22:14:27.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c01b5e0 con 0x7f6ea40b6660 2020-11-16T22:14:27.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c01b9a0 con 0x7f6ea40b6660 2020-11-16T22:14:27.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.755+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea4163690 con 0x7f6ea40b6660 2020-11-16T22:14:27.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.759+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6e9c017070 con 0x7f6ea40b6660 2020-11-16T22:14:27.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.759+0000 7f6e8b7fe700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 0x7f6e78057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:27.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.759+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e9c06fed0 con 0x7f6ea40b6660 2020-11-16T22:14:27.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.759+0000 7f6ea37fe700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 0x7f6e78057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:27.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.759+0000 7f6ea37fe700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 0x7f6e78057520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6e9400db10 tx=0x7f6e9400f040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:27.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.763+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6e9c0460a0 con 0x7f6ea40b6660 2020-11-16T22:14:27.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.915+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ea405d200 con 0x7f6ea40b6660 2020-11-16T22:14:27.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.919+0000 7f6e8b7fe700 1 -- 172.21.15.185:0/678073401 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f6e9c015030 con 0x7f6ea40b6660 2020-11-16T22:14:27.905 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:27.906 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.919+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 msgr2=0x7f6e78057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.919+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 0x7f6e78057520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6e9400db10 tx=0x7f6e9400f040).stop 2020-11-16T22:14:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.919+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 msgr2=0x7f6ea4163970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.919+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6e9c0090f0 tx=0x7f6e9c016000).stop 2020-11-16T22:14:27.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 shutdown_connections 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e78055060 0x7f6e78057520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ea40b6660 0x7f6ea4163970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea407f2d0 0x7f6ea4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 --2- 172.21.15.185:0/678073401 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ea40b18d0 0x7f6ea40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 >> 172.21.15.185:0/678073401 conn(0x7f6ea40655c0 msgr2=0x7f6ea40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:27.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 shutdown_connections 2020-11-16T22:14:27.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:27.923+0000 7f6ea8900700 1 -- 172.21.15.185:0/678073401 wait complete. 2020-11-16T22:14:27.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:27.928 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:28.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 Processor -- start 2020-11-16T22:14:28.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 -- start start 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 0x7fba700b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fba70143ef0 0x7fba700b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700b1310 con 0x7fba700b2e80 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7013e400 con 0x7fba70143ef0 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75e88700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700ac070 con 0x7fba700b3920 2020-11-16T22:14:28.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 0x7fba700b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba74e86700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fba700b2e80 0x7fba7013dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49858/0 (socket says 172.21.15.185:49858) 2020-11-16T22:14:28.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba74e86700 1 -- 172.21.15.185:0/3159634046 learned_addr learned my addr 172.21.15.185:0/3159634046 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:28.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba75687700 1 --2- 172.21.15.185:0/3159634046 >> v2:172.21.15.96:3300/0 conn(0x7fba70143ef0 0x7fba700b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6f7fe700 1 -- 172.21.15.185:0/3159634046 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808582188 0 0) 0x7fba700b1310 con 0x7fba700b2e80 2020-11-16T22:14:28.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 -- 172.21.15.185:0/3159634046 >> v2:172.21.15.96:3300/0 conn(0x7fba70143ef0 msgr2=0x7fba700b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:28.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 --2- 172.21.15.185:0/3159634046 >> v2:172.21.15.96:3300/0 conn(0x7fba70143ef0 0x7fba700b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 -- 172.21.15.185:0/3159634046 >> v1:172.21.15.185:6789/0 conn(0x7fba700b2e80 legacy=0x7fba7013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 -- 172.21.15.185:0/3159634046 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700ac310 con 0x7fba700b3920 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6ffff700 1 --2- 172.21.15.185:0/3159634046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 0x7fba700b3260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fba5800b9a0 tx=0x7fba58006df0).ready entity=mon.2 client_cookie=46734d895e05640 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6f7fe700 1 -- 172.21.15.185:0/3159634046 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba580097b0 con 0x7fba700b3920 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6f7fe700 1 -- 172.21.15.185:0/3159634046 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba58009910 con 0x7fba700b3920 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.067+0000 7fba6f7fe700 1 -- 172.21.15.185:0/3159634046 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58009c90 con 0x7fba700b3920 2020-11-16T22:14:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/3159634046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 msgr2=0x7fba700b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- 172.21.15.185:0/3159634046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 0x7fba700b3260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fba5800b9a0 tx=0x7fba58006df0).stop 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/3159634046 shutdown_connections 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- 172.21.15.185:0/3159634046 >> v2:172.21.15.96:3300/0 conn(0x7fba70143ef0 0x7fba700b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- 172.21.15.185:0/3159634046 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba700b3920 0x7fba700b3260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/3159634046 >> 172.21.15.185:0/3159634046 conn(0x7fba70060130 msgr2=0x7fba70060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/3159634046 shutdown_connections 2020-11-16T22:14:28.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/3159634046 wait complete. 2020-11-16T22:14:28.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 Processor -- start 2020-11-16T22:14:28.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- start start 2020-11-16T22:14:28.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba700b2e80 0x7fba701405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70140fc0 con 0x7fba700b2e80 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70157030 con 0x7fba700b3920 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba70141350 con 0x7fba70143ef0 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49872/0 (socket says 172.21.15.185:49872) 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56430/0 (socket says 172.21.15.185:56430) 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 -- 172.21.15.185:0/2269866994 learned_addr learned my addr 172.21.15.185:0/2269866994 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba74e86700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba700b2e80 0x7fba701405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 -- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 msgr2=0x7fba70140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 -- 172.21.15.185:0/2269866994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba700b2e80 msgr2=0x7fba701405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba700b2e80 0x7fba701405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 -- 172.21.15.185:0/2269866994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70157260 con 0x7fba70143ef0 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6ffff700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75687700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fba6000ebd0 tx=0x7fba60006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000b340 con 0x7fba70143ef0 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000c820 con 0x7fba70143ef0 2020-11-16T22:14:28.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba70146c40 con 0x7fba70143ef0 2020-11-16T22:14:28.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000cbe0 con 0x7fba70143ef0 2020-11-16T22:14:28.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.071+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba700aec40 con 0x7fba70143ef0 2020-11-16T22:14:28.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba700aee90 con 0x7fba70143ef0 2020-11-16T22:14:28.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fba6001c460 con 0x7fba70143ef0 2020-11-16T22:14:28.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba6cff9700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 0x7fba48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fba6006dff0 con 0x7fba70143ef0 2020-11-16T22:14:28.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba74e86700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 0x7fba48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.075+0000 7fba74e86700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 0x7fba48057520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fba68001520 tx=0x7fba68007d80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.083+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fba600441f0 con 0x7fba70143ef0 2020-11-16T22:14:28.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.235+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fba7005d200 con 0x7fba70143ef0 2020-11-16T22:14:28.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.235+0000 7fba6cff9700 1 -- 172.21.15.185:0/2269866994 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fba60047810 con 0x7fba70143ef0 2020-11-16T22:14:28.224 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:28.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 msgr2=0x7fba48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 0x7fba48057520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fba68001520 tx=0x7fba68007d80).stop 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 msgr2=0x7fba70147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fba6000ebd0 tx=0x7fba60006df0).stop 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 shutdown_connections 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fba48055060 0x7fba48057520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fba70143ef0 0x7fba70147790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba700b2e80 0x7fba701405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 --2- 172.21.15.185:0/2269866994 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fba700b3920 0x7fba70140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 >> 172.21.15.185:0/2269866994 conn(0x7fba70060130 msgr2=0x7fba70060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 shutdown_connections 2020-11-16T22:14:28.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.239+0000 7fba75e88700 1 -- 172.21.15.185:0/2269866994 wait complete. 2020-11-16T22:14:28.239 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:28.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 Processor -- start 2020-11-16T22:14:28.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 -- start start 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc6540b6650 0x7fc654147f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654063660 con 0x7fc6540b4d80 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540b2f80 con 0x7fc654144c60 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65901e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540638c0 con 0x7fc6540b6650 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65359e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc654144c60 0x7fc65413d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49876/0 (socket says 172.21.15.185:49876) 2020-11-16T22:14:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc65359e700 1 -- 172.21.15.185:0/258336197 learned_addr learned my addr 172.21.15.185:0/258336197 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:28.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc652d9d700 1 --2- 172.21.15.185:0/258336197 >> v2:172.21.15.96:3300/0 conn(0x7fc6540b6650 0x7fc654147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc651d9b700 1 -- 172.21.15.185:0/258336197 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438998960 0 0) 0x7fc6540b2f80 con 0x7fc654144c60 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc651d9b700 1 -- 172.21.15.185:0/258336197 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc640003240 con 0x7fc654144c60 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc652d9d700 1 -- 172.21.15.185:0/258336197 >> v1:172.21.15.185:6789/0 conn(0x7fc654144c60 legacy=0x7fc65413d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc652d9d700 1 -- 172.21.15.185:0/258336197 >> v1:172.21.15.96:6789/0 conn(0x7fc6540b4d80 legacy=0x7fc654144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc652d9d700 1 -- 172.21.15.185:0/258336197 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654063b60 con 0x7fc6540b6650 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc652d9d700 1 --2- 172.21.15.185:0/258336197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b6650 0x7fc654147f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc64400b9a0 tx=0x7fc644006df0).ready entity=mon.0 client_cookie=b6c4833fe3fa792a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc651d9b700 1 -- 172.21.15.185:0/258336197 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440097b0 con 0x7fc6540b6650 2020-11-16T22:14:28.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc651d9b700 1 -- 172.21.15.185:0/258336197 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644009910 con 0x7fc6540b6650 2020-11-16T22:14:28.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.379+0000 7fc651d9b700 1 -- 172.21.15.185:0/258336197 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644009c90 con 0x7fc6540b6650 2020-11-16T22:14:28.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- 172.21.15.185:0/258336197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b6650 msgr2=0x7fc654147f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 --2- 172.21.15.185:0/258336197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b6650 0x7fc654147f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fc64400b9a0 tx=0x7fc644006df0).stop 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- 172.21.15.185:0/258336197 shutdown_connections 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 --2- 172.21.15.185:0/258336197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b6650 0x7fc654147f40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- 172.21.15.185:0/258336197 >> 172.21.15.185:0/258336197 conn(0x7fc6540649a0 msgr2=0x7fc654062e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- 172.21.15.185:0/258336197 shutdown_connections 2020-11-16T22:14:28.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- 172.21.15.185:0/258336197 wait complete. 2020-11-16T22:14:28.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 Processor -- start 2020-11-16T22:14:28.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.383+0000 7fc65901e700 1 -- start start 2020-11-16T22:14:28.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 0x7fc654156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 0x7fc65413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654145920 con 0x7fc6540b4d80 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654145450 con 0x7fc654144c60 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654145680 con 0x7fc6540b6650 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 0x7fc65413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56444/0 (socket says 172.21.15.185:56444) 2020-11-16T22:14:28.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 -- 172.21.15.185:0/3368063272 learned_addr learned my addr 172.21.15.185:0/3368063272 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:28.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc652d9d700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 0x7fc654156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 -- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 msgr2=0x7fc65413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 0x7fc65413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 -- 172.21.15.185:0/3368063272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 msgr2=0x7fc654156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 0x7fc654156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 -- 172.21.15.185:0/3368063272 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654148fe0 con 0x7fc654144c60 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65259c700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 0x7fc65413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc652d9d700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 0x7fc654156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65359e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fc64c00cac0 tx=0x7fc64c007d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c005b80 con 0x7fc654144c60 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c005ce0 con 0x7fc654144c60 2020-11-16T22:14:28.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc654147670 con 0x7fc654144c60 2020-11-16T22:14:28.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c007320 con 0x7fc654144c60 2020-11-16T22:14:28.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65413ec50 con 0x7fc654144c60 2020-11-16T22:14:28.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.387+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654149170 con 0x7fc654144c60 2020-11-16T22:14:28.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.391+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc64c010030 con 0x7fc654144c60 2020-11-16T22:14:28.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.391+0000 7fc63affd700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 0x7fc630057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.391+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc64c06de00 con 0x7fc654144c60 2020-11-16T22:14:28.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.391+0000 7fc652d9d700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 0x7fc630057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.391+0000 7fc652d9d700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 0x7fc630057550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc644006e30 tx=0x7fc644015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.399+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc64c044060 con 0x7fc654144c60 2020-11-16T22:14:28.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.547+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc65405d4e0 con 0x7fc654144c60 2020-11-16T22:14:28.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc63affd700 1 -- 172.21.15.185:0/3368063272 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc64c047680 con 0x7fc654144c60 2020-11-16T22:14:28.537 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:28.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 msgr2=0x7fc630057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 0x7fc630057550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc644006e30 tx=0x7fc644015000).stop 2020-11-16T22:14:28.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 msgr2=0x7fc654149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fc64c00cac0 tx=0x7fc64c007d60).stop 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 shutdown_connections 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc630055090 0x7fc630057550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc654144c60 0x7fc654149870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6540b4d80 0x7fc654156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 --2- 172.21.15.185:0/3368063272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc6540b6650 0x7fc65413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 >> 172.21.15.185:0/3368063272 conn(0x7fc6540649a0 msgr2=0x7fc6540b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 shutdown_connections 2020-11-16T22:14:28.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.551+0000 7fc65901e700 1 -- 172.21.15.185:0/3368063272 wait complete. 2020-11-16T22:14:28.552 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:28.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 Processor -- start 2020-11-16T22:14:28.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 -- start start 2020-11-16T22:14:28.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 0x7fd5380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380ad490 con 0x7fd5380afc60 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380ad730 con 0x7fd5380b7440 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.691+0000 7fd540016700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380ad9d0 con 0x7fd5380ae580 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53e5b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 0x7fd5380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53ddb2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd5380b7440 0x7fd53813dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49886/0 (socket says 172.21.15.185:49886) 2020-11-16T22:14:28.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53ddb2700 1 -- 172.21.15.185:0/4038637279 learned_addr learned my addr 172.21.15.185:0/4038637279 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53cdb0700 1 -- 172.21.15.185:0/4038637279 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731193843 0 0) 0x7fd5380ad730 con 0x7fd5380b7440 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53e5b3700 1 -- 172.21.15.185:0/4038637279 >> v1:172.21.15.185:6789/0 conn(0x7fd5380b7440 legacy=0x7fd53813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53e5b3700 1 -- 172.21.15.185:0/4038637279 >> v1:172.21.15.96:6789/0 conn(0x7fd5380afc60 legacy=0x7fd5380ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53e5b3700 1 -- 172.21.15.185:0/4038637279 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53813e5b0 con 0x7fd5380ae580 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53e5b3700 1 --2- 172.21.15.185:0/4038637279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 0x7fd5380b7820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd530009cc0 tx=0x7fd5300091d0).ready entity=mon.2 client_cookie=5fc4d348e704d6ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53cdb0700 1 -- 172.21.15.185:0/4038637279 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000bbc0 con 0x7fd5380ae580 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53cdb0700 1 -- 172.21.15.185:0/4038637279 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000bd20 con 0x7fd5380ae580 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd53cdb0700 1 -- 172.21.15.185:0/4038637279 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530011610 con 0x7fd5380ae580 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- 172.21.15.185:0/4038637279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 msgr2=0x7fd5380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 --2- 172.21.15.185:0/4038637279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 0x7fd5380b7820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd530009cc0 tx=0x7fd5300091d0).stop 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- 172.21.15.185:0/4038637279 shutdown_connections 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 --2- 172.21.15.185:0/4038637279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380ae580 0x7fd5380b7820 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- 172.21.15.185:0/4038637279 >> 172.21.15.185:0/4038637279 conn(0x7fd538067860 msgr2=0x7fd538067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- 172.21.15.185:0/4038637279 shutdown_connections 2020-11-16T22:14:28.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- 172.21.15.185:0/4038637279 wait complete. 2020-11-16T22:14:28.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 Processor -- start 2020-11-16T22:14:28.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.695+0000 7fd540016700 1 -- start start 2020-11-16T22:14:28.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5380ae580 0x7fd538153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 0x7fd53814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 0x7fd538157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380b3690 con 0x7fd5380ae580 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380b3930 con 0x7fd5380afc60 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd538156ac0 con 0x7fd5380b7440 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53e5b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 0x7fd538157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 0x7fd53814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53e5b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 0x7fd538157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49902/0 (socket says 172.21.15.185:49902) 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53e5b3700 1 -- 172.21.15.185:0/200977557 learned_addr learned my addr 172.21.15.185:0/200977557 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53ddb2700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5380ae580 0x7fd538153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 -- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 msgr2=0x7fd538157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 0x7fd538157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 -- 172.21.15.185:0/200977557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5380ae580 msgr2=0x7fd538153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5380ae580 0x7fd538153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 -- 172.21.15.185:0/200977557 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd538156d60 con 0x7fd5380afc60 2020-11-16T22:14:28.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd53d5b1700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 0x7fd53814acb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd53400c710 tx=0x7fd534007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534007020 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534007180 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53814b1a0 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340064f0 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5380b1b10 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.699+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd538145010 con 0x7fd5380afc60 2020-11-16T22:14:28.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.703+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd534010030 con 0x7fd5380afc60 2020-11-16T22:14:28.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.703+0000 7fd525ffb700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 0x7fd510057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.703+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd53406d600 con 0x7fd5380afc60 2020-11-16T22:14:28.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.703+0000 7fd53ddb2700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 0x7fd510057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:28.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.703+0000 7fd53ddb2700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 0x7fd510057520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd5380b9ff0 tx=0x7fd5280079a0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:28.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.707+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd5340436f0 con 0x7fd5380afc60 2020-11-16T22:14:28.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd538156ef0 con 0x7fd5380afc60 2020-11-16T22:14:28.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd525ffb700 1 -- 172.21.15.185:0/200977557 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd534046d10 con 0x7fd5380afc60 2020-11-16T22:14:28.851 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 msgr2=0x7fd510057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 0x7fd510057520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd5380b9ff0 tx=0x7fd5280079a0).stop 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 msgr2=0x7fd53814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.863+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 0x7fd53814acb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd53400c710 tx=0x7fd534007b00).stop 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 shutdown_connections 2020-11-16T22:14:28.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd510055060 0x7fd510057520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5380b7440 0x7fd538157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5380ae580 0x7fd538153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 --2- 172.21.15.185:0/200977557 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5380afc60 0x7fd53814acb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 >> 172.21.15.185:0/200977557 conn(0x7fd538067860 msgr2=0x7fd538067c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 shutdown_connections 2020-11-16T22:14:28.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:28.867+0000 7fd540016700 1 -- 172.21.15.185:0/200977557 wait complete. 2020-11-16T22:14:28.865 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:28.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 Processor -- start 2020-11-16T22:14:28.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- start start 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f40d80ba800 0x7f40d80b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80b7290 con 0x7f40d80b5d70 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80b4960 con 0x7f40d8143470 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d80b4bf0 con 0x7f40d80ba800 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de002700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f40d8143470 0x7f40d81438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49902/0 (socket says 172.21.15.185:49902) 2020-11-16T22:14:28.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de002700 1 -- 172.21.15.185:0/1828712177 learned_addr learned my addr 172.21.15.185:0/1828712177 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:28.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40dd000700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f40d80ba800 0x7f40d80b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40dd000700 1 -- 172.21.15.185:0/1828712177 >> v1:172.21.15.185:6789/0 conn(0x7f40d8143470 legacy=0x7f40d81438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40dd000700 1 -- 172.21.15.185:0/1828712177 >> v1:172.21.15.96:6789/0 conn(0x7f40d80b5d70 legacy=0x7f40d80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40dd000700 1 -- 172.21.15.185:0/1828712177 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d80b74c0 con 0x7f40d80ba800 2020-11-16T22:14:29.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40dd000700 1 --2- 172.21.15.185:0/1828712177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80b4470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f40c800b9a0 tx=0x7f40c8006df0).ready entity=mon.1 client_cookie=135419051f1d1fec server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40cffff700 1 -- 172.21.15.185:0/1828712177 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80097b0 con 0x7f40d80ba800 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40cffff700 1 -- 172.21.15.185:0/1828712177 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8009910 con 0x7f40d80ba800 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40cffff700 1 -- 172.21.15.185:0/1828712177 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8009c90 con 0x7f40d80ba800 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- 172.21.15.185:0/1828712177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 msgr2=0x7f40d80b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 --2- 172.21.15.185:0/1828712177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80b4470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f40c800b9a0 tx=0x7f40c8006df0).stop 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- 172.21.15.185:0/1828712177 shutdown_connections 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 --2- 172.21.15.185:0/1828712177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80b4470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- 172.21.15.185:0/1828712177 >> 172.21.15.185:0/1828712177 conn(0x7f40d805ba80 msgr2=0x7f40d805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:29.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- 172.21.15.185:0/1828712177 shutdown_connections 2020-11-16T22:14:29.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.011+0000 7f40de803700 1 -- 172.21.15.185:0/1828712177 wait complete. 2020-11-16T22:14:29.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 Processor -- start 2020-11-16T22:14:29.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 -- start start 2020-11-16T22:14:29.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d80b5d70 0x7f40d813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40d8143470 0x7f40d80add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81483f0 con 0x7f40d80b5d70 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8148150 con 0x7f40d80ba800 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de803700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80b00e0 con 0x7f40d8143470 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40dd000700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40de002700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40d8143470 0x7f40d80add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40dd000700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56470/0 (socket says 172.21.15.185:56470) 2020-11-16T22:14:29.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40dd000700 1 -- 172.21.15.185:0/2561636584 learned_addr learned my addr 172.21.15.185:0/2561636584 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:29.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.015+0000 7f40dd801700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d80b5d70 0x7f40d813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 -- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40d8143470 msgr2=0x7f40d80add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40d8143470 0x7f40d80add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 -- 172.21.15.185:0/2561636584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d80b5d70 msgr2=0x7f40d813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d80b5d70 0x7f40d813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 -- 172.21.15.185:0/2561636584 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d80b0380 con 0x7f40d80ba800 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd000700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f40d80b4fa0 tx=0x7f40c8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011e50 con 0x7f40d80ba800 2020-11-16T22:14:29.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c801e460 con 0x7f40d80ba800 2020-11-16T22:14:29.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d813f120 con 0x7f40d80ba800 2020-11-16T22:14:29.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c801e800 con 0x7f40d80ba800 2020-11-16T22:14:29.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d8147940 con 0x7f40d80ba800 2020-11-16T22:14:29.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f40c8025030 con 0x7f40d80ba800 2020-11-16T22:14:29.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d8147f40 con 0x7f40d80ba800 2020-11-16T22:14:29.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 0x7f40b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f40c806f4f0 con 0x7f40d80ba800 2020-11-16T22:14:29.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.019+0000 7f40dd801700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 0x7f40b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.023+0000 7f40dd801700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 0x7f40b0057520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f40d4003230 tx=0x7f40d4000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.027+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f40c8045800 con 0x7f40d80ba800 2020-11-16T22:14:29.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40d805d200 con 0x7f40d80ba800 2020-11-16T22:14:29.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40cd7fa700 1 -- 172.21.15.185:0/2561636584 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f40c8023070 con 0x7f40d80ba800 2020-11-16T22:14:29.174 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:29.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 msgr2=0x7f40b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 0x7f40b0057520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f40d4003230 tx=0x7f40d4000b80).stop 2020-11-16T22:14:29.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 msgr2=0x7f40d80ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f40d80b4fa0 tx=0x7f40c8015000).stop 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 shutdown_connections 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f40b0055060 0x7f40b0057520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40d8143470 0x7f40d80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d80b5d70 0x7f40d813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 --2- 172.21.15.185:0/2561636584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40d80ba800 0x7f40d80ad870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 >> 172.21.15.185:0/2561636584 conn(0x7f40d805ba80 msgr2=0x7f40d8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:29.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 shutdown_connections 2020-11-16T22:14:29.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.187+0000 7f40de803700 1 -- 172.21.15.185:0/2561636584 wait complete. 2020-11-16T22:14:29.188 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:14:29.189 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:29.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.315+0000 7f82dffff700 1 Processor -- start 2020-11-16T22:14:29.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.315+0000 7f82dffff700 1 -- start start 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f82e00b3090 0x7f82e00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e0061fe0 con 0x7f82e00b1d30 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e00b2b60 con 0x7f82e00b3090 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e014c060 con 0x7f82e00ad110 2020-11-16T22:14:29.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49924/0 (socket says 172.21.15.185:49924) 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 -- 172.21.15.185:0/2397950753 learned_addr learned my addr 172.21.15.185:0/2397950753 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82de7fc700 1 --1- 172.21.15.185:0/2397950753 >> v1:172.21.15.185:6789/0 conn(0x7f82e00b1d30 0x7f82e014b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49914/0 (socket says 172.21.15.185:49914) 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82deffd700 1 --2- 172.21.15.185:0/2397950753 >> v2:172.21.15.96:3300/0 conn(0x7f82e00b3090 0x7f82e00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 -- 172.21.15.185:0/2397950753 >> v2:172.21.15.96:3300/0 conn(0x7f82e00b3090 msgr2=0x7f82e00b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 --2- 172.21.15.185:0/2397950753 >> v2:172.21.15.96:3300/0 conn(0x7f82e00b3090 0x7f82e00b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 -- 172.21.15.185:0/2397950753 >> v1:172.21.15.185:6789/0 conn(0x7f82e00b1d30 legacy=0x7f82e014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 -- 172.21.15.185:0/2397950753 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00b0fe0 con 0x7f82e00ad110 2020-11-16T22:14:29.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82df7fe700 1 --2- 172.21.15.185:0/2397950753 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f82d8009d50 tx=0x7f82d8009290).ready entity=mon.2 client_cookie=feaf74f4e7950533 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82ddffb700 1 -- 172.21.15.185:0/2397950753 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d800bc80 con 0x7f82e00ad110 2020-11-16T22:14:29.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82ddffb700 1 -- 172.21.15.185:0/2397950753 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d800bde0 con 0x7f82e00ad110 2020-11-16T22:14:29.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82ddffb700 1 -- 172.21.15.185:0/2397950753 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8011610 con 0x7f82e00ad110 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- 172.21.15.185:0/2397950753 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 msgr2=0x7f82e01477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 --2- 172.21.15.185:0/2397950753 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f82d8009d50 tx=0x7f82d8009290).stop 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- 172.21.15.185:0/2397950753 shutdown_connections 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 --2- 172.21.15.185:0/2397950753 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00ad110 0x7f82e01477b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 --2- 172.21.15.185:0/2397950753 >> v2:172.21.15.96:3300/0 conn(0x7f82e00b3090 0x7f82e00b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- 172.21.15.185:0/2397950753 >> 172.21.15.185:0/2397950753 conn(0x7f82e0060130 msgr2=0x7f82e0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:29.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- 172.21.15.185:0/2397950753 shutdown_connections 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- 172.21.15.185:0/2397950753 wait complete. 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 Processor -- start 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.319+0000 7f82dffff700 1 -- start start 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e00ad110 0x7f82e014f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00b1d30 0x7f82e014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00aecd0 con 0x7f82e00ad110 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e014ffd0 con 0x7f82e00b3090 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e0143700 con 0x7f82e00b1d30 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82de7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00b1d30 0x7f82e014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56482/0 (socket says 172.21.15.185:56482) 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 -- 172.21.15.185:0/1754874031 learned_addr learned my addr 172.21.15.185:0/1754874031 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:29.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82deffd700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e00ad110 0x7f82e014f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 -- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00b1d30 msgr2=0x7f82e014fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00b1d30 0x7f82e014fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 -- 172.21.15.185:0/1754874031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e00ad110 msgr2=0x7f82e014f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e00ad110 0x7f82e014f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 -- 172.21.15.185:0/1754874031 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00b0470 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82df7fe700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f82e00b0f10 tx=0x7f82d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d800bc80 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8009e90 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8010cb0 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e0158820 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e00aff10 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e005d200 con 0x7f82e00b3090 2020-11-16T22:14:29.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.323+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f82d8023030 con 0x7f82e00b3090 2020-11-16T22:14:29.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.327+0000 7f82c6ffd700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 0x7f82bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:29.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.327+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82d806f360 con 0x7f82e00b3090 2020-11-16T22:14:29.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.327+0000 7f82deffd700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 0x7f82bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:29.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.327+0000 7f82deffd700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 0x7f82bc057520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f82d000d5f0 tx=0x7f82d0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:29.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.335+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f82d80456f0 con 0x7f82e00b3090 2020-11-16T22:14:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e01589b0 con 0x7f82e00b3090 2020-11-16T22:14:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82c6ffd700 1 -- 172.21.15.185:0/1754874031 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2511 (secure 0 0 0) 0x7f82d8021020 con 0x7f82e00b3090 2020-11-16T22:14:29.522 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:29.522 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:11.589550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 msgr2=0x7f82bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 0x7f82bc057520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f82d000d5f0 tx=0x7f82d0008040).stop 2020-11-16T22:14:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 msgr2=0x7f82e0157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f82e00b0f10 tx=0x7f82d8016000).stop 2020-11-16T22:14:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 shutdown_connections 2020-11-16T22:14:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82bc055060 0x7f82bc057520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e00b3090 0x7f82e0157bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e00ad110 0x7f82e014f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 --2- 172.21.15.185:0/1754874031 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e00b1d30 0x7f82e014fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 >> 172.21.15.185:0/1754874031 conn(0x7f82e0060130 msgr2=0x7f82e00b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 shutdown_connections 2020-11-16T22:14:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:29.535+0000 7f82dffff700 1 -- 172.21.15.185:0/1754874031 wait complete. 2020-11-16T22:14:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 10 2020-11-16T22:14:29.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:14:07.542849+0000', 'modified': '2020-11-16T22:14:11.589550+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4278, 'mds_1': 4273, 'mds_2': 4217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4217': {'gid': 4217, 'name': 'a', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.185:6835/1427562659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1427562659}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1427562659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4273': {'gid': 4273, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6833/93135634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 93135634}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 93135634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3128886787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3128886787}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3128886787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:14:29.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:14:29.839 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:14:29.980 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":4217,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.398424917,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":29.956949581} 2020-11-16T22:14:29.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4217, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 19.398424917, "state": "up:active", "uptime": 29.956949581, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:14:29.981 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:14:30.116 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":4273,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.544116355,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":30.081876563000002} 2020-11-16T22:14:30.116 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4273, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 20.544116355, "state": "up:active", "uptime": 30.081876563, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:14:30.117 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:14:30.246 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4278,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.706421042999999,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":30.197379522999999} 2020-11-16T22:14:30.247 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4278, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 22.706421043, "state": "up:active", "uptime": 30.197379523, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:14:30.247 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:14:30.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.248 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:14:30.266 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:14:30.267 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:30.402 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:30.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:14:30.499 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:30.499 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:14:30.508 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:14:30.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.508 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:14:30.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:14:30.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:14:30.541 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-16T22:14:30.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-16T22:14:30.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:14:30.542 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-16T22:14:30.542 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:14:30.542 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-16T22:14:30.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.543 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:14:30.558 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.559 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:14:30.560 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:14:30.561 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:14:30.562 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:14:30.563 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.564 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:14:30.642 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:14:30.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.646 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:14:30.661 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:14:30.661 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:14:30.662 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:14:30.662 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:14:30.662 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:14:30.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.663 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:30.663 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:30.664 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:14:30.664 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:14:30.664 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:14:30.664 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:30.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.764 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:14:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:14:30.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.781 INFO:teuthology.orchestra.run.smithi062:> ip netns list-id 2020-11-16T22:14:30.841 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-11-16T22:14:30.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.843 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:30.843 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:30.843 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2020-11-16T22:14:30.843 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1 2020-11-16T22:14:30.844 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:30.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.928 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-16T22:14:30.958 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:14:30.958 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether f2:8c:c6:37:58:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:14:30.959 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-16T22:14:30.960 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.960 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::f08c:c6ff:fe37:5868/64 scope link 2020-11-16T22:14:30.960 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:30.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:30.961 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-16T22:14:31.037 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:31.038 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:14:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether f2:8c:c6:37:58:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:14:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-16T22:14:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::f08c:c6ff:fe37:5868/64 scope link 2020-11-16T22:14:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:14:31.041 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2020-11-16T22:14:31.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:31.042 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:31.042 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:31.043 INFO:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1 2020-11-16T22:14:31.043 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2020-11-16T22:14:31.043 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2020-11-16T22:14:31.043 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2020-11-16T22:14:31.044 INFO:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2020-11-16T22:14:31.044 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:31.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:31.204 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:14:31.205 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:14:31.205 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2020-11-16T22:14:31.205 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2020-11-16T22:14:31.205 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:14:31.289 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:31.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 Processor -- start 2020-11-16T22:14:31.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- start start 2020-11-16T22:14:31.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0df009c900 0x7f0df009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0df009b270 0x7f0df00b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df009d410 con 0x7f0df01476e0 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df00b6b20 con 0x7f0df009c900 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df009b010 con 0x7f0df009b270 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0df009b270 0x7f0df00b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0df009b270 0x7f0df00b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56494/0 (socket says 172.21.15.185:56494) 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 -- 172.21.15.185:0/4053260833 learned_addr learned my addr 172.21.15.185:0/4053260833 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:31.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dee59c700 1 --2- 172.21.15.185:0/4053260833 >> v2:172.21.15.96:3300/0 conn(0x7f0df009c900 0x7f0df009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 -- 172.21.15.185:0/4053260833 >> v2:172.21.15.96:3300/0 conn(0x7f0df009c900 msgr2=0x7f0df009cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 --2- 172.21.15.185:0/4053260833 >> v2:172.21.15.96:3300/0 conn(0x7f0df009c900 0x7f0df009cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 -- 172.21.15.185:0/4053260833 >> v1:172.21.15.96:6789/0 conn(0x7f0df01476e0 legacy=0x7f0df0147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 -- 172.21.15.185:0/4053260833 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df009a630 con 0x7f0df009b270 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0dedd9b700 1 --2- 172.21.15.185:0/4053260833 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df009b270 0x7f0df00b6630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0df00811a0 tx=0x7f0dd8006eb0).ready entity=mon.1 client_cookie=4b07d703b34c4512 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0ded59a700 1 -- 172.21.15.185:0/4053260833 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8009810 con 0x7f0df009b270 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0ded59a700 1 -- 172.21.15.185:0/4053260833 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8009970 con 0x7f0df009b270 2020-11-16T22:14:31.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0ded59a700 1 -- 172.21.15.185:0/4053260833 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8009cf0 con 0x7f0df009b270 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- 172.21.15.185:0/4053260833 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df009b270 msgr2=0x7f0df00b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 --2- 172.21.15.185:0/4053260833 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df009b270 0x7f0df00b6630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0df00811a0 tx=0x7f0dd8006eb0).stop 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- 172.21.15.185:0/4053260833 shutdown_connections 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 --2- 172.21.15.185:0/4053260833 >> v2:172.21.15.96:3300/0 conn(0x7f0df009c900 0x7f0df009cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 --2- 172.21.15.185:0/4053260833 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df009b270 0x7f0df00b6630 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- 172.21.15.185:0/4053260833 >> 172.21.15.185:0/4053260833 conn(0x7f0df0068330 msgr2=0x7f0df00ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:31.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- 172.21.15.185:0/4053260833 shutdown_connections 2020-11-16T22:14:31.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.435+0000 7f0df4e99700 1 -- 172.21.15.185:0/4053260833 wait complete. 2020-11-16T22:14:31.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 Processor -- start 2020-11-16T22:14:31.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- start start 2020-11-16T22:14:31.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df009c900 0x7f0df0061970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 0x7f0df00af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00626d0 con 0x7f0df009c900 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0154050 con 0x7f0df00b4970 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df008fc40 con 0x7f0df01476e0 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0dedd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 0x7f0df00af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56498/0 (socket says 172.21.15.185:56498) 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0dee59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df009c900 0x7f0df0061970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 -- 172.21.15.185:0/1368059789 learned_addr learned my addr 172.21.15.185:0/1368059789 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 -- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 msgr2=0x7f0df00af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 0x7f0df00af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 -- 172.21.15.185:0/1368059789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df009c900 msgr2=0x7f0df0061970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df009c900 0x7f0df0061970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 -- 172.21.15.185:0/1368059789 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0154280 con 0x7f0df00b4970 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0dedd9b700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 0x7f0df00af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0deed9d700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0de800ca90 tx=0x7f0de8007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800a6d0 con 0x7f0df00b4970 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800a830 con 0x7f0df00b4970 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df008fee0 con 0x7f0df00b4970 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800abb0 con 0x7f0df00b4970 2020-11-16T22:14:31.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.439+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df00affc0 con 0x7f0df00b4970 2020-11-16T22:14:31.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0de8019030 con 0x7f0df00b4970 2020-11-16T22:14:31.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df00616a0 con 0x7f0df00b4970 2020-11-16T22:14:31.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0de67fc700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 0x7f0dcc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0de806d5d0 con 0x7f0df00b4970 2020-11-16T22:14:31.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0dee59c700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 0x7f0dcc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.443+0000 7f0dee59c700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 0x7f0dcc057520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0df00620d0 tx=0x7f0de0006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.451+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0de8043770 con 0x7f0df00b4970 2020-11-16T22:14:31.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.599+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0df005d4e0 con 0x7f0df00b4970 2020-11-16T22:14:31.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.599+0000 7f0de67fc700 1 -- 172.21.15.185:0/1368059789 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f0de8046d90 con 0x7f0df00b4970 2020-11-16T22:14:31.587 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:31.587 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:31.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 msgr2=0x7f0dcc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 0x7f0dcc057520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0df00620d0 tx=0x7f0de0006df0).stop 2020-11-16T22:14:31.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 msgr2=0x7f0df00b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f0de800ca90 tx=0x7f0de8007eb0).stop 2020-11-16T22:14:31.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 shutdown_connections 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0dcc055060 0x7f0dcc057520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0df00b4970 0x7f0df00b0cd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0df009c900 0x7f0df0061970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 --2- 172.21.15.185:0/1368059789 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0df01476e0 0x7f0df00af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 >> 172.21.15.185:0/1368059789 conn(0x7f0df0068330 msgr2=0x7f0df0146e30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 shutdown_connections 2020-11-16T22:14:31.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.603+0000 7f0df4e99700 1 -- 172.21.15.185:0/1368059789 wait complete. 2020-11-16T22:14:31.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:31.603 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:31.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 Processor -- start 2020-11-16T22:14:31.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 -- start start 2020-11-16T22:14:31.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d8144490 con 0x7f35d80b5c40 2020-11-16T22:14:31.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d81446e0 con 0x7f35d8144000 2020-11-16T22:14:31.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35de1ef700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8144980 con 0x7f35d80ba800 2020-11-16T22:14:31.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dc9ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dd9ee700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f35d8144000 0x7f35d80b1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49946/0 (socket says 172.21.15.185:49946) 2020-11-16T22:14:31.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dd9ee700 1 -- 172.21.15.185:0/2805886276 learned_addr learned my addr 172.21.15.185:0/2805886276 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:31.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35c3fff700 1 -- 172.21.15.185:0/2805886276 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758511563 0 0) 0x7f35d81446e0 con 0x7f35d8144000 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35c3fff700 1 -- 172.21.15.185:0/2805886276 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35bc003240 con 0x7f35d8144000 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2805886276 >> v1:172.21.15.185:6789/0 conn(0x7f35d8144000 legacy=0x7f35d80b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2805886276 >> v1:172.21.15.96:6789/0 conn(0x7f35d80b5c40 legacy=0x7f35d813dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2805886276 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8144c20 con 0x7f35d80ba800 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.739+0000 7f35dc9ec700 1 --2- 172.21.15.185:0/2805886276 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80b6020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f35cc00b9d0 tx=0x7f35cc006df0).ready entity=mon.2 client_cookie=9af2c7a7065d67f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35c3fff700 1 -- 172.21.15.185:0/2805886276 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0097b0 con 0x7f35d80ba800 2020-11-16T22:14:31.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35c3fff700 1 -- 172.21.15.185:0/2805886276 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc009910 con 0x7f35d80ba800 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35c3fff700 1 -- 172.21.15.185:0/2805886276 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc009c90 con 0x7f35d80ba800 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- 172.21.15.185:0/2805886276 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 msgr2=0x7f35d80b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2805886276 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80b6020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f35cc00b9d0 tx=0x7f35cc006df0).stop 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- 172.21.15.185:0/2805886276 shutdown_connections 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2805886276 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80b6020 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- 172.21.15.185:0/2805886276 >> 172.21.15.185:0/2805886276 conn(0x7f35d805ba80 msgr2=0x7f35d805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:31.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- 172.21.15.185:0/2805886276 shutdown_connections 2020-11-16T22:14:31.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- 172.21.15.185:0/2805886276 wait complete. 2020-11-16T22:14:31.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 Processor -- start 2020-11-16T22:14:31.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- start start 2020-11-16T22:14:31.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d80b5c40 0x7f35d8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35d8144000 0x7f35d8149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8145ca0 con 0x7f35d80b5c40 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8145f40 con 0x7f35d8144000 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35de1ef700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d81461e0 con 0x7f35d80ba800 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49956/0 (socket says 172.21.15.185:49956) 2020-11-16T22:14:31.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2929237543 learned_addr learned my addr 172.21.15.185:0/2929237543 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dd9ee700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35d8144000 0x7f35d8149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dd1ed700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d80b5c40 0x7f35d8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35d8144000 msgr2=0x7f35d8149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35d8144000 0x7f35d8149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2929237543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d80b5c40 msgr2=0x7f35d8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d80b5c40 0x7f35d8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 -- 172.21.15.185:0/2929237543 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8146480 con 0x7f35d80ba800 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.743+0000 7f35dc9ec700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f35d80b83d0 tx=0x7f35cc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc011d50 con 0x7f35d80ba800 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc01e460 con 0x7f35d80ba800 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d81540b0 con 0x7f35d80ba800 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc01e7e0 con 0x7f35d80ba800 2020-11-16T22:14:31.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d80ad570 con 0x7f35d80ba800 2020-11-16T22:14:31.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f35cc025030 con 0x7f35d80ba800 2020-11-16T22:14:31.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8146610 con 0x7f35d80ba800 2020-11-16T22:14:31.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:31.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35dd1ed700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:31.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35cc06f5c0 con 0x7f35d80ba800 2020-11-16T22:14:31.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.747+0000 7f35dd1ed700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 0x7f35b0057520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f35d4007f80 tx=0x7f35d4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:31.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.751+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f35cc0457c0 con 0x7f35d80ba800 2020-11-16T22:14:31.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.903+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f35d805d200 con 0x7f35d80ba800 2020-11-16T22:14:31.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35c9ffb700 1 -- 172.21.15.185:0/2929237543 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f35cc023070 con 0x7f35d80ba800 2020-11-16T22:14:31.893 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:31.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 msgr2=0x7f35b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 0x7f35b0057520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f35d4007f80 tx=0x7f35d4009040).stop 2020-11-16T22:14:31.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 msgr2=0x7f35d80ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:31.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f35d80b83d0 tx=0x7f35cc015000).stop 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 shutdown_connections 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35b0055060 0x7f35b0057520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35d8144000 0x7f35d8149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d80b5c40 0x7f35d8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 --2- 172.21.15.185:0/2929237543 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35d80ba800 0x7f35d80ae340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:31.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 >> 172.21.15.185:0/2929237543 conn(0x7f35d805ba80 msgr2=0x7f35d813d390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:31.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 shutdown_connections 2020-11-16T22:14:31.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:31.907+0000 7f35de1ef700 1 -- 172.21.15.185:0/2929237543 wait complete. 2020-11-16T22:14:31.907 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:32.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 Processor -- start 2020-11-16T22:14:32.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 -- start start 2020-11-16T22:14:32.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b300b38a0 0x7f9b300b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9b3005bc50 0x7f9b30144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b30144d60 0x7f9b3013fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b300ad350 con 0x7f9b30144d60 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b300b2690 con 0x7f9b3005bc50 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b371ec700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b300b2930 con 0x7f9b300b38a0 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b359e9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9b3005bc50 0x7f9b30144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b361ea700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b300b38a0 0x7f9b300b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b359e9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9b3005bc50 0x7f9b30144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56520/0 (socket says 172.21.15.185:56520) 2020-11-16T22:14:32.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b359e9700 1 -- 172.21.15.185:0/2901665388 learned_addr learned my addr 172.21.15.185:0/2901665388 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.043+0000 7f9b369eb700 1 --2- 172.21.15.185:0/2901665388 >> v2:172.21.15.96:3300/0 conn(0x7f9b30144d60 0x7f9b3013fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 -- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b300b38a0 msgr2=0x7f9b300b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 --2- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b300b38a0 0x7f9b300b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 -- 172.21.15.185:0/2901665388 >> v2:172.21.15.96:3300/0 conn(0x7f9b30144d60 msgr2=0x7f9b3013fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 --2- 172.21.15.185:0/2901665388 >> v2:172.21.15.96:3300/0 conn(0x7f9b30144d60 0x7f9b3013fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 -- 172.21.15.185:0/2901665388 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b301443d0 con 0x7f9b3005bc50 2020-11-16T22:14:32.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b359e9700 1 --2- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b3005bc50 0x7f9b30144870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9b300659f0 tx=0x7f9b20006eb0).ready entity=mon.1 client_cookie=51ff1e88406007aa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b351e8700 1 -- 172.21.15.185:0/2901665388 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20009810 con 0x7f9b3005bc50 2020-11-16T22:14:32.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b351e8700 1 -- 172.21.15.185:0/2901665388 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20009970 con 0x7f9b3005bc50 2020-11-16T22:14:32.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b351e8700 1 -- 172.21.15.185:0/2901665388 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20009cf0 con 0x7f9b3005bc50 2020-11-16T22:14:32.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 -- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b3005bc50 msgr2=0x7f9b30144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b3005bc50 0x7f9b30144870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9b300659f0 tx=0x7f9b20006eb0).stop 2020-11-16T22:14:32.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 -- 172.21.15.185:0/2901665388 shutdown_connections 2020-11-16T22:14:32.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2901665388 >> v2:172.21.15.96:3300/0 conn(0x7f9b30144d60 0x7f9b3013fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b300b38a0 0x7f9b300b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2901665388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b3005bc50 0x7f9b30144870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 -- 172.21.15.185:0/2901665388 >> 172.21.15.185:0/2901665388 conn(0x7f9b30063df0 msgr2=0x7f9b300641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 -- 172.21.15.185:0/2901665388 shutdown_connections 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.047+0000 7f9b371ec700 1 -- 172.21.15.185:0/2901665388 wait complete. 2020-11-16T22:14:32.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 Processor -- start 2020-11-16T22:14:32.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- start start 2020-11-16T22:14:32.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 0x7f9b300ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b300adaf0 con 0x7f9b3005bc50 2020-11-16T22:14:32.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301535d0 con 0x7f9b300b38a0 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b300b0070 con 0x7f9b30144d60 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b359e9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b359e9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56526/0 (socket says 172.21.15.185:56526) 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:49972/0 (socket says 172.21.15.185:49972) 2020-11-16T22:14:32.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b359e9700 1 -- 172.21.15.185:0/2389459635 learned_addr learned my addr 172.21.15.185:0/2389459635 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b361ea700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 0x7f9b300ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 -- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 msgr2=0x7f9b3013d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 -- 172.21.15.185:0/2389459635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 msgr2=0x7f9b300ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 0x7f9b300ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 -- 172.21.15.185:0/2389459635 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30153840 con 0x7f9b30144d60 2020-11-16T22:14:32.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b361ea700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 0x7f9b300ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b359e9700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b369eb700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9b2800eb40 tx=0x7f9b28008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28017070 con 0x7f9b30144d60 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3013f060 con 0x7f9b30144d60 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b280095b0 con 0x7f9b30144d60 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3013dbb0 con 0x7f9b30144d60 2020-11-16T22:14:32.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.051+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28009930 con 0x7f9b30144d60 2020-11-16T22:14:32.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b301445b0 con 0x7f9b30144d60 2020-11-16T22:14:32.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9b28009d20 con 0x7f9b30144d60 2020-11-16T22:14:32.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b1e7fc700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 0x7f9b08057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9b2806dad0 con 0x7f9b30144d60 2020-11-16T22:14:32.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b361ea700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 0x7f9b08057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.055+0000 7f9b361ea700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 0x7f9b08057520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9b2c006fd0 tx=0x7f9b2c008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.059+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9b301445b0 con 0x7f9b30144d60 2020-11-16T22:14:32.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.211+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9b301445b0 con 0x7f9b30144d60 2020-11-16T22:14:32.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.211+0000 7f9b1e7fc700 1 -- 172.21.15.185:0/2389459635 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9b301445b0 con 0x7f9b30144d60 2020-11-16T22:14:32.200 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:32.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 msgr2=0x7f9b08057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 0x7f9b08057520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9b2c006fd0 tx=0x7f9b2c008040).stop 2020-11-16T22:14:32.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 msgr2=0x7f9b300af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f9b2800eb40 tx=0x7f9b28008040).stop 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 shutdown_connections 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b08055060 0x7f9b08057520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b30144d60 0x7f9b300af690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3005bc50 0x7f9b300ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 --2- 172.21.15.185:0/2389459635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b300b38a0 0x7f9b3013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 >> 172.21.15.185:0/2389459635 conn(0x7f9b30063df0 msgr2=0x7f9b3013b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 shutdown_connections 2020-11-16T22:14:32.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.215+0000 7f9b371ec700 1 -- 172.21.15.185:0/2389459635 wait complete. 2020-11-16T22:14:32.218 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:32.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 Processor -- start 2020-11-16T22:14:32.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 -- start start 2020-11-16T22:14:32.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f71800645e0 0x7f71800649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800b4200 con 0x7f7180061750 2020-11-16T22:14:32.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800b44a0 con 0x7f7180062f40 2020-11-16T22:14:32.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f71879f6700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800b4740 con 0x7f71800645e0 2020-11-16T22:14:32.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f71800645e0 0x7f71800649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f71800645e0 0x7f71800649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56532/0 (socket says 172.21.15.185:56532) 2020-11-16T22:14:32.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 -- 172.21.15.185:0/2279726572 learned_addr learned my addr 172.21.15.185:0/2279726572 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7184f91700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7180062f40 0x7f718013dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:49974/0 (socket says 172.21.15.185:49974) 2020-11-16T22:14:32.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f716ffff700 1 -- 172.21.15.185:0/2279726572 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005761127 0 0) 0x7f71800b44a0 con 0x7f7180062f40 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 -- 172.21.15.185:0/2279726572 >> v1:172.21.15.185:6789/0 conn(0x7f7180062f40 legacy=0x7f718013dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 -- 172.21.15.185:0/2279726572 >> v1:172.21.15.96:6789/0 conn(0x7f7180061750 legacy=0x7f71801476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 -- 172.21.15.185:0/2279726572 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800b49e0 con 0x7f71800645e0 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f7185792700 1 --2- 172.21.15.185:0/2279726572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f71800649c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f717c006fd0 tx=0x7f717c006eb0).ready entity=mon.1 client_cookie=1bb42f93366e9935 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f716ffff700 1 -- 172.21.15.185:0/2279726572 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0098d0 con 0x7f71800645e0 2020-11-16T22:14:32.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f716ffff700 1 -- 172.21.15.185:0/2279726572 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c009a30 con 0x7f71800645e0 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.355+0000 7f716ffff700 1 -- 172.21.15.185:0/2279726572 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c009db0 con 0x7f71800645e0 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- 172.21.15.185:0/2279726572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 msgr2=0x7f71800649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 --2- 172.21.15.185:0/2279726572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f71800649c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f717c006fd0 tx=0x7f717c006eb0).stop 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- 172.21.15.185:0/2279726572 shutdown_connections 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 --2- 172.21.15.185:0/2279726572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f71800649c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- 172.21.15.185:0/2279726572 >> 172.21.15.185:0/2279726572 conn(0x7f7180060130 msgr2=0x7f7180060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- 172.21.15.185:0/2279726572 shutdown_connections 2020-11-16T22:14:32.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- 172.21.15.185:0/2279726572 wait complete. 2020-11-16T22:14:32.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 Processor -- start 2020-11-16T22:14:32.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- start start 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180061750 0x7f71800b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7180062f40 0x7f71801408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180140720 con 0x7f7180061750 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b0780 con 0x7f71800645e0 2020-11-16T22:14:32.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f71879f6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b0fd0 con 0x7f7180062f40 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56540/0 (socket says 172.21.15.185:56540) 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7184f91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7180062f40 0x7f71801408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 -- 172.21.15.185:0/851660658 learned_addr learned my addr 172.21.15.185:0/851660658 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185792700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180061750 0x7f71800b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 -- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7180062f40 msgr2=0x7f71801408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7180062f40 0x7f71801408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 -- 172.21.15.185:0/851660658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180061750 msgr2=0x7f71800b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180061750 0x7f71800b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 -- 172.21.15.185:0/851660658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71801412d0 con 0x7f71800645e0 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.359+0000 7f7185f93700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f717800ca60 tx=0x7f7178007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800a730 con 0x7f71800645e0 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800a890 con 0x7f71800645e0 2020-11-16T22:14:32.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800b2890 con 0x7f71800645e0 2020-11-16T22:14:32.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800ac10 con 0x7f71800645e0 2020-11-16T22:14:32.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800b0300 con 0x7f71800645e0 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7178019030 con 0x7f71800645e0 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718005d4e0 con 0x7f71800645e0 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 0x7f7158057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f717806d4c0 con 0x7f71800645e0 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f7185792700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 0x7f7158057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.363+0000 7f7185792700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 0x7f7158057520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f718014b670 tx=0x7f717c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.371+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f71780437d0 con 0x7f71800645e0 2020-11-16T22:14:32.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.515+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71800638c0 con 0x7f71800645e0 2020-11-16T22:14:32.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f716d7fa700 1 -- 172.21.15.185:0/851660658 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7178046df0 con 0x7f71800645e0 2020-11-16T22:14:32.505 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:32.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 msgr2=0x7f7158057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 0x7f7158057520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f718014b670 tx=0x7f717c015000).stop 2020-11-16T22:14:32.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 msgr2=0x7f7180140de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f717800ca60 tx=0x7f7178007eb0).stop 2020-11-16T22:14:32.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 shutdown_connections 2020-11-16T22:14:32.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7158055060 0x7f7158057520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f71800645e0 0x7f7180140de0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7180061750 0x7f71800b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 --2- 172.21.15.185:0/851660658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7180062f40 0x7f71801408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 >> 172.21.15.185:0/851660658 conn(0x7f7180060130 msgr2=0x7f7180147870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 shutdown_connections 2020-11-16T22:14:32.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.519+0000 7f71879f6700 1 -- 172.21.15.185:0/851660658 wait complete. 2020-11-16T22:14:32.519 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:32.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 Processor -- start 2020-11-16T22:14:32.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 -- start start 2020-11-16T22:14:32.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcd1c0b3500 0x7fcd1c0677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c143f80 con 0x7fcd1c060bf0 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c067ca0 con 0x7fcd1c0b3500 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.659+0000 7fcd20aa7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c067f30 con 0x7fcd1c0b7510 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56546/0 (socket says 172.21.15.185:56546) 2020-11-16T22:14:32.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 -- 172.21.15.185:0/1241188364 learned_addr learned my addr 172.21.15.185:0/1241188364 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1affd700 1 --2- 172.21.15.185:0/1241188364 >> v2:172.21.15.96:3300/0 conn(0x7fcd1c0b3500 0x7fcd1c0677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 -- 172.21.15.185:0/1241188364 >> v2:172.21.15.96:3300/0 conn(0x7fcd1c0b3500 msgr2=0x7fcd1c0677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 --2- 172.21.15.185:0/1241188364 >> v2:172.21.15.96:3300/0 conn(0x7fcd1c0b3500 0x7fcd1c0677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 -- 172.21.15.185:0/1241188364 >> v1:172.21.15.185:6789/0 conn(0x7fcd1c060bf0 legacy=0x7fcd1c0667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 -- 172.21.15.185:0/1241188364 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0681d0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1a7fc700 1 -- 172.21.15.185:0/1241188364 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789063210 0 0) 0x7fcd1c143f80 con 0x7fcd1c060bf0 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1b7fe700 1 --2- 172.21.15.185:0/1241188364 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fcd0c006fd0 tx=0x7fcd0c006eb0).ready entity=mon.1 client_cookie=2de645fa1a626b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1a7fc700 1 -- 172.21.15.185:0/1241188364 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0097d0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1a7fc700 1 -- 172.21.15.185:0/1241188364 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c009930 con 0x7fcd1c0b7510 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd1a7fc700 1 -- 172.21.15.185:0/1241188364 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c009cb0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1241188364 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 msgr2=0x7fcd1c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1241188364 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fcd0c006fd0 tx=0x7fcd0c006eb0).stop 2020-11-16T22:14:32.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1241188364 shutdown_connections 2020-11-16T22:14:32.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1241188364 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c0b78f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1241188364 >> v2:172.21.15.96:3300/0 conn(0x7fcd1c0b3500 0x7fcd1c0677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1241188364 >> 172.21.15.185:0/1241188364 conn(0x7fcd1c064b90 msgr2=0x7fcd1c063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1241188364 shutdown_connections 2020-11-16T22:14:32.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.663+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1241188364 wait complete. 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 Processor -- start 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- start start 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd1c060bf0 0x7fcd1c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcd1c143090 0x7fcd1c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c13e660 con 0x7fcd1c060bf0 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c154280 con 0x7fcd1c0b7510 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c13f240 con 0x7fcd1c143090 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56554/0 (socket says 172.21.15.185:56554) 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 -- 172.21.15.185:0/1947594959 learned_addr learned my addr 172.21.15.185:0/1947594959 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1b7fe700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd1c060bf0 0x7fcd1c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1bfff700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcd1c143090 0x7fcd1c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 -- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcd1c143090 msgr2=0x7fcd1c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcd1c143090 0x7fcd1c143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 -- 172.21.15.185:0/1947594959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd1c060bf0 msgr2=0x7fcd1c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd1c060bf0 0x7fcd1c153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 -- 172.21.15.185:0/1947594959 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c13f4e0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd1affd700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcd1000ec00 tx=0x7fcd10006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000c8e0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000ca40 con 0x7fcd1c0b7510 2020-11-16T22:14:32.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c13f780 con 0x7fcd1c0b7510 2020-11-16T22:14:32.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000cdc0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c145bd0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.667+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c05d200 con 0x7fcd1c0b7510 2020-11-16T22:14:32.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.671+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcd1001a030 con 0x7fcd1c0b7510 2020-11-16T22:14:32.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.671+0000 7fccfb7fe700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 0x7fccf4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.671+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd1006d890 con 0x7fcd1c0b7510 2020-11-16T22:14:32.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.671+0000 7fcd1b7fe700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 0x7fccf4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.671+0000 7fcd1b7fe700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 0x7fccf4057520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcd0c00b1b0 tx=0x7fcd0c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.675+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcd10043ba0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.827+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd1c1462d0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fccfb7fe700 1 -- 172.21.15.185:0/1947594959 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcd100471c0 con 0x7fcd1c0b7510 2020-11-16T22:14:32.816 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:32.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 msgr2=0x7fccf4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 0x7fccf4057520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcd0c00b1b0 tx=0x7fcd0c015000).stop 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 msgr2=0x7fcd1c145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcd1000ec00 tx=0x7fcd10006df0).stop 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 shutdown_connections 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fccf4055060 0x7fccf4057520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcd1c143090 0x7fcd1c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd1c060bf0 0x7fcd1c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 --2- 172.21.15.185:0/1947594959 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcd1c0b7510 0x7fcd1c145660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 >> 172.21.15.185:0/1947594959 conn(0x7fcd1c064b90 msgr2=0x7fcd1c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 shutdown_connections 2020-11-16T22:14:32.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.831+0000 7fcd20aa7700 1 -- 172.21.15.185:0/1947594959 wait complete. 2020-11-16T22:14:32.831 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:14:32.832 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:32.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 Processor -- start 2020-11-16T22:14:32.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 -- start start 2020-11-16T22:14:32.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc0680b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc0680b64b0 0x7fc0680b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc068147f90 con 0x7fc0680acee0 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680af020 con 0x7fc0680b64b0 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.963+0000 7fc06cda3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b4810 con 0x7fc0680b72e0 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc0680b64b0 0x7fc0680b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc0677fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc0680b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc0680b64b0 0x7fc0680b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56560/0 (socket says 172.21.15.185:56560) 2020-11-16T22:14:32.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 -- 172.21.15.185:0/2284676736 learned_addr learned my addr 172.21.15.185:0/2284676736 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 -- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 msgr2=0x7fc0680b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc0680b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 -- 172.21.15.185:0/2284676736 >> v1:172.21.15.96:6789/0 conn(0x7fc0680acee0 legacy=0x7fc068144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 -- 172.21.15.185:0/2284676736 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680ad370 con 0x7fc0680b64b0 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc0677fe700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc0680b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:32.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc066ffd700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b6aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc050009d50 tx=0x7fc050009290).ready entity=mon.1 client_cookie=2bf9e78897b8775d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc0667fc700 1 -- 172.21.15.185:0/2284676736 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05000bbc0 con 0x7fc0680b64b0 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc0667fc700 1 -- 172.21.15.185:0/2284676736 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05000bd20 con 0x7fc0680b64b0 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc0667fc700 1 -- 172.21.15.185:0/2284676736 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050011610 con 0x7fc0680b64b0 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 -- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 msgr2=0x7fc0680b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b6aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc050009d50 tx=0x7fc050009290).stop 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 -- 172.21.15.185:0/2284676736 shutdown_connections 2020-11-16T22:14:32.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc0680b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 --2- 172.21.15.185:0/2284676736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b6aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 -- 172.21.15.185:0/2284676736 >> 172.21.15.185:0/2284676736 conn(0x7fc068067860 msgr2=0x7fc068067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:32.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 -- 172.21.15.185:0/2284676736 shutdown_connections 2020-11-16T22:14:32.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.967+0000 7fc06cda3700 1 -- 172.21.15.185:0/2284676736 wait complete. 2020-11-16T22:14:32.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 Processor -- start 2020-11-16T22:14:32.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- start start 2020-11-16T22:14:32.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0680acee0 0x7fc0680b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc068152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068114390 con 0x7fc0680acee0 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b3b70 con 0x7fc0680b64b0 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06813fd90 con 0x7fc0680b72e0 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc066ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc068152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc066ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56568/0 (socket says 172.21.15.185:56568) 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc066ffd700 1 -- 172.21.15.185:0/4091210460 learned_addr learned my addr 172.21.15.185:0/4091210460 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:32.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc0677fe700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0680acee0 0x7fc0680b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 -- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 msgr2=0x7fc0680b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 -- 172.21.15.185:0/4091210460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0680acee0 msgr2=0x7fc0680b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:32.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0680acee0 0x7fc0680b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:32.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 -- 172.21.15.185:0/4091210460 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068148460 con 0x7fc0680b72e0 2020-11-16T22:14:32.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc067fff700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc068152ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc06000c6e0 tx=0x7fc060007aa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000f470 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06000f5d0 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068113bf0 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000f950 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.971+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680b2680 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0681141d0 con 0x7fc0680b72e0 2020-11-16T22:14:32.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc060019030 con 0x7fc0680b72e0 2020-11-16T22:14:32.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc05f7fe700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 0x7fc038057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:32.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc06006ce90 con 0x7fc0680b72e0 2020-11-16T22:14:32.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc0677fe700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 0x7fc038057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:32.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.975+0000 7fc0677fe700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 0x7fc038057520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fc058006fd0 tx=0x7fc058008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:32.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:32.979+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc0600431a0 con 0x7fc0680b72e0 2020-11-16T22:14:33.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.179+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc06805d200 con 0x7fc0680b72e0 2020-11-16T22:14:33.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.179+0000 7fc05f7fe700 1 -- 172.21.15.185:0/4091210460 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2511 (secure 0 0 0) 0x7fc0600467c0 con 0x7fc0680b72e0 2020-11-16T22:14:33.168 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:33.168 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:11.589550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:33.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 msgr2=0x7fc038057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:33.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 0x7fc038057520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fc058006fd0 tx=0x7fc058008040).stop 2020-11-16T22:14:33.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 msgr2=0x7fc068152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:33.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc068152ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fc06000c6e0 tx=0x7fc060007aa0).stop 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 shutdown_connections 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc038055060 0x7fc038057520 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc0680b72e0 0x7fc068152ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0680acee0 0x7fc0680b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 --2- 172.21.15.185:0/4091210460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc0680b64b0 0x7fc0680b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 >> 172.21.15.185:0/4091210460 conn(0x7fc068067860 msgr2=0x7fc068147170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 shutdown_connections 2020-11-16T22:14:33.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:33.183+0000 7fc06cda3700 1 -- 172.21.15.185:0/4091210460 wait complete. 2020-11-16T22:14:33.172 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 10 2020-11-16T22:14:33.489 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:14:07.542849+0000', 'modified': '2020-11-16T22:14:11.589550+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4278, 'mds_1': 4273, 'mds_2': 4217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4217': {'gid': 4217, 'name': 'a', 'rank': 2, 'incarnation': 9, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.185:6835/1427562659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1427562659}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1427562659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4273': {'gid': 4273, 'name': 'c', 'rank': 1, 'incarnation': 7, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6833/93135634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 93135634}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 93135634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3128886787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3128886787}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3128886787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:14:33.490 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:14:33.490 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:14:33.631 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":4217,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.048243294999999,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":33.606768785} 2020-11-16T22:14:33.632 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4217, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 23.048243295, "state": "up:active", "uptime": 33.606768785, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:14:33.632 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:14:33.772 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":4273,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.200227942000001,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":33.737985526999999} 2020-11-16T22:14:33.772 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4273, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 24.200227942, "state": "up:active", "uptime": 33.737985527, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:14:33.773 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:14:33.907 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4278,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":26.366803772000001,"mdsmap_epoch":10,"osdmap_epoch":22,"osdmap_epoch_barrier":22,"uptime":33.857767912} 2020-11-16T22:14:33.907 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4278, "mdsmap_epoch": 10, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "rank_uptime": 26.366803772, "state": "up:active", "uptime": 33.857767912, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:14:33.907 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:14:33.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:33.908 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-16T22:14:33.926 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:14:33.926 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:33.995 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:33.996 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-16T22:14:34.020 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:14:34.020 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-16T22:14:34.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:14:34.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:34.028 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-16T22:14:35.054 INFO:teuthology.run_tasks:Running task check-counter... 2020-11-16T22:14:35.066 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-16T22:14:35.106 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:35.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 Processor -- start 2020-11-16T22:14:35.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 -- start start 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efe440669a0 0x7efe44066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 0x7efe44147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440abf90 con 0x7efe44061e60 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe440ac230 con 0x7efe440669a0 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4ac58700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440ac4d0 con 0x7efe440acc10 2020-11-16T22:14:35.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe49c56700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efe440669a0 0x7efe44066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe49455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 0x7efe44147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4a457700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7efe44061e60 0x7efe44147a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50016/0 (socket says 172.21.15.185:50016) 2020-11-16T22:14:35.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe4a457700 1 -- 172.21.15.185:0/2334606182 learned_addr learned my addr 172.21.15.185:0/2334606182 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:35.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.251+0000 7efe48c54700 1 -- 172.21.15.185:0/2334606182 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819028186 0 0) 0x7efe440abf90 con 0x7efe44061e60 2020-11-16T22:14:35.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe48c54700 1 -- 172.21.15.185:0/2334606182 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe24003240 con 0x7efe44061e60 2020-11-16T22:14:35.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 -- 172.21.15.185:0/2334606182 >> v2:172.21.15.185:3300/0 conn(0x7efe440669a0 msgr2=0x7efe44066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 --2- 172.21.15.185:0/2334606182 >> v2:172.21.15.185:3300/0 conn(0x7efe440669a0 0x7efe44066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 -- 172.21.15.185:0/2334606182 >> v1:172.21.15.185:6789/0 conn(0x7efe44061e60 legacy=0x7efe44147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 -- 172.21.15.185:0/2334606182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440ac770 con 0x7efe440acc10 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 --2- 172.21.15.185:0/2334606182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 0x7efe44147550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7efe2c009c90 tx=0x7efe2c0091d0).ready entity=mon.2 client_cookie=3833f2edd454bfbf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe48c54700 1 -- 172.21.15.185:0/2334606182 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00b9c0 con 0x7efe440acc10 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe48c54700 1 -- 172.21.15.185:0/2334606182 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00bb20 con 0x7efe440acc10 2020-11-16T22:14:35.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe48c54700 1 -- 172.21.15.185:0/2334606182 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00bea0 con 0x7efe440acc10 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- 172.21.15.185:0/2334606182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 msgr2=0x7efe44147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- 172.21.15.185:0/2334606182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 0x7efe44147550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7efe2c009c90 tx=0x7efe2c0091d0).stop 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- 172.21.15.185:0/2334606182 shutdown_connections 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- 172.21.15.185:0/2334606182 >> v2:172.21.15.185:3300/0 conn(0x7efe440669a0 0x7efe44066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- 172.21.15.185:0/2334606182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440acc10 0x7efe44147550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- 172.21.15.185:0/2334606182 >> 172.21.15.185:0/2334606182 conn(0x7efe44060130 msgr2=0x7efe44060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- 172.21.15.185:0/2334606182 shutdown_connections 2020-11-16T22:14:35.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- 172.21.15.185:0/2334606182 wait complete. 2020-11-16T22:14:35.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 Processor -- start 2020-11-16T22:14:35.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- start start 2020-11-16T22:14:35.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44061e60 0x7efe440aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe440b4df0 0x7efe440af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440ae000 con 0x7efe44061e60 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440b0630 con 0x7efe440b4df0 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4ac58700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe44114d90 con 0x7efe440669a0 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50026/0 (socket says 172.21.15.185:50026) 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49455700 1 -- 172.21.15.185:0/3073500584 learned_addr learned my addr 172.21.15.185:0/3073500584 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe4a457700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe440b4df0 0x7efe440af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.255+0000 7efe49c56700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44061e60 0x7efe440aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 -- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe440b4df0 msgr2=0x7efe440af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe440b4df0 0x7efe440af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 -- 172.21.15.185:0/3073500584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44061e60 msgr2=0x7efe440aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44061e60 0x7efe440aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 -- 172.21.15.185:0/3073500584 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe441145b0 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49455700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7efe2c00baf0 tx=0x7efe2c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c01d480 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe441148b0 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c01d5e0 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c01d960 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe440b57d0 con 0x7efe440669a0 2020-11-16T22:14:35.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efe2c024030 con 0x7efe440669a0 2020-11-16T22:14:35.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4405d200 con 0x7efe440669a0 2020-11-16T22:14:35.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 0x7efe20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe2c06f640 con 0x7efe440669a0 2020-11-16T22:14:35.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49c56700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 0x7efe20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.259+0000 7efe49c56700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 0x7efe20057520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7efe34006fd0 tx=0x7efe34008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.267+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efe2c045950 con 0x7efe440669a0 2020-11-16T22:14:35.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe44144f30 con 0x7efe440669a0 2020-11-16T22:14:35.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe41ffb700 1 -- 172.21.15.185:0/3073500584 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7efe2c0301d0 con 0x7efe440669a0 2020-11-16T22:14:35.407 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:35.407 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:35.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 msgr2=0x7efe20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 0x7efe20057520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7efe34006fd0 tx=0x7efe34008040).stop 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 msgr2=0x7efe440b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.419+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7efe2c00baf0 tx=0x7efe2c016000).stop 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 shutdown_connections 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe20055060 0x7efe20057520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe440b4df0 0x7efe440af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe44061e60 0x7efe440aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 --2- 172.21.15.185:0/3073500584 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe440669a0 0x7efe440b4900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 >> 172.21.15.185:0/3073500584 conn(0x7efe44060130 msgr2=0x7efe44149d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 shutdown_connections 2020-11-16T22:14:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.423+0000 7efe4ac58700 1 -- 172.21.15.185:0/3073500584 wait complete. 2020-11-16T22:14:35.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:35.466 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:35.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 Processor -- start 2020-11-16T22:14:35.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 -- start start 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03500b5c40 0x7f03500b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03501444b0 con 0x7f0350144000 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03501446e0 con 0x7f03500ba800 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0356bf3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0350144980 con 0x7f03500b5c40 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f03553f0700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f03500ba800 0x7f035013dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50030/0 (socket says 172.21.15.185:50030) 2020-11-16T22:14:35.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f03553f0700 1 -- 172.21.15.185:0/3084288209 learned_addr learned my addr 172.21.15.185:0/3084288209 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:35.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03500b5c40 0x7f03500b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411991504 0 0) 0x7f03501446e0 con 0x7f03500ba800 2020-11-16T22:14:35.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0338003240 con 0x7f03500ba800 2020-11-16T22:14:35.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4200081067 0 0) 0x7f0338003240 con 0x7f03500ba800 2020-11-16T22:14:35.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03500b5c40 msgr2=0x7f03500b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:35.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03500b5c40 0x7f03500b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 >> v1:172.21.15.96:6789/0 conn(0x7f0350144000 legacy=0x7f03500b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350144c20 con 0x7f03500ba800 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.611+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f03400014e0 con 0x7f03500ba800 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 >> v1:172.21.15.185:6789/0 conn(0x7f03500ba800 legacy=0x7f035013dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0338003730 0x7f0338003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0338004070 0x7f0338004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0338004af0 0x7f0338004f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0350144980 con 0x7f0338004af0 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033800b800 con 0x7f0338003730 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033800b9c0 con 0x7f0338004070 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0338004070 0x7f0338004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03500b5c40 0x7f03500b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0338003730 0x7f0338003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03563f2700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0338004af0 0x7f0338004f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 -- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0338003730 msgr2=0x7f0338003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0338003730 0x7f0338003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 -- 172.21.15.185:0/3084288209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0338004af0 msgr2=0x7f0338004f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0338004af0 0x7f0338004f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 -- 172.21.15.185:0/3084288209 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338003240 con 0x7f0338004070 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0338003730 0x7f0338003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03553f0700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0338004070 0x7f0338004540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f034000c410 tx=0x7f03400077d0).ready entity=mon.2 client_cookie=557cd5921760d07d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400070d0 con 0x7f0338004070 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340007230 con 0x7f0338004070 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0354bef700 1 -- 172.21.15.185:0/3084288209 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400055f0 con 0x7f0338004070 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0338004070 msgr2=0x7f0338004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3084288209 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0338004070 0x7f0338004540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f034000c410 tx=0x7f03400077d0).stop 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f03563f2700 1 -- 172.21.15.185:0/3084288209 reap_dead start 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- 172.21.15.185:0/3084288209 shutdown_connections 2020-11-16T22:14:35.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- 172.21.15.185:0/3084288209 >> 172.21.15.185:0/3084288209 conn(0x7f035005ba80 msgr2=0x7f035005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:35.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- 172.21.15.185:0/3084288209 shutdown_connections 2020-11-16T22:14:35.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- 172.21.15.185:0/3084288209 wait complete. 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 Processor -- start 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.615+0000 7f0356bf3700 1 -- start start 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035013e940 0x7f035013ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 0x7f03500b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500adea0 con 0x7f035013e940 2020-11-16T22:14:35.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500ae140 con 0x7f03500b1160 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500ae3e0 con 0x7f0350153db0 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03553f0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 0x7f03500b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0355bf1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035013e940 0x7f035013ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56602/0 (socket says 172.21.15.185:56602) 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03553f0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 0x7f03500b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50046/0 (socket says 172.21.15.185:50046) 2020-11-16T22:14:35.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 -- 172.21.15.185:0/3515368498 learned_addr learned my addr 172.21.15.185:0/3515368498 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 -- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 msgr2=0x7f03500b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 0x7f03500b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 -- 172.21.15.185:0/3515368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035013e940 msgr2=0x7f035013ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035013e940 0x7f035013ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 -- 172.21.15.185:0/3515368498 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500ae680 con 0x7f03500b1160 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f03563f2700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0350081070 tx=0x7f0348007e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800eec0 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348006a80 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0350154230 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348006e00 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0350140420 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0350144e00 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.619+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0348002230 con 0x7f03500b1160 2020-11-16T22:14:35.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.623+0000 7f033dffb700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 0x7f0328057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.623+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f034806d820 con 0x7f03500b1160 2020-11-16T22:14:35.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.623+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 0x7f0328057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.623+0000 7f0355bf1700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 0x7f0328057520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f034c006a90 tx=0x7f034c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.627+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0348043a80 con 0x7f03500b1160 2020-11-16T22:14:35.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.771+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f035005d200 con 0x7f03500b1160 2020-11-16T22:14:35.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f033dffb700 1 -- 172.21.15.185:0/3515368498 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f03480470a0 con 0x7f03500b1160 2020-11-16T22:14:35.761 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:35.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 msgr2=0x7f0328057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 0x7f0328057520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f034c006a90 tx=0x7f034c006df0).stop 2020-11-16T22:14:35.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 msgr2=0x7f035013ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0350081070 tx=0x7f0348007e60).stop 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 shutdown_connections 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0328055060 0x7f0328057520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03500b1160 0x7f035013ff30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035013e940 0x7f035013ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 --2- 172.21.15.185:0/3515368498 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0350153db0 0x7f03500b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 >> 172.21.15.185:0/3515368498 conn(0x7f035005ba80 msgr2=0x7f03500b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:35.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 shutdown_connections 2020-11-16T22:14:35.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.775+0000 7f0356bf3700 1 -- 172.21.15.185:0/3515368498 wait complete. 2020-11-16T22:14:35.775 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:35.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 Processor -- start 2020-11-16T22:14:35.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 -- start start 2020-11-16T22:14:35.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07500b4c30 0x7f0750145c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07500af570 con 0x7f0750063f90 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07500b3d00 con 0x7f07500af780 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.919+0000 7f0754d46700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07500af2d0 con 0x7f07500b4c30 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f07500af780 0x7f075014b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50050/0 (socket says 172.21.15.185:50050) 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074ffff700 1 -- 172.21.15.185:0/2990154528 learned_addr learned my addr 172.21.15.185:0/2990154528 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:35.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074effd700 1 --2- 172.21.15.185:0/2990154528 >> v2:172.21.15.96:3300/0 conn(0x7f07500b4c30 0x7f0750145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181223973 0 0) 0x7f07500b3d00 con 0x7f07500af780 2020-11-16T22:14:35.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f073c003240 con 0x7f07500af780 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1621662292 0 0) 0x7f073c003240 con 0x7f07500af780 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074effd700 1 -- 172.21.15.185:0/2990154528 >> v1:172.21.15.185:6789/0 conn(0x7f07500af780 legacy=0x7f075014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074effd700 1 -- 172.21.15.185:0/2990154528 >> v1:172.21.15.96:6789/0 conn(0x7f0750063f90 legacy=0x7f0750145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074effd700 1 -- 172.21.15.185:0/2990154528 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07500b3190 con 0x7f07500b4c30 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074effd700 1 --2- 172.21.15.185:0/2990154528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07500b4c30 0x7f0750145c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f074400b920 tx=0x7f0744006df0).ready entity=mon.0 client_cookie=466f2f82c174d211 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748001860 con 0x7f07500b4c30 2020-11-16T22:14:35.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744012460 con 0x7f07500b4c30 2020-11-16T22:14:35.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f074e7fc700 1 -- 172.21.15.185:0/2990154528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440127e0 con 0x7f07500b4c30 2020-11-16T22:14:35.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 -- 172.21.15.185:0/2990154528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07500b4c30 msgr2=0x7f0750145c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 --2- 172.21.15.185:0/2990154528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07500b4c30 0x7f0750145c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f074400b920 tx=0x7f0744006df0).stop 2020-11-16T22:14:35.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 -- 172.21.15.185:0/2990154528 shutdown_connections 2020-11-16T22:14:35.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 --2- 172.21.15.185:0/2990154528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07500b4c30 0x7f0750145c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 -- 172.21.15.185:0/2990154528 >> 172.21.15.185:0/2990154528 conn(0x7f0750067940 msgr2=0x7f0750067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:35.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 -- 172.21.15.185:0/2990154528 shutdown_connections 2020-11-16T22:14:35.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.923+0000 7f0754d46700 1 -- 172.21.15.185:0/2990154528 wait complete. 2020-11-16T22:14:35.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 Processor -- start 2020-11-16T22:14:35.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 -- start start 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0750063f90 0x7f07500ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f07500b4c30 0x7f0750151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0750152230 con 0x7f0750063f90 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0750144600 con 0x7f07500af780 2020-11-16T22:14:35.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f0754d46700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07501448a0 con 0x7f07500b4c30 2020-11-16T22:14:35.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f074effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f074effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56616/0 (socket says 172.21.15.185:56616) 2020-11-16T22:14:35.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f074effd700 1 -- 172.21.15.185:0/1116597474 learned_addr learned my addr 172.21.15.185:0/1116597474 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:35.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f074ffff700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f07500b4c30 0x7f0750151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.927+0000 7f074f7fe700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0750063f90 0x7f07500ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 -- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f07500b4c30 msgr2=0x7f0750151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f07500b4c30 0x7f0750151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 -- 172.21.15.185:0/1116597474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0750063f90 msgr2=0x7f07500ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0750063f90 0x7f07500ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 -- 172.21.15.185:0/1116597474 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750144b40 con 0x7f07500af780 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074effd700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f07440092c0 tx=0x7f0744006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440118e0 con 0x7f07500af780 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744017070 con 0x7f07500af780 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07501512a0 con 0x7f07500af780 2020-11-16T22:14:35.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744011a60 con 0x7f07500af780 2020-11-16T22:14:35.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07500ac7c0 con 0x7f07500af780 2020-11-16T22:14:35.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075005d200 con 0x7f07500af780 2020-11-16T22:14:35.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f074400f030 con 0x7f07500af780 2020-11-16T22:14:35.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 0x7f0728057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:35.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f074406e650 con 0x7f07500af780 2020-11-16T22:14:35.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.931+0000 7f074f7fe700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 0x7f0728057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:35.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.935+0000 7f074f7fe700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 0x7f0728057520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0740000bc0 tx=0x7f0740007a90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:35.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:35.939+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0744044960 con 0x7f07500af780 2020-11-16T22:14:36.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07500b4a10 con 0x7f07500af780 2020-11-16T22:14:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f072f7fe700 1 -- 172.21.15.185:0/1116597474 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0744047f80 con 0x7f07500af780 2020-11-16T22:14:36.082 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 msgr2=0x7f0728057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 0x7f0728057520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f0740000bc0 tx=0x7f0740007a90).stop 2020-11-16T22:14:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 msgr2=0x7f0750151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f07440092c0 tx=0x7f0744006df0).stop 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 shutdown_connections 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0728055060 0x7f0728057520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f07500b4c30 0x7f0750151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0750063f90 0x7f07500ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 --2- 172.21.15.185:0/1116597474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07500af780 0x7f0750151850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 >> 172.21.15.185:0/1116597474 conn(0x7f0750067940 msgr2=0x7f07500b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 shutdown_connections 2020-11-16T22:14:36.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.095+0000 7f0754d46700 1 -- 172.21.15.185:0/1116597474 wait complete. 2020-11-16T22:14:36.096 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:36.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 Processor -- start 2020-11-16T22:14:36.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 -- start start 2020-11-16T22:14:36.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fea040b3830 0x7fea04143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fea04060130 0x7fea04143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea04060970 con 0x7fea040ac820 2020-11-16T22:14:36.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040b4610 con 0x7fea040b3830 2020-11-16T22:14:36.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea09d6d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea04060ba0 con 0x7fea04060130 2020-11-16T22:14:36.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea03fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fea040ac820 0x7fea0413f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50060/0 (socket says 172.21.15.185:50060) 2020-11-16T22:14:36.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea03fff700 1 -- 172.21.15.185:0/4074994129 learned_addr learned my addr 172.21.15.185:0/4074994129 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:36.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea0956c700 1 --2- 172.21.15.185:0/4074994129 >> v2:172.21.15.185:3300/0 conn(0x7fea04060130 0x7fea04143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.239+0000 7fea08d6b700 1 --2- 172.21.15.185:0/4074994129 >> v2:172.21.15.96:3300/0 conn(0x7fea040b3830 0x7fea04143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea037fe700 1 -- 172.21.15.185:0/4074994129 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755190260 0 0) 0x7fea04060970 con 0x7fea040ac820 2020-11-16T22:14:36.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea0956c700 1 -- 172.21.15.185:0/4074994129 >> v2:172.21.15.96:3300/0 conn(0x7fea040b3830 msgr2=0x7fea04143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:36.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea0956c700 1 --2- 172.21.15.185:0/4074994129 >> v2:172.21.15.96:3300/0 conn(0x7fea040b3830 0x7fea04143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea0956c700 1 -- 172.21.15.185:0/4074994129 >> v1:172.21.15.185:6789/0 conn(0x7fea040ac820 legacy=0x7fea0413f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea0956c700 1 -- 172.21.15.185:0/4074994129 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04060e40 con 0x7fea04060130 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea0956c700 1 --2- 172.21.15.185:0/4074994129 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea04060130 0x7fea04143f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fea040b6c70 tx=0x7fe9fc0091d0).ready entity=mon.1 client_cookie=495adb4a327b933b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea037fe700 1 -- 172.21.15.185:0/4074994129 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00ba70 con 0x7fea04060130 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea037fe700 1 -- 172.21.15.185:0/4074994129 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00bbd0 con 0x7fea04060130 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea037fe700 1 -- 172.21.15.185:0/4074994129 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc011400 con 0x7fea04060130 2020-11-16T22:14:36.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- 172.21.15.185:0/4074994129 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea04060130 msgr2=0x7fea04143f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 --2- 172.21.15.185:0/4074994129 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea04060130 0x7fea04143f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fea040b6c70 tx=0x7fe9fc0091d0).stop 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- 172.21.15.185:0/4074994129 shutdown_connections 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 --2- 172.21.15.185:0/4074994129 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea04060130 0x7fea04143f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 --2- 172.21.15.185:0/4074994129 >> v2:172.21.15.96:3300/0 conn(0x7fea040b3830 0x7fea04143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- 172.21.15.185:0/4074994129 >> 172.21.15.185:0/4074994129 conn(0x7fea04063df0 msgr2=0x7fea040641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- 172.21.15.185:0/4074994129 shutdown_connections 2020-11-16T22:14:36.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- 172.21.15.185:0/4074994129 wait complete. 2020-11-16T22:14:36.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 Processor -- start 2020-11-16T22:14:36.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.243+0000 7fea09d6d700 1 -- start start 2020-11-16T22:14:36.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea04060130 0x7fea04143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 0x7fea04155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040b03e0 con 0x7fea04060130 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040b05f0 con 0x7fea040b3830 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea041546b0 con 0x7fea040ac820 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea0956c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 0x7fea04155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50074/0 (socket says 172.21.15.185:50074) 2020-11-16T22:14:36.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea0956c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 0x7fea04155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56632/0 (socket says 172.21.15.185:56632) 2020-11-16T22:14:36.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 -- 172.21.15.185:0/1495139623 learned_addr learned my addr 172.21.15.185:0/1495139623 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:36.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea08d6b700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea04060130 0x7fea04143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 -- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 msgr2=0x7fea04155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 0x7fea04155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 -- 172.21.15.185:0/1495139623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea04060130 msgr2=0x7fea04143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea04060130 0x7fea04143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 -- 172.21.15.185:0/1495139623 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea041442b0 con 0x7fea040ac820 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea03fff700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fe9f800c710 tx=0x7fe9f8007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80072e0 con 0x7fea040ac820 2020-11-16T22:14:36.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8007440 con 0x7fea040ac820 2020-11-16T22:14:36.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea041430d0 con 0x7fea040ac820 2020-11-16T22:14:36.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8005800 con 0x7fea040ac820 2020-11-16T22:14:36.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea04141410 con 0x7fea040ac820 2020-11-16T22:14:36.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.247+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0405d200 con 0x7fea040ac820 2020-11-16T22:14:36.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.251+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe9f8010070 con 0x7fea040ac820 2020-11-16T22:14:36.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.251+0000 7fea00ff9700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 0x7fe9dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.251+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9f806dc50 con 0x7fea040ac820 2020-11-16T22:14:36.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.251+0000 7fea08d6b700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 0x7fe9dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.251+0000 7fea08d6b700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 0x7fe9dc057520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe9f4006fd0 tx=0x7fe9f4006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.259+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe9f8043da0 con 0x7fea040ac820 2020-11-16T22:14:36.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea04061e20 con 0x7fea040ac820 2020-11-16T22:14:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea00ff9700 1 -- 172.21.15.185:0/1495139623 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe9f8007c00 con 0x7fea040ac820 2020-11-16T22:14:36.398 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 msgr2=0x7fe9dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 0x7fe9dc057520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe9f4006fd0 tx=0x7fe9f4006df0).stop 2020-11-16T22:14:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 msgr2=0x7fea04154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fe9f800c710 tx=0x7fe9f8007ac0).stop 2020-11-16T22:14:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 shutdown_connections 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe9dc055060 0x7fe9dc057520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea040b3830 0x7fea04155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea04060130 0x7fea04143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 --2- 172.21.15.185:0/1495139623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea040ac820 0x7fea04154b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 >> 172.21.15.185:0/1495139623 conn(0x7fea04063df0 msgr2=0x7fea040b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 shutdown_connections 2020-11-16T22:14:36.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.411+0000 7fea09d6d700 1 -- 172.21.15.185:0/1495139623 wait complete. 2020-11-16T22:14:36.413 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:36.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 Processor -- start 2020-11-16T22:14:36.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 -- start start 2020-11-16T22:14:36.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff5a40b7440 0x7ff5a40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40ad490 con 0x7ff5a40afc60 2020-11-16T22:14:36.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40ad730 con 0x7ff5a4144860 2020-11-16T22:14:36.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5ac00b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40ad9d0 con 0x7ff5a40b7440 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5aa5a8700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff5a4144860 0x7ff5a40ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50078/0 (socket says 172.21.15.185:50078) 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a9da7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff5a40b7440 0x7ff5a40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5aa5a8700 1 -- 172.21.15.185:0/195541718 learned_addr learned my addr 172.21.15.185:0/195541718 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a8da5700 1 -- 172.21.15.185:0/195541718 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668274501 0 0) 0x7ff5a40ad730 con 0x7ff5a4144860 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a8da5700 1 -- 172.21.15.185:0/195541718 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff58c003240 con 0x7ff5a4144860 2020-11-16T22:14:36.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a9da7700 1 -- 172.21.15.185:0/195541718 >> v1:172.21.15.185:6789/0 conn(0x7ff5a4144860 legacy=0x7ff5a40ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a9da7700 1 -- 172.21.15.185:0/195541718 >> v1:172.21.15.96:6789/0 conn(0x7ff5a40afc60 legacy=0x7ff5a413dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.551+0000 7ff5a9da7700 1 -- 172.21.15.185:0/195541718 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a413e5b0 con 0x7ff5a40b7440 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5a9da7700 1 --2- 172.21.15.185:0/195541718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a40b7440 0x7ff5a40b7820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff5a000b9a0 tx=0x7ff5a0006df0).ready entity=mon.0 client_cookie=fa088a792558063b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5a8da5700 1 -- 172.21.15.185:0/195541718 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00097b0 con 0x7ff5a40b7440 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5a8da5700 1 -- 172.21.15.185:0/195541718 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a0009910 con 0x7ff5a40b7440 2020-11-16T22:14:36.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5a8da5700 1 -- 172.21.15.185:0/195541718 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0009c90 con 0x7ff5a40b7440 2020-11-16T22:14:36.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- 172.21.15.185:0/195541718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a40b7440 msgr2=0x7ff5a40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/195541718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a40b7440 0x7ff5a40b7820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff5a000b9a0 tx=0x7ff5a0006df0).stop 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- 172.21.15.185:0/195541718 shutdown_connections 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/195541718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a40b7440 0x7ff5a40b7820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- 172.21.15.185:0/195541718 >> 172.21.15.185:0/195541718 conn(0x7ff5a4067860 msgr2=0x7ff5a4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- 172.21.15.185:0/195541718 shutdown_connections 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- 172.21.15.185:0/195541718 wait complete. 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 Processor -- start 2020-11-16T22:14:36.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.555+0000 7ff5ac00b700 1 -- start start 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 0x7ff5a4154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 0x7ff5a414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 0x7ff5a40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40b36b0 con 0x7ff5a4144860 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4153970 con 0x7ff5a40b7440 2020-11-16T22:14:36.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4153c10 con 0x7ff5a40afc60 2020-11-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a9da7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 0x7ff5a4154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 0x7ff5a414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a9da7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 0x7ff5a4154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50086/0 (socket says 172.21.15.185:50086) 2020-11-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a9da7700 1 -- 172.21.15.185:0/2836288355 learned_addr learned my addr 172.21.15.185:0/2836288355 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:36.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5aa5a8700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 0x7ff5a40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 -- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 msgr2=0x7ff5a4154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 0x7ff5a4154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 -- 172.21.15.185:0/2836288355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 msgr2=0x7ff5a40b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 0x7ff5a40b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 -- 172.21.15.185:0/2836288355 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4153eb0 con 0x7ff5a40b7440 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5aa5a8700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 0x7ff5a40b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5a95a6700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 0x7ff5a414b260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff59400c740 tx=0x7ff594007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594007020 con 0x7ff5a40b7440 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594007180 con 0x7ff5a40b7440 2020-11-16T22:14:36.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a414b750 con 0x7ff5a40b7440 2020-11-16T22:14:36.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940064f0 con 0x7ff5a40b7440 2020-11-16T22:14:36.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.559+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a40b4210 con 0x7ff5a40b7440 2020-11-16T22:14:36.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a4145040 con 0x7ff5a40b7440 2020-11-16T22:14:36.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff594010030 con 0x7ff5a40b7440 2020-11-16T22:14:36.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff591ffb700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 0x7ff57c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff59406d590 con 0x7ff5a40b7440 2020-11-16T22:14:36.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff5a9da7700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 0x7ff57c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.563+0000 7ff5a9da7700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 0x7ff57c057520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff5a000c010 tx=0x7ff5a0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.567+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff594043790 con 0x7ff5a40b7440 2020-11-16T22:14:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.719+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5a40b4400 con 0x7ff5a40b7440 2020-11-16T22:14:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.719+0000 7ff591ffb700 1 -- 172.21.15.185:0/2836288355 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff594007c20 con 0x7ff5a40b7440 2020-11-16T22:14:36.709 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 msgr2=0x7ff57c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 0x7ff57c057520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff5a000c010 tx=0x7ff5a0015000).stop 2020-11-16T22:14:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 msgr2=0x7ff5a414b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 0x7ff5a414b260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff59400c740 tx=0x7ff594007b00).stop 2020-11-16T22:14:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 shutdown_connections 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff57c055060 0x7ff57c057520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5a4144860 0x7ff5a40b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5a40afc60 0x7ff5a4154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 --2- 172.21.15.185:0/2836288355 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5a40b7440 0x7ff5a414b260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 >> 172.21.15.185:0/2836288355 conn(0x7ff5a4067860 msgr2=0x7ff5a40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 shutdown_connections 2020-11-16T22:14:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.723+0000 7ff5ac00b700 1 -- 172.21.15.185:0/2836288355 wait complete. 2020-11-16T22:14:36.722 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:36.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.855+0000 7fdff299b700 1 Processor -- start 2020-11-16T22:14:36.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.855+0000 7fdff299b700 1 -- start start 2020-11-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 0x7fdfec067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdfec1441f0 0x7fdfec144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdfec144b60 0x7fdfec147590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0b3020 con 0x7fdfec1441f0 2020-11-16T22:14:36.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0b3280 con 0x7fdfec144b60 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b3520 con 0x7fdfec0b7610 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdfec144b60 0x7fdfec147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff1999700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 0x7fdfec067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdfec144b60 0x7fdfec147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56654/0 (socket says 172.21.15.185:56654) 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 -- 172.21.15.185:0/3505995272 learned_addr learned my addr 172.21.15.185:0/3505995272 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:36.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff1198700 1 --2- 172.21.15.185:0/3505995272 >> v2:172.21.15.96:3300/0 conn(0x7fdfec1441f0 0x7fdfec144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 -- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 msgr2=0x7fdfec067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 0x7fdfec067910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 -- 172.21.15.185:0/3505995272 >> v2:172.21.15.96:3300/0 conn(0x7fdfec1441f0 msgr2=0x7fdfec144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 --2- 172.21.15.185:0/3505995272 >> v2:172.21.15.96:3300/0 conn(0x7fdfec1441f0 0x7fdfec144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 -- 172.21.15.185:0/3505995272 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec067f40 con 0x7fdfec144b60 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff1999700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 0x7fdfec067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:36.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff219a700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec144b60 0x7fdfec147590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fdfec065660 tx=0x7fdfe4009290).ready entity=mon.1 client_cookie=c5d64aa4d8271c78 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff0997700 1 -- 172.21.15.185:0/3505995272 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bba0 con 0x7fdfec144b60 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff0997700 1 -- 172.21.15.185:0/3505995272 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400bd00 con 0x7fdfec144b60 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff0997700 1 -- 172.21.15.185:0/3505995272 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011610 con 0x7fdfec144b60 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec144b60 msgr2=0x7fdfec147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec144b60 0x7fdfec147590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fdfec065660 tx=0x7fdfe4009290).stop 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- 172.21.15.185:0/3505995272 shutdown_connections 2020-11-16T22:14:36.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec144b60 0x7fdfec147590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- 172.21.15.185:0/3505995272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec0b7610 0x7fdfec067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 --2- 172.21.15.185:0/3505995272 >> v2:172.21.15.96:3300/0 conn(0x7fdfec1441f0 0x7fdfec144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- 172.21.15.185:0/3505995272 >> 172.21.15.185:0/3505995272 conn(0x7fdfec0637b0 msgr2=0x7fdfec0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:36.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- 172.21.15.185:0/3505995272 shutdown_connections 2020-11-16T22:14:36.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.859+0000 7fdff299b700 1 -- 172.21.15.185:0/3505995272 wait complete. 2020-11-16T22:14:36.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 Processor -- start 2020-11-16T22:14:36.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- start start 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec1441f0 0x7fdfec142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfec144b60 0x7fdfec0ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec153390 con 0x7fdfec144b60 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec142620 con 0x7fdfec0b7610 2020-11-16T22:14:36.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec1535f0 con 0x7fdfec1441f0 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56656/0 (socket says 172.21.15.185:56656) 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 -- 172.21.15.185:0/108147179 learned_addr learned my addr 172.21.15.185:0/108147179 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1198700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec1441f0 0x7fdfec142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 -- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec1441f0 msgr2=0x7fdfec142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec1441f0 0x7fdfec142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 -- 172.21.15.185:0/108147179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfec144b60 msgr2=0x7fdfec0ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:36.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfec144b60 0x7fdfec0ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 -- 172.21.15.185:0/108147179 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec153890 con 0x7fdfec0b7610 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff1999700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fdfe800ef50 tx=0x7fdfe8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800cbf0 con 0x7fdfec0b7610 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800cd50 con 0x7fdfec0b7610 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8012600 con 0x7fdfec0b7610 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec153b30 con 0x7fdfec0b7610 2020-11-16T22:14:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec0ad7d0 con 0x7fdfec0b7610 2020-11-16T22:14:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.863+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdfe8012970 con 0x7fdfec0b7610 2020-11-16T22:14:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.867+0000 7fdfd9ffb700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 0x7fdfc4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:36.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.867+0000 7fdff1198700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 0x7fdfc4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:36.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.867+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdfe806e4d0 con 0x7fdfec0b7610 2020-11-16T22:14:36.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.867+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec0b5650 con 0x7fdfec0b7610 2020-11-16T22:14:36.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.867+0000 7fdff1198700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 0x7fdfc4057520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdfec0ac3e0 tx=0x7fdfdc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:36.871+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdfec0b5650 con 0x7fdfec0b7610 2020-11-16T22:14:37.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.027+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfec0b5650 con 0x7fdfec0b7610 2020-11-16T22:14:37.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdfd9ffb700 1 -- 172.21.15.185:0/108147179 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fdfec0b5650 con 0x7fdfec0b7610 2020-11-16T22:14:37.018 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:37.018 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:37.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 msgr2=0x7fdfc4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 0x7fdfc4057520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdfec0ac3e0 tx=0x7fdfdc008040).stop 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 msgr2=0x7fdfec0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fdfe800ef50 tx=0x7fdfe8006df0).stop 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 shutdown_connections 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdfc4055060 0x7fdfc4057520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdfec0b7610 0x7fdfec0abef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfec144b60 0x7fdfec0ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:14:37.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 --2- 172.21.15.185:0/108147179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdfec1441f0 0x7fdfec142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 >> 172.21.15.185:0/108147179 conn(0x7fdfec0637b0 msgr2=0x7fdfec0b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 shutdown_connections 2020-11-16T22:14:37.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.031+0000 7fdff299b700 1 -- 172.21.15.185:0/108147179 wait complete. 2020-11-16T22:14:37.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:37.032 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:37.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.171+0000 7fd4dc5a7700 1 Processor -- start 2020-11-16T22:14:37.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.171+0000 7fd4dc5a7700 1 -- start start 2020-11-16T22:14:37.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 0x7fd4d40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d4144d50 con 0x7fd4d413dea0 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40ad490 con 0x7fd4d40b7440 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ad730 con 0x7fd4d40afc60 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4da343700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 0x7fd4d40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4da343700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 0x7fd4d40b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56666/0 (socket says 172.21.15.185:56666) 2020-11-16T22:14:37.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50108/0 (socket says 172.21.15.185:50108) 2020-11-16T22:14:37.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4da343700 1 -- 172.21.15.185:0/4173340329 learned_addr learned my addr 172.21.15.185:0/4173340329 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:37.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 -- 172.21.15.185:0/4173340329 >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 msgr2=0x7fd4d40b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:37.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 --2- 172.21.15.185:0/4173340329 >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 0x7fd4d40b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 -- 172.21.15.185:0/4173340329 >> v1:172.21.15.96:6789/0 conn(0x7fd4d413dea0 legacy=0x7fd4d40ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 -- 172.21.15.185:0/4173340329 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40ad9d0 con 0x7fd4d40afc60 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9b42700 1 --2- 172.21.15.185:0/4173340329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd4c400b9a0 tx=0x7fd4c4006df0).ready entity=mon.2 client_cookie=d21a5ccfb920ac21 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9341700 1 -- 172.21.15.185:0/4173340329 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40097b0 con 0x7fd4d40afc60 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9341700 1 -- 172.21.15.185:0/4173340329 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4009910 con 0x7fd4d40afc60 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4d9341700 1 -- 172.21.15.185:0/4173340329 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009c90 con 0x7fd4d40afc60 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/4173340329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 msgr2=0x7fd4d4144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/4173340329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd4c400b9a0 tx=0x7fd4c4006df0).stop 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/4173340329 shutdown_connections 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/4173340329 >> v2:172.21.15.185:3300/0 conn(0x7fd4d40b7440 0x7fd4d40b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/4173340329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d40afc60 0x7fd4d4144860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/4173340329 >> 172.21.15.185:0/4173340329 conn(0x7fd4d4067860 msgr2=0x7fd4d4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/4173340329 shutdown_connections 2020-11-16T22:14:37.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/4173340329 wait complete. 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 Processor -- start 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.175+0000 7fd4dc5a7700 1 -- start start 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4d40afc60 0x7fd4d4140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4d40b7440 0x7fd4d413f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4140900 con 0x7fd4d40afc60 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4147b20 con 0x7fd4d40b7440 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4147dc0 con 0x7fd4d413dea0 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50116/0 (socket says 172.21.15.185:50116) 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 -- 172.21.15.185:0/3461555590 learned_addr learned my addr 172.21.15.185:0/3461555590 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4d9b42700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4d40b7440 0x7fd4d413f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4da343700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4d40afc60 0x7fd4d4140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 -- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4d40b7440 msgr2=0x7fd4d413f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4d40b7440 0x7fd4d413f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 -- 172.21.15.185:0/3461555590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4d40afc60 msgr2=0x7fd4d4140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4d40afc60 0x7fd4d4140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 -- 172.21.15.185:0/3461555590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d413f9b0 con 0x7fd4d413dea0 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dab44700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd4cc00ca60 tx=0x7fd4cc007eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00a7f0 con 0x7fd4d413dea0 2020-11-16T22:14:37.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00a950 con 0x7fd4d413dea0 2020-11-16T22:14:37.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d4143e10 con 0x7fd4d413dea0 2020-11-16T22:14:37.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00acd0 con 0x7fd4d413dea0 2020-11-16T22:14:37.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.179+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d413c6c0 con 0x7fd4d413dea0 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d40adbb0 con 0x7fd4d413dea0 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd4cc019030 con 0x7fd4d413dea0 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4c27fc700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 0x7fd4ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd4cc06d640 con 0x7fd4d413dea0 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4da343700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 0x7fd4ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.183+0000 7fd4da343700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 0x7fd4ac057520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd4d0006a10 tx=0x7fd4d0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.187+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd4d40adbb0 con 0x7fd4d413dea0 2020-11-16T22:14:37.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.339+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd4d40adbb0 con 0x7fd4d413dea0 2020-11-16T22:14:37.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.339+0000 7fd4c27fc700 1 -- 172.21.15.185:0/3461555590 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd4d40adbb0 con 0x7fd4d413dea0 2020-11-16T22:14:37.328 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:37.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 msgr2=0x7fd4ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 0x7fd4ac057520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd4d0006a10 tx=0x7fd4d0008040).stop 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 msgr2=0x7fd4d4143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd4cc00ca60 tx=0x7fd4cc007eb0).stop 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 shutdown_connections 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4ac055060 0x7fd4ac057520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4d413dea0 0x7fd4d4143920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4d40afc60 0x7fd4d4140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 --2- 172.21.15.185:0/3461555590 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4d40b7440 0x7fd4d413f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 >> 172.21.15.185:0/3461555590 conn(0x7fd4d4067860 msgr2=0x7fd4d413d720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 shutdown_connections 2020-11-16T22:14:37.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.343+0000 7fd4dc5a7700 1 -- 172.21.15.185:0/3461555590 wait complete. 2020-11-16T22:14:37.346 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:37.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 Processor -- start 2020-11-16T22:14:37.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 -- start start 2020-11-16T22:14:37.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1a080ac600 0x7f1a08065b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08067130 con 0x7f1a08068180 2020-11-16T22:14:37.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08065300 con 0x7f1a080657a0 2020-11-16T22:14:37.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0f329700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a08068640 con 0x7f1a080ac600 2020-11-16T22:14:37.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0e327700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f1a080657a0 0x7f1a080abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50116/0 (socket says 172.21.15.185:50116) 2020-11-16T22:14:37.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0e327700 1 -- 172.21.15.185:0/3360586578 learned_addr learned my addr 172.21.15.185:0/3360586578 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:37.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0eb28700 1 --2- 172.21.15.185:0/3360586578 >> v2:172.21.15.96:3300/0 conn(0x7f1a080ac600 0x7f1a08065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245525767 0 0) 0x7f1a08065300 con 0x7f1a080657a0 2020-11-16T22:14:37.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003240 con 0x7f1a080657a0 2020-11-16T22:14:37.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3494362026 0 0) 0x7f19f0003240 con 0x7f1a080657a0 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0eb28700 1 -- 172.21.15.185:0/3360586578 >> v1:172.21.15.185:6789/0 conn(0x7f1a080657a0 legacy=0x7f1a080abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0eb28700 1 -- 172.21.15.185:0/3360586578 >> v1:172.21.15.96:6789/0 conn(0x7f1a08068180 legacy=0x7f1a0813f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0eb28700 1 -- 172.21.15.185:0/3360586578 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08067360 con 0x7f1a080ac600 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0eb28700 1 --2- 172.21.15.185:0/3360586578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a08065b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f1a00009c10 tx=0x7f1a000091d0).ready entity=mon.0 client_cookie=5b0e124a857098a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040014e0 con 0x7f1a080ac600 2020-11-16T22:14:37.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000b710 con 0x7f1a080ac600 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.491+0000 7f1a0d325700 1 -- 172.21.15.185:0/3360586578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000ba90 con 0x7f1a080ac600 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- 172.21.15.185:0/3360586578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 msgr2=0x7f1a08065b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 --2- 172.21.15.185:0/3360586578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a08065b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f1a00009c10 tx=0x7f1a000091d0).stop 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- 172.21.15.185:0/3360586578 shutdown_connections 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 --2- 172.21.15.185:0/3360586578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a08065b80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- 172.21.15.185:0/3360586578 >> 172.21.15.185:0/3360586578 conn(0x7f1a08060130 msgr2=0x7f1a08060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- 172.21.15.185:0/3360586578 shutdown_connections 2020-11-16T22:14:37.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- 172.21.15.185:0/3360586578 wait complete. 2020-11-16T22:14:37.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 Processor -- start 2020-11-16T22:14:37.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 -- start start 2020-11-16T22:14:37.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.495+0000 7f1a0f329700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 0x7f1a080ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a0813b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080aeda0 con 0x7f1a080ac600 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08141340 con 0x7f1a080657a0 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080af1d0 con 0x7f1a08068180 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0e327700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 0x7f1a080ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50128/0 (socket says 172.21.15.185:50128) 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 -- 172.21.15.185:0/1647502630 learned_addr learned my addr 172.21.15.185:0/1647502630 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0eb28700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a0813b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 -- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 msgr2=0x7f1a080ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 0x7f1a080ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 -- 172.21.15.185:0/1647502630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 msgr2=0x7f1a0813b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a0813b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 -- 172.21.15.185:0/1647502630 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080b0160 con 0x7f1a08068180 2020-11-16T22:14:37.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0eb28700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a0813b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:37.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0e327700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 0x7f1a080ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:37.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0db26700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f19f8001e70 tx=0x7f19f8007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800e560 con 0x7f1a08068180 2020-11-16T22:14:37.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800e6c0 con 0x7f1a08068180 2020-11-16T22:14:37.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a08140ac0 con 0x7f1a08068180 2020-11-16T22:14:37.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ea80 con 0x7f1a08068180 2020-11-16T22:14:37.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a080b06b0 con 0x7f1a08068180 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0805d200 con 0x7f1a08068180 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f19f8019030 con 0x7f1a08068180 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.499+0000 7f19f67fc700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 0x7f19e4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.503+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19f806d450 con 0x7f1a08068180 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.503+0000 7f1a0e327700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 0x7f19e4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.503+0000 7f1a0e327700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 0x7f19e4057550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f1a04007260 tx=0x7f1a04000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.507+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f19f8043650 con 0x7f1a08068180 2020-11-16T22:14:37.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a080af310 con 0x7f1a08068180 2020-11-16T22:14:37.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f19f67fc700 1 -- 172.21.15.185:0/1647502630 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19f8046c70 con 0x7f1a08068180 2020-11-16T22:14:37.645 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:37.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 msgr2=0x7f19e4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 0x7f19e4057550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f1a04007260 tx=0x7f1a04000b80).stop 2020-11-16T22:14:37.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 msgr2=0x7f1a081405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f19f8001e70 tx=0x7f19f8007b50).stop 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 shutdown_connections 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f19e4055090 0x7f19e4057550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a080657a0 0x7f1a080ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a080ac600 0x7f1a0813b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 --2- 172.21.15.185:0/1647502630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a08068180 0x7f1a081405d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 >> 172.21.15.185:0/1647502630 conn(0x7f1a08060130 msgr2=0x7f1a08141780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 shutdown_connections 2020-11-16T22:14:37.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.659+0000 7f1a0f329700 1 -- 172.21.15.185:0/1647502630 wait complete. 2020-11-16T22:14:37.666 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:37.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 Processor -- start 2020-11-16T22:14:37.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 -- start start 2020-11-16T22:14:37.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff1100b38a0 0x7ff1100b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 0x7ff110143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 0x7ff1101438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff110061050 con 0x7ff1101481b0 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100abf90 con 0x7ff1100b38a0 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11005bb20 con 0x7ff110060130 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10e7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 0x7ff110143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff1100b38a0 0x7ff1100b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10e7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 0x7ff110143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50136/0 (socket says 172.21.15.185:50136) 2020-11-16T22:14:37.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10e7fc700 1 -- 172.21.15.185:0/3676958914 learned_addr learned my addr 172.21.15.185:0/3676958914 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:37.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10f7fe700 1 --2- 172.21.15.185:0/3676958914 >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 0x7ff1101438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 -- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 msgr2=0x7ff110143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:37.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 --2- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 0x7ff110143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 -- 172.21.15.185:0/3676958914 >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 msgr2=0x7ff1101438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:37.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 --2- 172.21.15.185:0/3676958914 >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 0x7ff1101438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10effd700 1 -- 172.21.15.185:0/3676958914 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11005bd80 con 0x7ff1100b38a0 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.807+0000 7ff10f7fe700 1 --2- 172.21.15.185:0/3676958914 >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 0x7ff1101438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10effd700 1 --2- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff1100b2f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff110061d80 tx=0x7ff100006df0).ready entity=mon.1 client_cookie=9e2129aeaded0755 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10dffb700 1 -- 172.21.15.185:0/3676958914 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000097b0 con 0x7ff1100b38a0 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10dffb700 1 -- 172.21.15.185:0/3676958914 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100009910 con 0x7ff1100b38a0 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10dffb700 1 -- 172.21.15.185:0/3676958914 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100009c90 con 0x7ff1100b38a0 2020-11-16T22:14:37.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 msgr2=0x7ff1100b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 --2- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff1100b2f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff110061d80 tx=0x7ff100006df0).stop 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- 172.21.15.185:0/3676958914 shutdown_connections 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 --2- 172.21.15.185:0/3676958914 >> v2:172.21.15.96:3300/0 conn(0x7ff1101481b0 0x7ff1101438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 --2- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff1100b2f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 --2- 172.21.15.185:0/3676958914 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff110060130 0x7ff110143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- 172.21.15.185:0/3676958914 >> 172.21.15.185:0/3676958914 conn(0x7ff110063810 msgr2=0x7ff110063df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- 172.21.15.185:0/3676958914 shutdown_connections 2020-11-16T22:14:37.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- 172.21.15.185:0/3676958914 wait complete. 2020-11-16T22:14:37.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 Processor -- start 2020-11-16T22:14:37.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.811+0000 7ff10ffff700 1 -- start start 2020-11-16T22:14:37.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff110149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11014a4d0 0x7ff1100ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101411c0 con 0x7ff11014a4d0 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11014b0f0 con 0x7ff1100b38a0 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101413f0 con 0x7ff1101481b0 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50144/0 (socket says 172.21.15.185:50144) 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 -- 172.21.15.185:0/1238646326 learned_addr learned my addr 172.21.15.185:0/1238646326 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff110149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10f7fe700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11014a4d0 0x7ff1100ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 -- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 msgr2=0x7ff110149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff110149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 -- 172.21.15.185:0/1238646326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11014a4d0 msgr2=0x7ff1100ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11014a4d0 0x7ff1100ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 -- 172.21.15.185:0/1238646326 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110142990 con 0x7ff1101481b0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10e7fc700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff0f800ebd0 tx=0x7ff0f8006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800c820 con 0x7ff1101481b0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800c980 con 0x7ff1101481b0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff110142680 con 0x7ff1101481b0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800cd20 con 0x7ff1101481b0 2020-11-16T22:14:37.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff110140de0 con 0x7ff1101481b0 2020-11-16T22:14:37.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11005d200 con 0x7ff1101481b0 2020-11-16T22:14:37.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff0f8022070 con 0x7ff1101481b0 2020-11-16T22:14:37.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 0x7ff0ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:37.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff0f806e170 con 0x7ff1101481b0 2020-11-16T22:14:37.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10effd700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 0x7ff0ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:37.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.815+0000 7ff10effd700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 0x7ff0ec057520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff100006e30 tx=0x7ff100015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:37.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.823+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff0f80443d0 con 0x7ff1101481b0 2020-11-16T22:14:37.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1100ac230 con 0x7ff1101481b0 2020-11-16T22:14:37.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff106ffd700 1 -- 172.21.15.185:0/1238646326 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff0f80160a0 con 0x7ff1101481b0 2020-11-16T22:14:37.961 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:37.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 msgr2=0x7ff0ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 0x7ff0ec057520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff100006e30 tx=0x7ff100015000).stop 2020-11-16T22:14:37.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 msgr2=0x7ff1100ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff0f800ebd0 tx=0x7ff0f8006eb0).stop 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 shutdown_connections 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff0ec055060 0x7ff0ec057520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff1100b38a0 0x7ff110149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff11014a4d0 0x7ff1100ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 --2- 172.21.15.185:0/1238646326 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff1101481b0 0x7ff1100ae190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:37.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 >> 172.21.15.185:0/1238646326 conn(0x7ff110060130 msgr2=0x7ff1100b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:37.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 shutdown_connections 2020-11-16T22:14:37.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:37.975+0000 7ff10ffff700 1 -- 172.21.15.185:0/1238646326 wait complete. 2020-11-16T22:14:37.975 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:38.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 Processor -- start 2020-11-16T22:14:38.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 -- start start 2020-11-16T22:14:38.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa234061240 0x7fa2341475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340b3080 con 0x7fa2340af130 2020-11-16T22:14:38.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340b37f0 con 0x7fa23413df60 2020-11-16T22:14:38.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa238cfa700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa234064500 con 0x7fa234061240 2020-11-16T22:14:38.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa2337fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa23413df60 0x7fa234063df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50144/0 (socket says 172.21.15.185:50144) 2020-11-16T22:14:38.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa2337fe700 1 -- 172.21.15.185:0/1498465038 learned_addr learned my addr 172.21.15.185:0/1498465038 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:38.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa232ffd700 1 --2- 172.21.15.185:0/1498465038 >> v2:172.21.15.96:3300/0 conn(0x7fa234061240 0x7fa2341475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775880777 0 0) 0x7fa2340b37f0 con 0x7fa23413df60 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa214003240 con 0x7fa23413df60 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa232ffd700 1 -- 172.21.15.185:0/1498465038 >> v1:172.21.15.185:6789/0 conn(0x7fa23413df60 legacy=0x7fa234063df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2813751573 0 0) 0x7fa214003240 con 0x7fa23413df60 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa232ffd700 1 -- 172.21.15.185:0/1498465038 >> v1:172.21.15.96:6789/0 conn(0x7fa2340af130 legacy=0x7fa2340b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa232ffd700 1 -- 172.21.15.185:0/1498465038 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234148540 con 0x7fa234061240 2020-11-16T22:14:38.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.119+0000 7fa232ffd700 1 --2- 172.21.15.185:0/1498465038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa2341475d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa21c009c90 tx=0x7fa21c0091d0).ready entity=mon.0 client_cookie=8d9170cb58c51a15 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240014e0 con 0x7fa234061240 2020-11-16T22:14:38.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c00b760 con 0x7fa234061240 2020-11-16T22:14:38.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa2327fc700 1 -- 172.21.15.185:0/1498465038 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00bae0 con 0x7fa234061240 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- 172.21.15.185:0/1498465038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 msgr2=0x7fa2341475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 --2- 172.21.15.185:0/1498465038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa2341475d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa21c009c90 tx=0x7fa21c0091d0).stop 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- 172.21.15.185:0/1498465038 shutdown_connections 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 --2- 172.21.15.185:0/1498465038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa2341475d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- 172.21.15.185:0/1498465038 >> 172.21.15.185:0/1498465038 conn(0x7fa234067940 msgr2=0x7fa234067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:38.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- 172.21.15.185:0/1498465038 shutdown_connections 2020-11-16T22:14:38.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- 172.21.15.185:0/1498465038 wait complete. 2020-11-16T22:14:38.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 Processor -- start 2020-11-16T22:14:38.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.123+0000 7fa238cfa700 1 -- start start 2020-11-16T22:14:38.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa23413e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa23413df60 0x7fa2341498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234156910 con 0x7fa234061240 2020-11-16T22:14:38.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23413b990 con 0x7fa2340af130 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa234140c40 con 0x7fa23413df60 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56712/0 (socket says 172.21.15.185:56712) 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 -- 172.21.15.185:0/601144008 learned_addr learned my addr 172.21.15.185:0/601144008 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa233fff700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa23413df60 0x7fa2341498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa2337fe700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa23413e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 -- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa23413df60 msgr2=0x7fa2341498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa23413df60 0x7fa2341498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 -- 172.21.15.185:0/601144008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 msgr2=0x7fa23413e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa23413e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 -- 172.21.15.185:0/601144008 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234140ee0 con 0x7fa2340af130 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa232ffd700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa21c0092c0 tx=0x7fa21c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c011b40 con 0x7fa2340af130 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c011ca0 con 0x7fa2340af130 2020-11-16T22:14:38.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234156b00 con 0x7fa2340af130 2020-11-16T22:14:38.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c01d650 con 0x7fa2340af130 2020-11-16T22:14:38.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23413ec00 con 0x7fa2340af130 2020-11-16T22:14:38.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23405d200 con 0x7fa2340af130 2020-11-16T22:14:38.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa21c017070 con 0x7fa2340af130 2020-11-16T22:14:38.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 0x7fa210057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa21c06ef90 con 0x7fa2340af130 2020-11-16T22:14:38.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.127+0000 7fa2337fe700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 0x7fa210057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.131+0000 7fa2337fe700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 0x7fa210057520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa224001e90 tx=0x7fa224001e20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.139+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa21c0451f0 con 0x7fa2340af130 2020-11-16T22:14:38.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.291+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa23405c720 con 0x7fa2340af130 2020-11-16T22:14:38.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.291+0000 7fa22b7fe700 1 -- 172.21.15.185:0/601144008 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa21c048810 con 0x7fa2340af130 2020-11-16T22:14:38.279 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:38.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 msgr2=0x7fa210057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 0x7fa210057520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa224001e90 tx=0x7fa224001e20).stop 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 msgr2=0x7fa234157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa21c0092c0 tx=0x7fa21c0091d0).stop 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 shutdown_connections 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa210055060 0x7fa210057520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa23413df60 0x7fa2341498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa234061240 0x7fa23413e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 --2- 172.21.15.185:0/601144008 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa2340af130 0x7fa234157240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 >> 172.21.15.185:0/601144008 conn(0x7fa234067940 msgr2=0x7fa234067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:38.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 shutdown_connections 2020-11-16T22:14:38.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.295+0000 7fa238cfa700 1 -- 172.21.15.185:0/601144008 wait complete. 2020-11-16T22:14:38.297 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:38.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 Processor -- start 2020-11-16T22:14:38.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 -- start start 2020-11-16T22:14:38.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f95a80b8830 0x7f95a8061450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 0x7f95a8143140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a80b4d60 con 0x7f95a8062c10 2020-11-16T22:14:38.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80630b0 con 0x7f95a80b8830 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80b41a0 con 0x7f95a80b48e0 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a67fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f95a8062c10 0x7f95a81441f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50160/0 (socket says 172.21.15.185:50160) 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a67fc700 1 -- 172.21.15.185:0/3697709058 learned_addr learned my addr 172.21.15.185:0/3697709058 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a77fe700 1 --2- 172.21.15.185:0/3697709058 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 0x7f95a8143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a6ffd700 1 --2- 172.21.15.185:0/3697709058 >> v2:172.21.15.96:3300/0 conn(0x7f95a80b8830 0x7f95a8061450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a5ffb700 1 -- 172.21.15.185:0/3697709058 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374790627 0 0) 0x7f95a80b4d60 con 0x7f95a8062c10 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a5ffb700 1 -- 172.21.15.185:0/3697709058 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9588003240 con 0x7f95a8062c10 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a77fe700 1 -- 172.21.15.185:0/3697709058 >> v2:172.21.15.96:3300/0 conn(0x7f95a80b8830 msgr2=0x7f95a8061450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a77fe700 1 --2- 172.21.15.185:0/3697709058 >> v2:172.21.15.96:3300/0 conn(0x7f95a80b8830 0x7f95a8061450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a77fe700 1 -- 172.21.15.185:0/3697709058 >> v1:172.21.15.185:6789/0 conn(0x7f95a8062c10 legacy=0x7f95a81441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.431+0000 7f95a77fe700 1 -- 172.21.15.185:0/3697709058 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80b4440 con 0x7f95a80b48e0 2020-11-16T22:14:38.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a77fe700 1 --2- 172.21.15.185:0/3697709058 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 0x7f95a8143140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f95a0009c90 tx=0x7f95a00091d0).ready entity=mon.2 client_cookie=336dcf578ed86dd8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a5ffb700 1 -- 172.21.15.185:0/3697709058 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000ba70 con 0x7f95a80b48e0 2020-11-16T22:14:38.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a5ffb700 1 -- 172.21.15.185:0/3697709058 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a000bbd0 con 0x7f95a80b48e0 2020-11-16T22:14:38.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a5ffb700 1 -- 172.21.15.185:0/3697709058 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011400 con 0x7f95a80b48e0 2020-11-16T22:14:38.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- 172.21.15.185:0/3697709058 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 msgr2=0x7f95a8143140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- 172.21.15.185:0/3697709058 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 0x7f95a8143140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f95a0009c90 tx=0x7f95a00091d0).stop 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- 172.21.15.185:0/3697709058 shutdown_connections 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- 172.21.15.185:0/3697709058 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b48e0 0x7f95a8143140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- 172.21.15.185:0/3697709058 >> v2:172.21.15.96:3300/0 conn(0x7f95a80b8830 0x7f95a8061450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- 172.21.15.185:0/3697709058 >> 172.21.15.185:0/3697709058 conn(0x7f95a8060130 msgr2=0x7f95a8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- 172.21.15.185:0/3697709058 shutdown_connections 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- 172.21.15.185:0/3697709058 wait complete. 2020-11-16T22:14:38.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 Processor -- start 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- start start 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a8062c10 0x7f95a8146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 0x7f95a80b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 0x7f95a814ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8146010 con 0x7f95a8062c10 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80af720 con 0x7f95a80b48e0 2020-11-16T22:14:38.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80af9c0 con 0x7f95a80b8830 2020-11-16T22:14:38.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 0x7f95a80b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 0x7f95a814ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 0x7f95a80b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56728/0 (socket says 172.21.15.185:56728) 2020-11-16T22:14:38.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a67fc700 1 -- 172.21.15.185:0/1273685548 learned_addr learned my addr 172.21.15.185:0/1273685548 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:38.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.435+0000 7f95a6ffd700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a8062c10 0x7f95a8146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 -- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 msgr2=0x7f95a80b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 0x7f95a80b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 -- 172.21.15.185:0/1273685548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a8062c10 msgr2=0x7f95a8146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a8062c10 0x7f95a8146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 -- 172.21.15.185:0/1273685548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8149ed0 con 0x7f95a80b8830 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a77fe700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 0x7f95a814ad10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f95a0013000 tx=0x7f95a0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0006ea0 con 0x7f95a80b8830 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a001c6c0 con 0x7f95a80b8830 2020-11-16T22:14:38.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a001ca80 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a80b7d30 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a814b3a0 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a80afb80 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f95a0024030 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 0x7f9584057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f95a006f550 con 0x7f95a80b8830 2020-11-16T22:14:38.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a6ffd700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 0x7f9584057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:38.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.439+0000 7f95a6ffd700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 0x7f9584057550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f959800d5f0 tx=0x7f9598008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:38.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.447+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f95a0045860 con 0x7f95a80b8830 2020-11-16T22:14:38.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.599+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95a805d200 con 0x7f95a80b8830 2020-11-16T22:14:38.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f959effd700 1 -- 172.21.15.185:0/1273685548 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f95a00301d0 con 0x7f95a80b8830 2020-11-16T22:14:38.589 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:38.589 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 msgr2=0x7f9584057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 0x7f9584057550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f959800d5f0 tx=0x7f9598008040).stop 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 msgr2=0x7f95a814ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 0x7f95a814ad10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f95a0013000 tx=0x7f95a0016000).stop 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 shutdown_connections 2020-11-16T22:14:38.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9584055090 0x7f9584057550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f95a80b8830 0x7f95a814ad10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95a8062c10 0x7f95a8146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 --2- 172.21.15.185:0/1273685548 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f95a80b48e0 0x7f95a80b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 >> 172.21.15.185:0/1273685548 conn(0x7f95a8060130 msgr2=0x7f95a8060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 shutdown_connections 2020-11-16T22:14:38.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:38.603+0000 7f95a7fff700 1 -- 172.21.15.185:0/1273685548 wait complete. 2020-11-16T22:14:38.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:42.794 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:42.797 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:42.803 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:42.806 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:14:42.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.947+0000 7f1bb072f700 1 Processor -- start 2020-11-16T22:14:42.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.947+0000 7f1bb072f700 1 -- start start 2020-11-16T22:14:42.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 0x7f1ba80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1ba8144860 0x7f1ba80ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba80ad490 con 0x7f1ba80b7440 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80ad730 con 0x7f1ba80afc60 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80ad9d0 con 0x7f1ba8144860 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1ba8144860 0x7f1ba80ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1ba8144860 0x7f1ba80ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56738/0 (socket says 172.21.15.185:56738) 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 -- 172.21.15.185:0/2569722251 learned_addr learned my addr 172.21.15.185:0/2569722251 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1badcca700 1 --2- 172.21.15.185:0/2569722251 >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 0x7f1ba80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 -- 172.21.15.185:0/2569722251 >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 msgr2=0x7f1ba80b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 --2- 172.21.15.185:0/2569722251 >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 0x7f1ba80b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 -- 172.21.15.185:0/2569722251 >> v1:172.21.15.185:6789/0 conn(0x7f1ba80b7440 legacy=0x7f1ba813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 -- 172.21.15.185:0/2569722251 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba813e5b0 con 0x7f1ba8144860 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1badcca700 1 --2- 172.21.15.185:0/2569722251 >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 0x7f1ba80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1baeccc700 1 --2- 172.21.15.185:0/2569722251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba8144860 0x7f1ba80ade70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f1ba80b5290 tx=0x7f1ba0009290).ready entity=mon.1 client_cookie=52d4b812cbb1d40 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bad4c9700 1 -- 172.21.15.185:0/2569722251 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000bbc0 con 0x7f1ba8144860 2020-11-16T22:14:42.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bad4c9700 1 -- 172.21.15.185:0/2569722251 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba000bd20 con 0x7f1ba8144860 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bad4c9700 1 -- 172.21.15.185:0/2569722251 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0011610 con 0x7f1ba8144860 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- 172.21.15.185:0/2569722251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba8144860 msgr2=0x7f1ba80ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2569722251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba8144860 0x7f1ba80ade70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f1ba80b5290 tx=0x7f1ba0009290).stop 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- 172.21.15.185:0/2569722251 shutdown_connections 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2569722251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba8144860 0x7f1ba80ade70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2569722251 >> v2:172.21.15.96:3300/0 conn(0x7f1ba80afc60 0x7f1ba80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:42.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- 172.21.15.185:0/2569722251 >> 172.21.15.185:0/2569722251 conn(0x7f1ba8067860 msgr2=0x7f1ba8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:42.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- 172.21.15.185:0/2569722251 shutdown_connections 2020-11-16T22:14:42.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.951+0000 7f1bb072f700 1 -- 172.21.15.185:0/2569722251 wait complete. 2020-11-16T22:14:42.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 Processor -- start 2020-11-16T22:14:42.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- start start 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba80afc60 0x7f1ba814a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 0x7f1ba8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba8156a20 con 0x7f1ba80afc60 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80b0960 con 0x7f1ba80b7440 2020-11-16T22:14:42.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80b2080 con 0x7f1ba8144860 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1badcca700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 0x7f1ba8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50186/0 (socket says 172.21.15.185:50186) 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1badcca700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 0x7f1ba8156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56744/0 (socket says 172.21.15.185:56744) 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 -- 172.21.15.185:0/2377140926 learned_addr learned my addr 172.21.15.185:0/2377140926 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bae4cb700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba80afc60 0x7f1ba814a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 -- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 msgr2=0x7f1ba8156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 0x7f1ba8156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 -- 172.21.15.185:0/2377140926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba80afc60 msgr2=0x7f1ba814a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba80afc60 0x7f1ba814a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 -- 172.21.15.185:0/2377140926 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba80b2320 con 0x7f1ba8144860 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1baeccc700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f1ba00092d0 tx=0x7f1ba0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000bbc0 con 0x7f1ba8144860 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba000be90 con 0x7f1ba8144860 2020-11-16T22:14:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba80b25c0 con 0x7f1ba8144860 2020-11-16T22:14:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010cf0 con 0x7f1ba8144860 2020-11-16T22:14:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.955+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba8157870 con 0x7f1ba8144860 2020-11-16T22:14:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1ba0023030 con 0x7f1ba8144860 2020-11-16T22:14:42.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba80b3840 con 0x7f1ba8144860 2020-11-16T22:14:42.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1b967fc700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 0x7f1b80057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:42.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1bae4cb700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 0x7f1b80057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:42.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ba006f450 con 0x7f1ba8144860 2020-11-16T22:14:42.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.959+0000 7f1bae4cb700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 0x7f1b80057550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f1ba4003300 tx=0x7f1ba4007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:42.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:42.963+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1ba00456d0 con 0x7f1ba8144860 2020-11-16T22:14:43.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.119+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1ba805d4e0 con 0x7f1ba8144860 2020-11-16T22:14:43.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.119+0000 7f1b967fc700 1 -- 172.21.15.185:0/2377140926 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1ba0021020 con 0x7f1ba8144860 2020-11-16T22:14:43.107 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:43.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 msgr2=0x7f1b80057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 0x7f1b80057550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f1ba4003300 tx=0x7f1ba4007b90).stop 2020-11-16T22:14:43.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 msgr2=0x7f1ba8157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f1ba00092d0 tx=0x7f1ba0016000).stop 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 shutdown_connections 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b80055090 0x7f1b80057550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1ba8144860 0x7f1ba8157300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba80afc60 0x7f1ba814a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 --2- 172.21.15.185:0/2377140926 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1ba80b7440 0x7f1ba8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 >> 172.21.15.185:0/2377140926 conn(0x7f1ba8067860 msgr2=0x7f1ba8147b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 shutdown_connections 2020-11-16T22:14:43.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.123+0000 7f1bb072f700 1 -- 172.21.15.185:0/2377140926 wait complete. 2020-11-16T22:14:43.121 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:43.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.259+0000 7f1a7e854700 1 Processor -- start 2020-11-16T22:14:43.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.259+0000 7f1a7e854700 1 -- start start 2020-11-16T22:14:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e854700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a780b7690 0x7f1a78067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e854700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1a780b1f20 0x7f1a780b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e854700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a780b3f50 con 0x7f1a78141690 2020-11-16T22:14:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e854700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a780b41f0 con 0x7f1a780b1f20 2020-11-16T22:14:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e854700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a780b4490 con 0x7f1a780b7690 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a780b7690 0x7f1a78067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a780b7690 0x7f1a78067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56748/0 (socket says 172.21.15.185:56748) 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 -- 172.21.15.185:0/1317730143 learned_addr learned my addr 172.21.15.185:0/1317730143 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7e053700 1 --2- 172.21.15.185:0/1317730143 >> v2:172.21.15.96:3300/0 conn(0x7f1a780b1f20 0x7f1a780b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 -- 172.21.15.185:0/1317730143 >> v2:172.21.15.96:3300/0 conn(0x7f1a780b1f20 msgr2=0x7f1a780b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 --2- 172.21.15.185:0/1317730143 >> v2:172.21.15.96:3300/0 conn(0x7f1a780b1f20 0x7f1a780b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 -- 172.21.15.185:0/1317730143 >> v1:172.21.15.96:6789/0 conn(0x7f1a78141690 legacy=0x7f1a78144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 -- 172.21.15.185:0/1317730143 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780b4e20 con 0x7f1a780b7690 2020-11-16T22:14:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7d852700 1 --2- 172.21.15.185:0/1317730143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b7690 0x7f1a78067420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1a78065270 tx=0x7f1a74009290).ready entity=mon.1 client_cookie=5b3997975739bc77 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7c850700 1 -- 172.21.15.185:0/1317730143 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400bbc0 con 0x7f1a780b7690 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7c850700 1 -- 172.21.15.185:0/1317730143 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7400bd20 con 0x7f1a780b7690 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.263+0000 7f1a7c850700 1 -- 172.21.15.185:0/1317730143 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74011610 con 0x7f1a780b7690 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- 172.21.15.185:0/1317730143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b7690 msgr2=0x7f1a78067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 --2- 172.21.15.185:0/1317730143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b7690 0x7f1a78067420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1a78065270 tx=0x7f1a74009290).stop 2020-11-16T22:14:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- 172.21.15.185:0/1317730143 shutdown_connections 2020-11-16T22:14:43.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 --2- 172.21.15.185:0/1317730143 >> v2:172.21.15.96:3300/0 conn(0x7f1a780b1f20 0x7f1a780b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 --2- 172.21.15.185:0/1317730143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b7690 0x7f1a78067420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- 172.21.15.185:0/1317730143 >> 172.21.15.185:0/1317730143 conn(0x7f1a78064b90 msgr2=0x7f1a78063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- 172.21.15.185:0/1317730143 shutdown_connections 2020-11-16T22:14:43.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- 172.21.15.185:0/1317730143 wait complete. 2020-11-16T22:14:43.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 Processor -- start 2020-11-16T22:14:43.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.267+0000 7f1a7e854700 1 -- start start 2020-11-16T22:14:43.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 0x7f1a7813d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a78141690 0x7f1a781500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a78145100 con 0x7f1a780b7690 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780afaa0 con 0x7f1a780b1f20 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a781537a0 con 0x7f1a78141690 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56754/0 (socket says 172.21.15.185:56754) 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 -- 172.21.15.185:0/4152250636 learned_addr learned my addr 172.21.15.185:0/4152250636 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e053700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a78141690 0x7f1a781500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d051700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 0x7f1a7813d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 -- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a78141690 msgr2=0x7f1a781500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a78141690 0x7f1a781500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 -- 172.21.15.185:0/4152250636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 msgr2=0x7f1a7813d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 0x7f1a7813d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 -- 172.21.15.185:0/4152250636 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a78153a40 con 0x7f1a780b1f20 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d051700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 0x7f1a7813d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:43.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7d852700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a7400bcf0 tx=0x7f1a74016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401c780 con 0x7f1a780b1f20 2020-11-16T22:14:43.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7401c8e0 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401ca90 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a78153ce0 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a781505b0 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.271+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a78150860 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.275+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1a7402c070 con 0x7f1a780b1f20 2020-11-16T22:14:43.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.275+0000 7f1a65ffb700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.275+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a74071c00 con 0x7f1a780b1f20 2020-11-16T22:14:43.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.275+0000 7f1a7d051700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.275+0000 7f1a7d051700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f1a7813e060 tx=0x7f1a68009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.283+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1a74047e60 con 0x7f1a780b1f20 2020-11-16T22:14:43.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.431+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a7805d200 con 0x7f1a780b1f20 2020-11-16T22:14:43.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.431+0000 7f1a65ffb700 1 -- 172.21.15.185:0/4152250636 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a7402a0a0 con 0x7f1a780b1f20 2020-11-16T22:14:43.420 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:43.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 msgr2=0x7f1a50057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f1a7813e060 tx=0x7f1a68009040).stop 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 msgr2=0x7f1a78154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a7400bcf0 tx=0x7f1a74016000).stop 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 shutdown_connections 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a780b1f20 0x7f1a78154180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a78141690 0x7f1a781500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 --2- 172.21.15.185:0/4152250636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a780b7690 0x7f1a7813d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 >> 172.21.15.185:0/4152250636 conn(0x7f1a78064b90 msgr2=0x7f1a7813cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 shutdown_connections 2020-11-16T22:14:43.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.435+0000 7f1a7e854700 1 -- 172.21.15.185:0/4152250636 wait complete. 2020-11-16T22:14:43.433 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:43.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 Processor -- start 2020-11-16T22:14:43.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- start start 2020-11-16T22:14:43.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fded80b5d70 0x7fded80ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fded80b4470 0x7fded80b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded80b7290 con 0x7fded81475d0 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded80b4bf0 con 0x7fded80b4470 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded80b74c0 con 0x7fded80b5d70 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fded80b5d70 0x7fded80ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fded80b5d70 0x7fded80ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56762/0 (socket says 172.21.15.185:56762) 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 -- 172.21.15.185:0/2557272247 learned_addr learned my addr 172.21.15.185:0/2557272247 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:43.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded77fe700 1 --2- 172.21.15.185:0/2557272247 >> v2:172.21.15.96:3300/0 conn(0x7fded80b4470 0x7fded80b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 -- 172.21.15.185:0/2557272247 >> v2:172.21.15.96:3300/0 conn(0x7fded80b4470 msgr2=0x7fded80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 --2- 172.21.15.185:0/2557272247 >> v2:172.21.15.96:3300/0 conn(0x7fded80b4470 0x7fded80b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 -- 172.21.15.185:0/2557272247 >> v1:172.21.15.96:6789/0 conn(0x7fded81475d0 legacy=0x7fded8147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 -- 172.21.15.185:0/2557272247 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80b7760 con 0x7fded80b5d70 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded7fff700 1 --2- 172.21.15.185:0/2557272247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded80b5d70 0x7fded80ba800 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fded805c5f0 tx=0x7fdec8006eb0).ready entity=mon.1 client_cookie=8641e312864176c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded6ffd700 1 -- 172.21.15.185:0/2557272247 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80098d0 con 0x7fded80b5d70 2020-11-16T22:14:43.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded6ffd700 1 -- 172.21.15.185:0/2557272247 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8009a30 con 0x7fded80b5d70 2020-11-16T22:14:43.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fded6ffd700 1 -- 172.21.15.185:0/2557272247 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8009db0 con 0x7fded80b5d70 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- 172.21.15.185:0/2557272247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded80b5d70 msgr2=0x7fded80ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 --2- 172.21.15.185:0/2557272247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded80b5d70 0x7fded80ba800 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fded805c5f0 tx=0x7fdec8006eb0).stop 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- 172.21.15.185:0/2557272247 shutdown_connections 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 --2- 172.21.15.185:0/2557272247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded80b5d70 0x7fded80ba800 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 --2- 172.21.15.185:0/2557272247 >> v2:172.21.15.96:3300/0 conn(0x7fded80b4470 0x7fded80b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- 172.21.15.185:0/2557272247 >> 172.21.15.185:0/2557272247 conn(0x7fded805ba80 msgr2=0x7fded805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- 172.21.15.185:0/2557272247 shutdown_connections 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.571+0000 7fdedd122700 1 -- 172.21.15.185:0/2557272247 wait complete. 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 Processor -- start 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- start start 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded80b5d70 0x7fded8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded81475d0 0x7fded80b9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded80af7b0 con 0x7fded80b5d70 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded8156d30 con 0x7fded81475d0 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded8156fd0 con 0x7fded80b4470 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50212/0 (socket says 172.21.15.185:50212) 2020-11-16T22:14:43.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 -- 172.21.15.185:0/2292627196 learned_addr learned my addr 172.21.15.185:0/2292627196 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded77fe700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded80b5d70 0x7fded8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedc921700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded81475d0 0x7fded80b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 -- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded81475d0 msgr2=0x7fded80b9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded81475d0 0x7fded80b9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 -- 172.21.15.185:0/2292627196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded80b5d70 msgr2=0x7fded8157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded80b5d70 0x7fded8157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 -- 172.21.15.185:0/2292627196 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8157960 con 0x7fded80b4470 2020-11-16T22:14:43.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fded7fff700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fded80b7600 tx=0x7fdec8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8011d50 con 0x7fded80b4470 2020-11-16T22:14:43.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec801e460 con 0x7fded80b4470 2020-11-16T22:14:43.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded8114e40 con 0x7fded80b4470 2020-11-16T22:14:43.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec801e7e0 con 0x7fded80b4470 2020-11-16T22:14:43.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.575+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded80ba030 con 0x7fded80b4470 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded80af920 con 0x7fded80b4470 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdec8025030 con 0x7fded80b4470 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fdeb7fff700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 0x7fdeb0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdec806f280 con 0x7fded80b4470 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fded77fe700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 0x7fdeb0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.579+0000 7fded77fe700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 0x7fdeb0057520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fded80b7440 tx=0x7fdecc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.591+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fded80af920 con 0x7fded80b4470 2020-11-16T22:14:43.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fded80af920 con 0x7fded80b4470 2020-11-16T22:14:43.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdeb7fff700 1 -- 172.21.15.185:0/2292627196 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fded80af920 con 0x7fded80b4470 2020-11-16T22:14:43.730 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:43.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 msgr2=0x7fdeb0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 0x7fdeb0057520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fded80b7440 tx=0x7fdecc008040).stop 2020-11-16T22:14:43.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 msgr2=0x7fded80aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fded80b7600 tx=0x7fdec8015000).stop 2020-11-16T22:14:43.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 shutdown_connections 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdeb0055060 0x7fdeb0057520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fded81475d0 0x7fded80b9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fded80b4470 0x7fded80aea60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 --2- 172.21.15.185:0/2292627196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded80b5d70 0x7fded8157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 >> 172.21.15.185:0/2292627196 conn(0x7fded805ba80 msgr2=0x7fded8144590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 shutdown_connections 2020-11-16T22:14:43.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.743+0000 7fdedd122700 1 -- 172.21.15.185:0/2292627196 wait complete. 2020-11-16T22:14:43.744 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:43.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 Processor -- start 2020-11-16T22:14:43.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- start start 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc3cc0b7440 0x7fc3cc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc0afc60 0x7fc3cc144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc144d50 con 0x7fc3cc13dea0 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0ad490 con 0x7fc3cc0b7440 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0ad730 con 0x7fc3cc0afc60 2020-11-16T22:14:43.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cbfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc3cc13dea0 0x7fc3cc0ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50216/0 (socket says 172.21.15.185:50216) 2020-11-16T22:14:43.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc3cc0b7440 0x7fc3cc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3caffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc0afc60 0x7fc3cc144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cbfff700 1 -- 172.21.15.185:0/1218699973 learned_addr learned my addr 172.21.15.185:0/1218699973 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:43.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc0afc60 msgr2=0x7fc3cc144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:43.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 --2- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc0afc60 0x7fc3cc144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/1218699973 >> v1:172.21.15.185:6789/0 conn(0x7fc3cc13dea0 legacy=0x7fc3cc0ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/1218699973 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc0ad9d0 con 0x7fc3cc0b7440 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3cb7fe700 1 --2- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc0b7820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc3b400b9a0 tx=0x7fc3b4006df0).ready entity=mon.1 client_cookie=bd75ef279d7b6918 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3ca7fc700 1 -- 172.21.15.185:0/1218699973 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40097b0 con 0x7fc3cc0b7440 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3ca7fc700 1 -- 172.21.15.185:0/1218699973 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4009910 con 0x7fc3cc0b7440 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3ca7fc700 1 -- 172.21.15.185:0/1218699973 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4009c90 con 0x7fc3cc0b7440 2020-11-16T22:14:43.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0b7440 msgr2=0x7fc3cc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc0b7820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc3b400b9a0 tx=0x7fc3b4006df0).stop 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/1218699973 shutdown_connections 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc0b7820 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/1218699973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc0afc60 0x7fc3cc144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/1218699973 >> 172.21.15.185:0/1218699973 conn(0x7fc3cc067860 msgr2=0x7fc3cc067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/1218699973 shutdown_connections 2020-11-16T22:14:43.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.887+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/1218699973 wait complete. 2020-11-16T22:14:43.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 Processor -- start 2020-11-16T22:14:43.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- start start 2020-11-16T22:14:43.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc13ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc13dea0 0x7fc3cc0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b3ba0 con 0x7fc3cc0b7440 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b49e0 con 0x7fc3cc0afc60 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b3900 con 0x7fc3cc13dea0 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56782/0 (socket says 172.21.15.185:56782) 2020-11-16T22:14:43.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/384976791 learned_addr learned my addr 172.21.15.185:0/384976791 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cbfff700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc13dea0 0x7fc3cc0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3caffd700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc13ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc13dea0 msgr2=0x7fc3cc0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc13dea0 0x7fc3cc0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/384976791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3cc0b7440 msgr2=0x7fc3cc13ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc13ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:43.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 -- 172.21.15.185:0/384976791 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc05cd50 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3cb7fe700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc3b40098e0 tx=0x7fc3b4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4011d50 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b401e460 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc1480e0 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b401e7e0 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc0b8d40 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc05d4e0 con 0x7fc3cc0afc60 2020-11-16T22:14:43.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.891+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc3b4025030 con 0x7fc3cc0afc60 2020-11-16T22:14:43.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.895+0000 7fc3c37fe700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 0x7fc3a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:43.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.895+0000 7fc3caffd700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 0x7fc3a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:43.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.895+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc3b406f630 con 0x7fc3cc0afc60 2020-11-16T22:14:43.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.895+0000 7fc3caffd700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 0x7fc3a8057520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc3cc0b6220 tx=0x7fc3bc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:43.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:43.899+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc3b4045810 con 0x7fc3cc0afc60 2020-11-16T22:14:44.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.047+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3cc144fe0 con 0x7fc3cc0afc60 2020-11-16T22:14:44.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3c37fe700 1 -- 172.21.15.185:0/384976791 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc3b4023020 con 0x7fc3cc0afc60 2020-11-16T22:14:44.037 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:44.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 msgr2=0x7fc3a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 0x7fc3a8057520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc3cc0b6220 tx=0x7fc3bc008040).stop 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 msgr2=0x7fc3cc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fc3b40098e0 tx=0x7fc3b4015000).stop 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 shutdown_connections 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3a8055060 0x7fc3a8057520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:44.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc3cc0afc60 0x7fc3cc0b25d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:44.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc3cc13dea0 0x7fc3cc0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 --2- 172.21.15.185:0/384976791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3cc0b7440 0x7fc3cc13ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 >> 172.21.15.185:0/384976791 conn(0x7fc3cc067860 msgr2=0x7fc3cc0ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 shutdown_connections 2020-11-16T22:14:48.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:44.051+0000 7fc3d1ef9700 1 -- 172.21.15.185:0/384976791 wait complete. 2020-11-16T22:14:48.220 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:48.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 Processor -- start 2020-11-16T22:14:48.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 -- start start 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 0x7f88e012d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e00b82e0 con 0x7f88e00b4720 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e00b7650 con 0x7f88e0144b90 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e586d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00b78f0 con 0x7f88e012cca0 2020-11-16T22:14:48.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88dffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 0x7f88e012d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88dffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56790/0 (socket says 172.21.15.185:56790) 2020-11-16T22:14:48.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88dffff700 1 -- 172.21.15.185:0/2165861510 learned_addr learned my addr 172.21.15.185:0/2165861510 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:48.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 -- 172.21.15.185:0/2165861510 >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 msgr2=0x7f88e00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:48.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 --2- 172.21.15.185:0/2165861510 >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 -- 172.21.15.185:0/2165861510 >> v1:172.21.15.96:6789/0 conn(0x7f88e00b4720 legacy=0x7f88e00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 -- 172.21.15.185:0/2165861510 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00b1ee0 con 0x7f88e012cca0 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88dffff700 1 --2- 172.21.15.185:0/2165861510 >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88e506c700 1 --2- 172.21.15.185:0/2165861510 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 0x7f88e012d130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f88d8009c90 tx=0x7f88d80091d0).ready entity=mon.2 client_cookie=bc840b96101c2b91 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.363+0000 7f88df7fe700 1 -- 172.21.15.185:0/2165861510 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800ba70 con 0x7f88e012cca0 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88df7fe700 1 -- 172.21.15.185:0/2165861510 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800bbd0 con 0x7f88e012cca0 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88df7fe700 1 -- 172.21.15.185:0/2165861510 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011400 con 0x7f88e012cca0 2020-11-16T22:14:48.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- 172.21.15.185:0/2165861510 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 msgr2=0x7f88e012d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- 172.21.15.185:0/2165861510 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 0x7f88e012d130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f88d8009c90 tx=0x7f88d80091d0).stop 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- 172.21.15.185:0/2165861510 shutdown_connections 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- 172.21.15.185:0/2165861510 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e012cca0 0x7f88e012d130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- 172.21.15.185:0/2165861510 >> v2:172.21.15.185:3300/0 conn(0x7f88e0144b90 0x7f88e00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- 172.21.15.185:0/2165861510 >> 172.21.15.185:0/2165861510 conn(0x7f88e0067860 msgr2=0x7f88e0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- 172.21.15.185:0/2165861510 shutdown_connections 2020-11-16T22:14:48.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- 172.21.15.185:0/2165861510 wait complete. 2020-11-16T22:14:48.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 Processor -- start 2020-11-16T22:14:48.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- start start 2020-11-16T22:14:48.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 0x7f88e01478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88e012cca0 0x7f88e0147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0129ad0 con 0x7f88e012cca0 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e01569a0 con 0x7f88e00b4720 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.367+0000 7f88e586d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00aefe0 con 0x7f88e0144b90 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e486b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 0x7f88e01478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50244/0 (socket says 172.21.15.185:50244) 2020-11-16T22:14:48.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 -- 172.21.15.185:0/3874348108 learned_addr learned my addr 172.21.15.185:0/3874348108 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88dffff700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88e012cca0 0x7f88e0147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 -- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 msgr2=0x7f88e01478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 0x7f88e01478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 -- 172.21.15.185:0/3874348108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88e012cca0 msgr2=0x7f88e0147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88e012cca0 0x7f88e0147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 -- 172.21.15.185:0/3874348108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00af280 con 0x7f88e0144b90 2020-11-16T22:14:48.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e486b700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 0x7f88e01478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:48.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e506c700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f88d800c010 tx=0x7f88d8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d801c5f0 con 0x7f88e0144b90 2020-11-16T22:14:48.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d801c750 con 0x7f88e0144b90 2020-11-16T22:14:48.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d801cad0 con 0x7f88e0144b90 2020-11-16T22:14:48.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e00af520 con 0x7f88e0144b90 2020-11-16T22:14:48.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e0146fc0 con 0x7f88e0144b90 2020-11-16T22:14:48.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.371+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e005d200 con 0x7f88e0144b90 2020-11-16T22:14:48.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.375+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f88d8017070 con 0x7f88e0144b90 2020-11-16T22:14:48.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.375+0000 7f88dcff9700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 0x7f88b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.375+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f88d806ffc0 con 0x7f88e0144b90 2020-11-16T22:14:48.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.375+0000 7f88e486b700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 0x7f88b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.375+0000 7f88e486b700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 0x7f88b8057520 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f88e0067f50 tx=0x7f88cc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.383+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f88d80462a0 con 0x7f88e0144b90 2020-11-16T22:14:48.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88e0148540 con 0x7f88e0144b90 2020-11-16T22:14:48.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88dcff9700 1 -- 172.21.15.185:0/3874348108 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f88d8015030 con 0x7f88e0144b90 2020-11-16T22:14:48.523 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:48.523 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:48.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 msgr2=0x7f88b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 0x7f88b8057520 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f88e0067f50 tx=0x7f88cc009040).stop 2020-11-16T22:14:48.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 msgr2=0x7f88e0146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.535+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f88d800c010 tx=0x7f88d8016000).stop 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 shutdown_connections 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88b8055060 0x7f88b8057520 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88e00b4720 0x7f88e01478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88e0144b90 0x7f88e0146a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 --2- 172.21.15.185:0/3874348108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88e012cca0 0x7f88e0147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 >> 172.21.15.185:0/3874348108 conn(0x7f88e0067860 msgr2=0x7f88e00b98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 shutdown_connections 2020-11-16T22:14:48.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.539+0000 7f88e586d700 1 -- 172.21.15.185:0/3874348108 wait complete. 2020-11-16T22:14:48.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:48.538 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:48.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.679+0000 7fec65798700 1 Processor -- start 2020-11-16T22:14:48.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.679+0000 7fec65798700 1 -- start start 2020-11-16T22:14:48.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fec6005ba80 0x7fec60144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec600b2f30 con 0x7fec60147fb0 2020-11-16T22:14:48.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec60143f90 con 0x7fec600b1a20 2020-11-16T22:14:48.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec60144fc0 con 0x7fec6005ba80 2020-11-16T22:14:48.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fec600b1a20 0x7fec601411b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50242/0 (socket says 172.21.15.185:50242) 2020-11-16T22:14:48.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5ffff700 1 -- 172.21.15.185:0/2480021622 learned_addr learned my addr 172.21.15.185:0/2480021622 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:48.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec64f97700 1 --2- 172.21.15.185:0/2480021622 >> v2:172.21.15.96:3300/0 conn(0x7fec6005ba80 0x7fec60144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969361907 0 0) 0x7fec60143f90 con 0x7fec600b1a20 2020-11-16T22:14:48.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec40003240 con 0x7fec600b1a20 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3439053006 0 0) 0x7fec40003240 con 0x7fec600b1a20 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 >> v2:172.21.15.96:3300/0 conn(0x7fec6005ba80 msgr2=0x7fec60144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 --2- 172.21.15.185:0/2480021622 >> v2:172.21.15.96:3300/0 conn(0x7fec6005ba80 0x7fec60144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 >> v1:172.21.15.96:6789/0 conn(0x7fec60147fb0 legacy=0x7fec601443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60060fe0 con 0x7fec600b1a20 2020-11-16T22:14:48.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fec500014e0 con 0x7fec600b1a20 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 >> v1:172.21.15.185:6789/0 conn(0x7fec600b1a20 legacy=0x7fec601411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec40003730 0x7fec40003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 0x7fec40004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec40004e50 0x7fec4000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60144fc0 con 0x7fec40003730 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40004350 con 0x7fec400044e0 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4000bc50 con 0x7fec40004e50 2020-11-16T22:14:48.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 0x7fec40004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5ffff700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec40003730 0x7fec40003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec64f97700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec40004e50 0x7fec4000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 -- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec40004e50 msgr2=0x7fec4000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec40004e50 0x7fec4000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 -- 172.21.15.185:0/2480021622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec40003730 msgr2=0x7fec40003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec40003730 0x7fec40003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 -- 172.21.15.185:0/2480021622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40003240 con 0x7fec400044e0 2020-11-16T22:14:48.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec64f97700 1 -- 172.21.15.185:0/2480021622 reap_dead start 2020-11-16T22:14:48.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec577fe700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 0x7fec40004960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fec480018e0 tx=0x7fec48007b90).ready entity=mon.1 client_cookie=b9638b83072d2838 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48007020 con 0x7fec400044e0 2020-11-16T22:14:48.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec48007180 con 0x7fec400044e0 2020-11-16T22:14:48.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec5f7fe700 1 -- 172.21.15.185:0/2480021622 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4800c830 con 0x7fec400044e0 2020-11-16T22:14:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 -- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 msgr2=0x7fec40004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.683+0000 7fec65798700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 0x7fec40004960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fec480018e0 tx=0x7fec48007b90).stop 2020-11-16T22:14:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- 172.21.15.185:0/2480021622 shutdown_connections 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 --2- 172.21.15.185:0/2480021622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec400044e0 0x7fec40004960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- 172.21.15.185:0/2480021622 >> 172.21.15.185:0/2480021622 conn(0x7fec60063df0 msgr2=0x7fec600641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- 172.21.15.185:0/2480021622 shutdown_connections 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- 172.21.15.185:0/2480021622 wait complete. 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 Processor -- start 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- start start 2020-11-16T22:14:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec6013e740 0x7fec60143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 0x7fec60143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec600ad8c0 con 0x7fec600b0f80 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec5ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60141ab0 con 0x7fec6013e740 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec65798700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec60141d50 con 0x7fec400044e0 2020-11-16T22:14:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec5ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50260/0 (socket says 172.21.15.185:50260) 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec5ffff700 1 -- 172.21.15.185:0/875913639 learned_addr learned my addr 172.21.15.185:0/875913639 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec577fe700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec6013e740 0x7fec60143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.687+0000 7fec64f97700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 0x7fec60143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 -- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec6013e740 msgr2=0x7fec60143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec6013e740 0x7fec60143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 -- 172.21.15.185:0/875913639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 msgr2=0x7fec60143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 0x7fec60143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 -- 172.21.15.185:0/875913639 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60143040 con 0x7fec400044e0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec64f97700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 0x7fec60143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5ffff700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fec500077a0 tx=0x7fec500077d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000f960 con 0x7fec400044e0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5000fae0 con 0x7fec400044e0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec601432e0 con 0x7fec400044e0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000fea0 con 0x7fec400044e0 2020-11-16T22:14:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec60140450 con 0x7fec400044e0 2020-11-16T22:14:48.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec60141f10 con 0x7fec400044e0 2020-11-16T22:14:48.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fec5001b030 con 0x7fec400044e0 2020-11-16T22:14:48.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 0x7fec3c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec577fe700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 0x7fec3c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec5006d640 con 0x7fec400044e0 2020-11-16T22:14:48.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.691+0000 7fec577fe700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 0x7fec3c057520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fec60145d10 tx=0x7fec48015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.699+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fec50043950 con 0x7fec400044e0 2020-11-16T22:14:48.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fec6005d200 con 0x7fec400044e0 2020-11-16T22:14:48.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec5cff9700 1 -- 172.21.15.185:0/875913639 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fec50046f70 con 0x7fec400044e0 2020-11-16T22:14:48.838 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:48.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 msgr2=0x7fec3c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 0x7fec3c057520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fec60145d10 tx=0x7fec48015000).stop 2020-11-16T22:14:48.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 msgr2=0x7fec601423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fec500077a0 tx=0x7fec500077d0).stop 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 shutdown_connections 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec3c055060 0x7fec3c057520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec600b0f80 0x7fec60143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec400044e0 0x7fec601423f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 --2- 172.21.15.185:0/875913639 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec6013e740 0x7fec60143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 >> 172.21.15.185:0/875913639 conn(0x7fec60063df0 msgr2=0x7fec60145350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.851+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 shutdown_connections 2020-11-16T22:14:48.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.855+0000 7fec65798700 1 -- 172.21.15.185:0/875913639 wait complete. 2020-11-16T22:14:48.853 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:48.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 Processor -- start 2020-11-16T22:14:48.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 -- start start 2020-11-16T22:14:48.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5fa8144c00 0x7f5fa80afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80b02e0 con 0x7f5fa80ade00 2020-11-16T22:14:48.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80b0580 con 0x7f5fa80b74b0 2020-11-16T22:14:48.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5faebc5700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa8143fa0 con 0x7f5fa8144c00 2020-11-16T22:14:48.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5fa8144c00 0x7f5fa80afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fac961700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5fa80b74b0 0x7f5fa80ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50262/0 (socket says 172.21.15.185:50262) 2020-11-16T22:14:48.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fac961700 1 -- 172.21.15.185:0/2549419886 learned_addr learned my addr 172.21.15.185:0/2549419886 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:48.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa7fff700 1 -- 172.21.15.185:0/2549419886 >> v1:172.21.15.185:6789/0 conn(0x7f5fa80b74b0 legacy=0x7f5fa80ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa7fff700 1 -- 172.21.15.185:0/2549419886 >> v1:172.21.15.96:6789/0 conn(0x7f5fa80ade00 legacy=0x7f5fa8142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa77fe700 1 -- 172.21.15.185:0/2549419886 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938947149 0 0) 0x7f5fa80b0580 con 0x7f5fa80b74b0 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa7fff700 1 -- 172.21.15.185:0/2549419886 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80ad250 con 0x7f5fa8144c00 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa7fff700 1 --2- 172.21.15.185:0/2549419886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa8144c00 0x7f5fa80afc40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5f9800b990 tx=0x7f5f98006d90).ready entity=mon.1 client_cookie=b3c222a74ddc807f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa77fe700 1 -- 172.21.15.185:0/2549419886 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f980096b0 con 0x7f5fa8144c00 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa77fe700 1 -- 172.21.15.185:0/2549419886 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f98009810 con 0x7f5fa8144c00 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.995+0000 7f5fa77fe700 1 -- 172.21.15.185:0/2549419886 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98009b90 con 0x7f5fa8144c00 2020-11-16T22:14:48.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- 172.21.15.185:0/2549419886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa8144c00 msgr2=0x7f5fa80afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 --2- 172.21.15.185:0/2549419886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa8144c00 0x7f5fa80afc40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5f9800b990 tx=0x7f5f98006d90).stop 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- 172.21.15.185:0/2549419886 shutdown_connections 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 --2- 172.21.15.185:0/2549419886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa8144c00 0x7f5fa80afc40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- 172.21.15.185:0/2549419886 >> 172.21.15.185:0/2549419886 conn(0x7f5fa8067860 msgr2=0x7f5fa8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- 172.21.15.185:0/2549419886 shutdown_connections 2020-11-16T22:14:48.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- 172.21.15.185:0/2549419886 wait complete. 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 Processor -- start 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- start start 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 0x7f5fa8140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fa80b74b0 0x7f5fa80b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 0x7f5fa80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa813f070 con 0x7f5fa80b74b0 2020-11-16T22:14:48.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8147fa0 con 0x7f5fa80ade00 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5faebc5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81454a0 con 0x7f5fa8144c00 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 0x7f5fa8140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fad162700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 0x7f5fa80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fad162700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 0x7f5fa80b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50276/0 (socket says 172.21.15.185:50276) 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fad162700 1 -- 172.21.15.185:0/3850396029 learned_addr learned my addr 172.21.15.185:0/3850396029 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fa7fff700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fa80b74b0 0x7f5fa80b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 -- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 msgr2=0x7f5fa80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 0x7f5fa80b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 -- 172.21.15.185:0/3850396029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fa80b74b0 msgr2=0x7f5fa80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:48.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fa80b74b0 0x7f5fa80b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:48.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 -- 172.21.15.185:0/3850396029 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80b47f0 con 0x7f5fa80ade00 2020-11-16T22:14:48.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:48.999+0000 7f5fac961700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 0x7f5fa8140bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5f9000c6e0 tx=0x7f5f900079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900072e0 con 0x7f5fa80ade00 2020-11-16T22:14:48.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f90007460 con 0x7f5fa80ade00 2020-11-16T22:14:48.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa8145970 con 0x7f5fa80ade00 2020-11-16T22:14:48.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90005840 con 0x7f5fa80ade00 2020-11-16T22:14:48.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa8156830 con 0x7f5fa80ade00 2020-11-16T22:14:48.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8146cb0 con 0x7f5fa80ade00 2020-11-16T22:14:48.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5f90010070 con 0x7f5fa80ade00 2020-11-16T22:14:48.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 0x7f5f84057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:48.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa7fff700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 0x7f5f84057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:48.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5f9006d160 con 0x7f5fa80ade00 2020-11-16T22:14:48.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.003+0000 7f5fa7fff700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 0x7f5f84057520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f5f98006dd0 tx=0x7f5f98015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:48.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.011+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5fa8146cb0 con 0x7f5fa80ade00 2020-11-16T22:14:49.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.163+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5fa80b47f0 con 0x7f5fa80ade00 2020-11-16T22:14:49.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.163+0000 7f5fa4ff9700 1 -- 172.21.15.185:0/3850396029 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5fa80b47f0 con 0x7f5fa80ade00 2020-11-16T22:14:49.152 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:49.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 msgr2=0x7f5f84057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 0x7f5f84057520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f5f98006dd0 tx=0x7f5f98015000).stop 2020-11-16T22:14:49.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 msgr2=0x7f5fa8140bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 0x7f5fa8140bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5f9000c6e0 tx=0x7f5f900079e0).stop 2020-11-16T22:14:49.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 shutdown_connections 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5f84055060 0x7f5f84057520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5fa80ade00 0x7f5fa8140bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5fa8144c00 0x7f5fa80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 --2- 172.21.15.185:0/3850396029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fa80b74b0 0x7f5fa80b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 >> 172.21.15.185:0/3850396029 conn(0x7f5fa8067860 msgr2=0x7f5fa80b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 shutdown_connections 2020-11-16T22:14:49.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.167+0000 7f5faebc5700 1 -- 172.21.15.185:0/3850396029 wait complete. 2020-11-16T22:14:49.171 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:49.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 Processor -- start 2020-11-16T22:14:49.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 -- start start 2020-11-16T22:14:49.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb09c0675e0 0x7fb09c0679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09c142f70 con 0x7fb09c0b2ec0 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09c068680 con 0x7fb09c0675e0 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a33ab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0b33d0 con 0x7fb09c0b45a0 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a1948700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a0946700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb09c0b2ec0 0x7fb09c13bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50278/0 (socket says 172.21.15.185:50278) 2020-11-16T22:14:49.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a1948700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50284/0 (socket says 172.21.15.185:50284) 2020-11-16T22:14:49.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a0946700 1 -- 172.21.15.185:0/2930895457 learned_addr learned my addr 172.21.15.185:0/2930895457 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:49.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.315+0000 7fb0a1147700 1 --2- 172.21.15.185:0/2930895457 >> v2:172.21.15.96:3300/0 conn(0x7fb09c0675e0 0x7fb09c0679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb093fff700 1 -- 172.21.15.185:0/2930895457 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214401531 0 0) 0x7fb08c0010d0 con 0x7fb09c0b2ec0 2020-11-16T22:14:49.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a1948700 1 -- 172.21.15.185:0/2930895457 >> v2:172.21.15.96:3300/0 conn(0x7fb09c0675e0 msgr2=0x7fb09c0679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a1948700 1 --2- 172.21.15.185:0/2930895457 >> v2:172.21.15.96:3300/0 conn(0x7fb09c0675e0 0x7fb09c0679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a1948700 1 -- 172.21.15.185:0/2930895457 >> v1:172.21.15.185:6789/0 conn(0x7fb09c0b2ec0 legacy=0x7fb09c13bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a1948700 1 -- 172.21.15.185:0/2930895457 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c0b3670 con 0x7fb09c0b45a0 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a1948700 1 --2- 172.21.15.185:0/2930895457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb094009c90 tx=0x7fb0940091d0).ready entity=mon.2 client_cookie=dc6b1f4e07ff6921 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb093fff700 1 -- 172.21.15.185:0/2930895457 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400ba70 con 0x7fb09c0b45a0 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb093fff700 1 -- 172.21.15.185:0/2930895457 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09400bbd0 con 0x7fb09c0b45a0 2020-11-16T22:14:49.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb093fff700 1 -- 172.21.15.185:0/2930895457 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094011400 con 0x7fb09c0b45a0 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- 172.21.15.185:0/2930895457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 msgr2=0x7fb09c145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/2930895457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb094009c90 tx=0x7fb0940091d0).stop 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- 172.21.15.185:0/2930895457 shutdown_connections 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/2930895457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b45a0 0x7fb09c145af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/2930895457 >> v2:172.21.15.96:3300/0 conn(0x7fb09c0675e0 0x7fb09c0679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- 172.21.15.185:0/2930895457 >> 172.21.15.185:0/2930895457 conn(0x7fb09c061240 msgr2=0x7fb09c061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- 172.21.15.185:0/2930895457 shutdown_connections 2020-11-16T22:14:49.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- 172.21.15.185:0/2930895457 wait complete. 2020-11-16T22:14:49.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 Processor -- start 2020-11-16T22:14:49.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.319+0000 7fb0a33ab700 1 -- start start 2020-11-16T22:14:49.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b2ec0 0x7fb09c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb09c0b45a0 0x7fb09c0ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0ac8d0 con 0x7fb09c0b45a0 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c142750 con 0x7fb09c0675e0 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c1414b0 con 0x7fb09c0b2ec0 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56844/0 (socket says 172.21.15.185:56844) 2020-11-16T22:14:49.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 -- 172.21.15.185:0/3660484526 learned_addr learned my addr 172.21.15.185:0/3660484526 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a0946700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b2ec0 0x7fb09c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 -- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b2ec0 msgr2=0x7fb09c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b2ec0 0x7fb09c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 -- 172.21.15.185:0/3660484526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb09c0b45a0 msgr2=0x7fb09c0ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb09c0b45a0 0x7fb09c0ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 -- 172.21.15.185:0/3660484526 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c143da0 con 0x7fb09c0675e0 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a1147700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb09800ebd0 tx=0x7fb098006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800ca60 con 0x7fb09c0675e0 2020-11-16T22:14:49.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800cbc0 con 0x7fb09c0675e0 2020-11-16T22:14:49.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0980123f0 con 0x7fb09c0675e0 2020-11-16T22:14:49.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c1416e0 con 0x7fb09c0675e0 2020-11-16T22:14:49.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c1442a0 con 0x7fb09c0675e0 2020-11-16T22:14:49.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb098012760 con 0x7fb09c0675e0 2020-11-16T22:14:49.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.323+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c0b3850 con 0x7fb09c0675e0 2020-11-16T22:14:49.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.327+0000 7fb0917fa700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 0x7fb074057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.327+0000 7fb0a0946700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 0x7fb074057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.327+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb09806e2a0 con 0x7fb09c0675e0 2020-11-16T22:14:49.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.327+0000 7fb0a0946700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 0x7fb074057550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb09c1494f0 tx=0x7fb08c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.331+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb0980443f0 con 0x7fb09c0675e0 2020-11-16T22:14:49.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb09c13d840 con 0x7fb09c0675e0 2020-11-16T22:14:49.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0917fa700 1 -- 172.21.15.185:0/3660484526 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb098047a10 con 0x7fb09c0675e0 2020-11-16T22:14:49.469 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:49.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 msgr2=0x7fb074057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 0x7fb074057550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb09c1494f0 tx=0x7fb08c009040).stop 2020-11-16T22:14:49.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 msgr2=0x7fb09c13d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb09800ebd0 tx=0x7fb098006df0).stop 2020-11-16T22:14:49.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 shutdown_connections 2020-11-16T22:14:49.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb074055090 0x7fb074057550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb09c0675e0 0x7fb09c13d2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb09c0b45a0 0x7fb09c0ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-16T22:14:49.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 --2- 172.21.15.185:0/3660484526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb09c0b2ec0 0x7fb09c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 >> 172.21.15.185:0/3660484526 conn(0x7fb09c061240 msgr2=0x7fb09c062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 shutdown_connections 2020-11-16T22:14:49.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.483+0000 7fb0a33ab700 1 -- 172.21.15.185:0/3660484526 wait complete. 2020-11-16T22:14:49.483 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:49.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.623+0000 7fdbcb1fa700 1 Processor -- start 2020-11-16T22:14:49.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.623+0000 7fdbcb1fa700 1 -- start start 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 0x7fdbc40666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc40b7c20 con 0x7fdbc40b3570 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc40b3a80 con 0x7fdbc413de10 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40b3d20 con 0x7fdbc413e1f0 2020-11-16T22:14:49.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50298/0 (socket says 172.21.15.185:50298) 2020-11-16T22:14:49.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 -- 172.21.15.185:0/4232324863 learned_addr learned my addr 172.21.15.185:0/4232324863 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:49.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbca1f8700 1 --2- 172.21.15.185:0/4232324863 >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 0x7fdbc40666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 -- 172.21.15.185:0/4232324863 >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 msgr2=0x7fdbc40666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:49.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/4232324863 >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 0x7fdbc40666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 -- 172.21.15.185:0/4232324863 >> v1:172.21.15.96:6789/0 conn(0x7fdbc40b3570 legacy=0x7fdbc40b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 -- 172.21.15.185:0/4232324863 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc40b3fc0 con 0x7fdbc413e1f0 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbca1f8700 1 --2- 172.21.15.185:0/4232324863 >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 0x7fdbc40666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/4232324863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdbb4006fd0 tx=0x7fdbb4006eb0).ready entity=mon.2 client_cookie=5a42129be6a09846 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc91f6700 1 -- 172.21.15.185:0/4232324863 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40098d0 con 0x7fdbc413e1f0 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc91f6700 1 -- 172.21.15.185:0/4232324863 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4009a30 con 0x7fdbc413e1f0 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbc91f6700 1 -- 172.21.15.185:0/4232324863 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4009db0 con 0x7fdbc413e1f0 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/4232324863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 msgr2=0x7fdbc4143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/4232324863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdbb4006fd0 tx=0x7fdbb4006eb0).stop 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/4232324863 shutdown_connections 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/4232324863 >> v2:172.21.15.96:3300/0 conn(0x7fdbc413de10 0x7fdbc40666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/4232324863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413e1f0 0x7fdbc4143f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/4232324863 >> 172.21.15.185:0/4232324863 conn(0x7fdbc4064b90 msgr2=0x7fdbc4063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/4232324863 shutdown_connections 2020-11-16T22:14:49.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/4232324863 wait complete. 2020-11-16T22:14:49.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.627+0000 7fdbcb1fa700 1 Processor -- start 2020-11-16T22:14:49.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- start start 2020-11-16T22:14:49.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 0x7fdbc4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc4142e70 0x7fdbc41432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc413e920 con 0x7fdbc4142e70 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40afb80 con 0x7fdbc40b3570 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc41437c0 con 0x7fdbc413de10 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56858/0 (socket says 172.21.15.185:56858) 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 -- 172.21.15.185:0/3117990808 learned_addr learned my addr 172.21.15.185:0/3117990808 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 0x7fdbc4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca9f9700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc4142e70 0x7fdbc41432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 -- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 msgr2=0x7fdbc4153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 0x7fdbc4153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 -- 172.21.15.185:0/3117990808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc4142e70 msgr2=0x7fdbc41432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc4142e70 0x7fdbc41432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 -- 172.21.15.185:0/3117990808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc413f1e0 con 0x7fdbc40b3570 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 0x7fdbc4153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:49.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbca1f8700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fdbc000db60 tx=0x7fdbc0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0009790 con 0x7fdbc40b3570 2020-11-16T22:14:49.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc00098f0 con 0x7fdbc40b3570 2020-11-16T22:14:49.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc413f480 con 0x7fdbc40b3570 2020-11-16T22:14:49.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0009c90 con 0x7fdbc40b3570 2020-11-16T22:14:49.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc40af070 con 0x7fdbc40b3570 2020-11-16T22:14:49.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.631+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc413fbc0 con 0x7fdbc40b3570 2020-11-16T22:14:49.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.635+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdbc0013480 con 0x7fdbc40b3570 2020-11-16T22:14:49.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.635+0000 7fdbb27fc700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 0x7fdba0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.635+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbc006e3f0 con 0x7fdbc40b3570 2020-11-16T22:14:49.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.635+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 0x7fdba0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.635+0000 7fdbc99f7700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 0x7fdba0057520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdbb4006ef0 tx=0x7fdbb4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.643+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdbc413fbc0 con 0x7fdbc40b3570 2020-11-16T22:14:49.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbc405d200 con 0x7fdbc40b3570 2020-11-16T22:14:49.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbb27fc700 1 -- 172.21.15.185:0/3117990808 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdbc00444e0 con 0x7fdbc40b3570 2020-11-16T22:14:49.783 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:49.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 msgr2=0x7fdba0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 0x7fdba0057520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdbb4006ef0 tx=0x7fdbb4015000).stop 2020-11-16T22:14:49.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 msgr2=0x7fdbc40aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fdbc000db60 tx=0x7fdbc0008040).stop 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 shutdown_connections 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba0055060 0x7fdba0057520 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbc40b3570 0x7fdbc40aeb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbc4142e70 0x7fdbc41432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 --2- 172.21.15.185:0/3117990808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbc413de10 0x7fdbc4153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.795+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 >> 172.21.15.185:0/3117990808 conn(0x7fdbc4064b90 msgr2=0x7fdbc4067410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.799+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 shutdown_connections 2020-11-16T22:14:49.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.799+0000 7fdbcb1fa700 1 -- 172.21.15.185:0/3117990808 wait complete. 2020-11-16T22:14:49.803 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:14:49.805 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2020-11-16T22:14:49.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.943+0000 7f82efe5e700 1 Processor -- start 2020-11-16T22:14:49.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.943+0000 7f82efe5e700 1 -- start start 2020-11-16T22:14:49.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 0x7f82e81437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f82e80b6b30 0x7f82e8143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e81444d0 con 0x7f82e80b6b30 2020-11-16T22:14:49.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e8144770 con 0x7f82e809d260 2020-11-16T22:14:49.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82efe5e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8144a10 con 0x7f82e80b5b30 2020-11-16T22:14:49.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82ed3f9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 0x7f82e81437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50310/0 (socket says 172.21.15.185:50310) 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 -- 172.21.15.185:0/371170090 learned_addr learned my addr 172.21.15.185:0/371170090 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82ed3f9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 0x7f82e81437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56868/0 (socket says 172.21.15.185:56868) 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82ee3fb700 1 --2- 172.21.15.185:0/371170090 >> v2:172.21.15.96:3300/0 conn(0x7f82e80b6b30 0x7f82e8143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 -- 172.21.15.185:0/371170090 >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 msgr2=0x7f82e81437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:49.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 --2- 172.21.15.185:0/371170090 >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 0x7f82e81437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 -- 172.21.15.185:0/371170090 >> v2:172.21.15.96:3300/0 conn(0x7f82e80b6b30 msgr2=0x7f82e8143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:49.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 --2- 172.21.15.185:0/371170090 >> v2:172.21.15.96:3300/0 conn(0x7f82e80b6b30 0x7f82e8143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 -- 172.21.15.185:0/371170090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8144cb0 con 0x7f82e80b5b30 2020-11-16T22:14:49.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.947+0000 7f82edbfa700 1 --2- 172.21.15.185:0/371170090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f82e8080600 tx=0x7f82e4006eb0).ready entity=mon.2 client_cookie=59ca0bcdd873363f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82ecbf8700 1 -- 172.21.15.185:0/371170090 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4009810 con 0x7f82e80b5b30 2020-11-16T22:14:49.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82ecbf8700 1 -- 172.21.15.185:0/371170090 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4009970 con 0x7f82e80b5b30 2020-11-16T22:14:49.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82ecbf8700 1 -- 172.21.15.185:0/371170090 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4009cf0 con 0x7f82e80b5b30 2020-11-16T22:14:49.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- 172.21.15.185:0/371170090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 msgr2=0x7f82e80b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- 172.21.15.185:0/371170090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f82e8080600 tx=0x7f82e4006eb0).stop 2020-11-16T22:14:49.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- 172.21.15.185:0/371170090 shutdown_connections 2020-11-16T22:14:49.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- 172.21.15.185:0/371170090 >> v2:172.21.15.96:3300/0 conn(0x7f82e80b6b30 0x7f82e8143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- 172.21.15.185:0/371170090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e80b5f10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- 172.21.15.185:0/371170090 >> v2:172.21.15.185:3300/0 conn(0x7f82e809d260 0x7f82e81437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- 172.21.15.185:0/371170090 >> 172.21.15.185:0/371170090 conn(0x7f82e8068330 msgr2=0x7f82e80ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:49.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- 172.21.15.185:0/371170090 shutdown_connections 2020-11-16T22:14:49.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- 172.21.15.185:0/371170090 wait complete. 2020-11-16T22:14:49.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 Processor -- start 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- start start 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80b6b30 0x7f82e80b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e81411e0 0x7f82e81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b0a60 con 0x7f82e80b6b30 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8156ad0 con 0x7f82e81411e0 2020-11-16T22:14:49.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.951+0000 7f82efe5e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b0c80 con 0x7f82e80b5b30 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50316/0 (socket says 172.21.15.185:50316) 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 -- 172.21.15.185:0/1429328342 learned_addr learned my addr 172.21.15.185:0/1429328342 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82ee3fb700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e81411e0 0x7f82e81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82ed3f9700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80b6b30 0x7f82e80b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 -- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e81411e0 msgr2=0x7f82e81571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e81411e0 0x7f82e81571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 -- 172.21.15.185:0/1429328342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80b6b30 msgr2=0x7f82e80b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:49.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80b6b30 0x7f82e80b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:49.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 -- 172.21.15.185:0/1429328342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80b0f20 con 0x7f82e80b5b30 2020-11-16T22:14:49.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82edbfa700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f82e4009e10 tx=0x7f82e4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4011c80 con 0x7f82e80b5b30 2020-11-16T22:14:49.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4011de0 con 0x7f82e80b5b30 2020-11-16T22:14:49.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e8156d00 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e401e670 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e8157690 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f82e4025030 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e805d4e0 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 0x7f82c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82e406f3c0 con 0x7f82e80b5b30 2020-11-16T22:14:49.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82ed3f9700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 0x7f82c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:49.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.955+0000 7f82ed3f9700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 0x7f82c0057520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f82e814a600 tx=0x7f82d8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:49.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:49.959+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f82e40456d0 con 0x7f82e80b5b30 2020-11-16T22:14:50.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.163+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7f82e8157910 con 0x7f82e80b5b30 2020-11-16T22:14:50.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.955+0000 7f82d5ffb700 1 -- 172.21.15.185:0/1429328342 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v57) v1 ==== 139+0+0 (secure 0 0 0) 0x7f82e4023020 con 0x7f82e80b5b30 2020-11-16T22:14:50.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 msgr2=0x7f82c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:50.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 0x7f82c0057520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f82e814a600 tx=0x7f82d8006df0).stop 2020-11-16T22:14:50.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 msgr2=0x7f82e8148360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:50.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f82e4009e10 tx=0x7f82e4015000).stop 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 shutdown_connections 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82c0055060 0x7f82c0057520 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82e81411e0 0x7f82e81571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82e80b5b30 0x7f82e8148360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 --2- 172.21.15.185:0/1429328342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80b6b30 0x7f82e80b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:50.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 >> 172.21.15.185:0/1429328342 conn(0x7f82e8068330 msgr2=0x7f82e8143220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:50.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 shutdown_connections 2020-11-16T22:14:50.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:50.959+0000 7f82efe5e700 1 -- 172.21.15.185:0/1429328342 wait complete. 2020-11-16T22:14:50.963 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:14:51.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 Processor -- start 2020-11-16T22:14:51.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 -- start start 2020-11-16T22:14:51.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc140b7440 0x7fcc140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140ad490 con 0x7fcc14144860 2020-11-16T22:14:51.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140ad730 con 0x7fcc140afc60 2020-11-16T22:14:51.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc1b48c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc140ad9d0 con 0x7fcc140b7440 2020-11-16T22:14:51.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc18a27700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fcc140afc60 0x7fcc1413dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50324/0 (socket says 172.21.15.185:50324) 2020-11-16T22:14:51.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.107+0000 7fcc18a27700 1 -- 172.21.15.185:0/3925919004 learned_addr learned my addr 172.21.15.185:0/3925919004 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:51.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc19228700 1 --2- 172.21.15.185:0/3925919004 >> v2:172.21.15.96:3300/0 conn(0x7fcc140b7440 0x7fcc140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc03fff700 1 -- 172.21.15.185:0/3925919004 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376329268 0 0) 0x7fcc140ad730 con 0x7fcc140afc60 2020-11-16T22:14:51.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc03fff700 1 -- 172.21.15.185:0/3925919004 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc003240 con 0x7fcc140afc60 2020-11-16T22:14:51.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc19228700 1 -- 172.21.15.185:0/3925919004 >> v1:172.21.15.185:6789/0 conn(0x7fcc140afc60 legacy=0x7fcc1413dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc19228700 1 -- 172.21.15.185:0/3925919004 >> v1:172.21.15.96:6789/0 conn(0x7fcc14144860 legacy=0x7fcc140ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc19228700 1 -- 172.21.15.185:0/3925919004 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1413e5b0 con 0x7fcc140b7440 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc19228700 1 --2- 172.21.15.185:0/3925919004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140b7440 0x7fcc140b7820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc1000b9a0 tx=0x7fcc10006df0).ready entity=mon.0 client_cookie=322c42aea6eb48b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc03fff700 1 -- 172.21.15.185:0/3925919004 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100097b0 con 0x7fcc140b7440 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc03fff700 1 -- 172.21.15.185:0/3925919004 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10009910 con 0x7fcc140b7440 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc03fff700 1 -- 172.21.15.185:0/3925919004 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10009c90 con 0x7fcc140b7440 2020-11-16T22:14:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- 172.21.15.185:0/3925919004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140b7440 msgr2=0x7fcc140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/3925919004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140b7440 0x7fcc140b7820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fcc1000b9a0 tx=0x7fcc10006df0).stop 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- 172.21.15.185:0/3925919004 shutdown_connections 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/3925919004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140b7440 0x7fcc140b7820 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- 172.21.15.185:0/3925919004 >> 172.21.15.185:0/3925919004 conn(0x7fcc14067860 msgr2=0x7fcc14067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- 172.21.15.185:0/3925919004 shutdown_connections 2020-11-16T22:14:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- 172.21.15.185:0/3925919004 wait complete. 2020-11-16T22:14:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 Processor -- start 2020-11-16T22:14:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.111+0000 7fcc1b48c700 1 -- start start 2020-11-16T22:14:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140afc60 0x7fcc1414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 0x7fcc140b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 0x7fcc140b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141540f0 con 0x7fcc140afc60 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140b2e20 con 0x7fcc14144860 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140b30c0 con 0x7fcc140b7440 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 0x7fcc140b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc19a29700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 0x7fcc140b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc19a29700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 0x7fcc140b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56894/0 (socket says 172.21.15.185:56894) 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc19a29700 1 -- 172.21.15.185:0/1325782107 learned_addr learned my addr 172.21.15.185:0/1325782107 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc19228700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140afc60 0x7fcc1414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 -- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 msgr2=0x7fcc140b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 0x7fcc140b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 -- 172.21.15.185:0/1325782107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140afc60 msgr2=0x7fcc1414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140afc60 0x7fcc1414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 -- 172.21.15.185:0/1325782107 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140b3360 con 0x7fcc140b7440 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc18a27700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 0x7fcc140b0c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fcc0400c6e0 tx=0x7fcc040079c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040072e0 con 0x7fcc140b7440 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04007440 con 0x7fcc140b7440 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1414ad90 con 0x7fcc140b7440 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04005820 con 0x7fcc140b7440 2020-11-16T22:14:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc14153c80 con 0x7fcc140b7440 2020-11-16T22:14:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc14153f60 con 0x7fcc140b7440 2020-11-16T22:14:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.115+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcc04010070 con 0x7fcc140b7440 2020-11-16T22:14:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.119+0000 7fcc017fa700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 0x7fcbec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.119+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcc0406d300 con 0x7fcc140b7440 2020-11-16T22:14:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.119+0000 7fcc19228700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 0x7fcbec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.119+0000 7fcc19228700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 0x7fcbec057520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcc10006e30 tx=0x7fcc10015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.127+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcc04043560 con 0x7fcc140b7440 2020-11-16T22:14:51.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.279+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fcc140b0940 con 0x7fcc140b7440 2020-11-16T22:14:51.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.287+0000 7fcc017fa700 1 -- 172.21.15.185:0/1325782107 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc140b0940 con 0x7fcc140b7440 2020-11-16T22:14:51.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 msgr2=0x7fcbec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 0x7fcbec057520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcc10006e30 tx=0x7fcc10015000).stop 2020-11-16T22:14:51.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 msgr2=0x7fcc140b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 0x7fcc140b0c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fcc0400c6e0 tx=0x7fcc040079c0).stop 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 shutdown_connections 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcbec055060 0x7fcbec057520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc14144860 0x7fcc140b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc140afc60 0x7fcc1414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 --2- 172.21.15.185:0/1325782107 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc140b7440 0x7fcc140b0c00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 >> 172.21.15.185:0/1325782107 conn(0x7fcc14067860 msgr2=0x7fcc14067c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:51.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 shutdown_connections 2020-11-16T22:14:51.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.291+0000 7fcc1b48c700 1 -- 172.21.15.185:0/1325782107 wait complete. 2020-11-16T22:14:51.293 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:14:51.293 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:14:51.341 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:14:51.373 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:14:51.601 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:14:51.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:14:51.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:14:51.602 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2020-11-16T22:14:51.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:14:51.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:51.685 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:14:51.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-16T22:14:51.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:14:51.709 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2020-11-16T22:14:51.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:14:51.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:14:51.778 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-16T22:14:51.796 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:14:51.796 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:51.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 Processor -- start 2020-11-16T22:14:51.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 -- start start 2020-11-16T22:14:51.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a800acc10 0x7f0a801432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a800abf90 con 0x7f0a800669a0 2020-11-16T22:14:51.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a800ac230 con 0x7f0a80061e60 2020-11-16T22:14:51.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.939+0000 7f0a86b2d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a800ac4d0 con 0x7f0a800acc10 2020-11-16T22:14:51.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8632c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f0a80061e60 0x7f0a801437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50336/0 (socket says 172.21.15.185:50336) 2020-11-16T22:14:51.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8632c700 1 -- 172.21.15.185:0/1818176564 learned_addr learned my addr 172.21.15.185:0/1818176564 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:51.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8532a700 1 --2- 172.21.15.185:0/1818176564 >> v2:172.21.15.96:3300/0 conn(0x7f0a800acc10 0x7f0a801432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a84b29700 1 -- 172.21.15.185:0/1818176564 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053271847 0 0) 0x7f0a800ac230 con 0x7f0a80061e60 2020-11-16T22:14:51.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a84b29700 1 -- 172.21.15.185:0/1818176564 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a68003240 con 0x7f0a80061e60 2020-11-16T22:14:51.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8532a700 1 -- 172.21.15.185:0/1818176564 >> v1:172.21.15.185:6789/0 conn(0x7f0a80061e60 legacy=0x7f0a801437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8532a700 1 -- 172.21.15.185:0/1818176564 >> v1:172.21.15.96:6789/0 conn(0x7f0a800669a0 legacy=0x7f0a80066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8532a700 1 -- 172.21.15.185:0/1818176564 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800ac770 con 0x7f0a800acc10 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a8532a700 1 --2- 172.21.15.185:0/1818176564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a800acc10 0x7f0a801432e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0a7000b9a0 tx=0x7f0a70006df0).ready entity=mon.0 client_cookie=181a4ed2edf11c5e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a84b29700 1 -- 172.21.15.185:0/1818176564 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700097b0 con 0x7f0a800acc10 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a84b29700 1 -- 172.21.15.185:0/1818176564 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70009910 con 0x7f0a800acc10 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a84b29700 1 -- 172.21.15.185:0/1818176564 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70009c90 con 0x7f0a800acc10 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- 172.21.15.185:0/1818176564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a800acc10 msgr2=0x7f0a801432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/1818176564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a800acc10 0x7f0a801432e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0a7000b9a0 tx=0x7f0a70006df0).stop 2020-11-16T22:14:51.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- 172.21.15.185:0/1818176564 shutdown_connections 2020-11-16T22:14:51.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/1818176564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a800acc10 0x7f0a801432e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- 172.21.15.185:0/1818176564 >> 172.21.15.185:0/1818176564 conn(0x7f0a80060130 msgr2=0x7f0a80060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:51.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- 172.21.15.185:0/1818176564 shutdown_connections 2020-11-16T22:14:51.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- 172.21.15.185:0/1818176564 wait complete. 2020-11-16T22:14:51.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 Processor -- start 2020-11-16T22:14:51.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 -- start start 2020-11-16T22:14:51.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 0x7f0a80146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.943+0000 7f0a86b2d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0a80146790 0x7f0a800ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8013f360 con 0x7f0a80061e60 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800aff50 con 0x7f0a80146790 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800b01f0 con 0x7f0a800669a0 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8632c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0a80146790 0x7f0a800ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50350/0 (socket says 172.21.15.185:50350) 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 -- 172.21.15.185:0/4064381740 learned_addr learned my addr 172.21.15.185:0/4064381740 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:51.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a85b2b700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 0x7f0a80146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 -- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0a80146790 msgr2=0x7f0a800ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0a80146790 0x7f0a800ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 -- 172.21.15.185:0/4064381740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 msgr2=0x7f0a80146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 0x7f0a80146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 -- 172.21.15.185:0/4064381740 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800b0490 con 0x7f0a800669a0 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a85b2b700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 0x7f0a80146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:51.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a8532a700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0a7000c010 tx=0x7f0a70015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70011d50 con 0x7f0a800669a0 2020-11-16T22:14:51.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7001e460 con 0x7f0a800669a0 2020-11-16T22:14:51.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7001e7e0 con 0x7f0a800669a0 2020-11-16T22:14:51.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a801406e0 con 0x7f0a800669a0 2020-11-16T22:14:51.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a800ae670 con 0x7f0a800669a0 2020-11-16T22:14:51.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0a70025030 con 0x7f0a800669a0 2020-11-16T22:14:51.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a800afdb0 con 0x7f0a800669a0 2020-11-16T22:14:51.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 0x7f0a58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:51.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0a7006f300 con 0x7f0a800669a0 2020-11-16T22:14:51.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a85b2b700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 0x7f0a58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:51.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.947+0000 7f0a85b2b700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 0x7f0a58057520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f0a80146470 tx=0x7f0a7c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:51.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:51.951+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0a800afdb0 con 0x7f0a800669a0 2020-11-16T22:14:52.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.155+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a800afdb0 con 0x7f0a800669a0 2020-11-16T22:14:52.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.155+0000 7f0a6dffb700 1 -- 172.21.15.185:0/4064381740 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2511 (secure 0 0 0) 0x7f0a800afdb0 con 0x7f0a800669a0 2020-11-16T22:14:52.143 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:52.143 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:11.589550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:52.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 msgr2=0x7f0a58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 0x7f0a58057520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f0a80146470 tx=0x7f0a7c009040).stop 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 msgr2=0x7f0a801401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0a7000c010 tx=0x7f0a70015000).stop 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 shutdown_connections 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a58055060 0x7f0a58057520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0a80146790 0x7f0a800ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a80061e60 0x7f0a80146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 --2- 172.21.15.185:0/4064381740 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0a800669a0 0x7f0a801401f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 >> 172.21.15.185:0/4064381740 conn(0x7f0a80060130 msgr2=0x7f0a80067f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 shutdown_connections 2020-11-16T22:14:52.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.159+0000 7f0a86b2d700 1 -- 172.21.15.185:0/4064381740 wait complete. 2020-11-16T22:14:52.147 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 10 2020-11-16T22:14:52.165 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:52.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.299+0000 7f65cddf1700 1 Processor -- start 2020-11-16T22:14:52.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.299+0000 7f65cddf1700 1 -- start start 2020-11-16T22:14:52.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65c80acc10 0x7f65c81432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f65c8061e60 0x7f65c81437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80abf90 con 0x7f65c80669a0 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80ac230 con 0x7f65c80acc10 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80ac4d0 con 0x7f65c8061e60 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f65c8061e60 0x7f65c81437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f65c8061e60 0x7f65c81437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56916/0 (socket says 172.21.15.185:56916) 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 -- 172.21.15.185:0/3053008773 learned_addr learned my addr 172.21.15.185:0/3053008773 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65c7fff700 1 --2- 172.21.15.185:0/3053008773 >> v2:172.21.15.96:3300/0 conn(0x7f65c80acc10 0x7f65c81432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 -- 172.21.15.185:0/3053008773 >> v2:172.21.15.96:3300/0 conn(0x7f65c80acc10 msgr2=0x7f65c81432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 --2- 172.21.15.185:0/3053008773 >> v2:172.21.15.96:3300/0 conn(0x7f65c80acc10 0x7f65c81432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 -- 172.21.15.185:0/3053008773 >> v1:172.21.15.96:6789/0 conn(0x7f65c80669a0 legacy=0x7f65c8066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 -- 172.21.15.185:0/3053008773 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80ac770 con 0x7f65c8061e60 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cd5f0700 1 --2- 172.21.15.185:0/3053008773 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c81437d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f65c8066500 tx=0x7f65c0009290).ready entity=mon.1 client_cookie=f215a7680ef06684 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65c77fe700 1 -- 172.21.15.185:0/3053008773 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000bb80 con 0x7f65c8061e60 2020-11-16T22:14:52.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65c77fe700 1 -- 172.21.15.185:0/3053008773 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000bce0 con 0x7f65c8061e60 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65c77fe700 1 -- 172.21.15.185:0/3053008773 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0011610 con 0x7f65c8061e60 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- 172.21.15.185:0/3053008773 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 msgr2=0x7f65c81437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 --2- 172.21.15.185:0/3053008773 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c81437d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f65c8066500 tx=0x7f65c0009290).stop 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- 172.21.15.185:0/3053008773 shutdown_connections 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 --2- 172.21.15.185:0/3053008773 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c81437d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 --2- 172.21.15.185:0/3053008773 >> v2:172.21.15.96:3300/0 conn(0x7f65c80acc10 0x7f65c81432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- 172.21.15.185:0/3053008773 >> 172.21.15.185:0/3053008773 conn(0x7f65c8060130 msgr2=0x7f65c8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- 172.21.15.185:0/3053008773 shutdown_connections 2020-11-16T22:14:52.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- 172.21.15.185:0/3053008773 wait complete. 2020-11-16T22:14:52.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 Processor -- start 2020-11-16T22:14:52.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.303+0000 7f65cddf1700 1 -- start start 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f65c80669a0 0x7f65c80ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 0x7f65c813efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8157880 con 0x7f65c80adc30 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81463f0 con 0x7f65c8061e60 2020-11-16T22:14:52.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8146690 con 0x7f65c80669a0 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65c7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f65c80669a0 0x7f65c80ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56920/0 (socket says 172.21.15.185:56920) 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 -- 172.21.15.185:0/2269795275 learned_addr learned my addr 172.21.15.185:0/2269795275 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cd5f0700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 0x7f65c813efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 -- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f65c80669a0 msgr2=0x7f65c80ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f65c80669a0 0x7f65c80ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 -- 172.21.15.185:0/2269795275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 msgr2=0x7f65c813efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 0x7f65c813efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 -- 172.21.15.185:0/2269795275 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8146930 con 0x7f65c8061e60 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cd5f0700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 0x7f65c813efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65ccdef700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f65b800c730 tx=0x7f65b8007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800e560 con 0x7f65c8061e60 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800e6e0 con 0x7f65c8061e60 2020-11-16T22:14:52.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c8146bd0 con 0x7f65c8061e60 2020-11-16T22:14:52.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800eaa0 con 0x7f65c8061e60 2020-11-16T22:14:52.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c813f6b0 con 0x7f65c8061e60 2020-11-16T22:14:52.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.307+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c805d200 con 0x7f65c8061e60 2020-11-16T22:14:52.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.311+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f65b8010030 con 0x7f65c8061e60 2020-11-16T22:14:52.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.311+0000 7f65c4ff9700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 0x7f6598057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.311+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65b806dcc0 con 0x7f65c8061e60 2020-11-16T22:14:52.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.311+0000 7f65c7fff700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 0x7f6598057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.311+0000 7f65c7fff700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 0x7f6598057520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f65c80b0e50 tx=0x7f65b0006f90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.315+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f65b8043f20 con 0x7f65c8061e60 2020-11-16T22:14:52.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.467+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65c80b3930 con 0x7f65c8061e60 2020-11-16T22:14:52.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65c4ff9700 1 -- 172.21.15.185:0/2269795275 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f65b8047540 con 0x7f65c8061e60 2020-11-16T22:14:52.456 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:52.457 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:52.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 msgr2=0x7f6598057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 0x7f6598057520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f65c80b0e50 tx=0x7f65b0006f90).stop 2020-11-16T22:14:52.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 msgr2=0x7f65c8157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f65b800c730 tx=0x7f65b8007c30).stop 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 shutdown_connections 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6598055060 0x7f6598057520 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f65c8061e60 0x7f65c8157390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c80adc30 0x7f65c813efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 --2- 172.21.15.185:0/2269795275 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f65c80669a0 0x7f65c80ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 >> 172.21.15.185:0/2269795275 conn(0x7f65c8060130 msgr2=0x7f65c8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 shutdown_connections 2020-11-16T22:14:52.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.471+0000 7f65cddf1700 1 -- 172.21.15.185:0/2269795275 wait complete. 2020-11-16T22:14:52.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:14:05.836112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:14:06.559613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:14:52.471 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:14:52.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 Processor -- start 2020-11-16T22:14:52.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 -- start start 2020-11-16T22:14:52.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f27600ba800 0x7f27600b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27600b7290 con 0x7f27600b5d70 2020-11-16T22:14:52.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600b4960 con 0x7f27600ba800 2020-11-16T22:14:52.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27665dd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600b4bf0 con 0x7f2760143470 2020-11-16T22:14:52.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f2765ddc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f2765ddc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50374/0 (socket says 172.21.15.185:50374) 2020-11-16T22:14:52.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f2765ddc700 1 -- 172.21.15.185:0/2821332722 learned_addr learned my addr 172.21.15.185:0/2821332722 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f27655db700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f27600b5d70 0x7f27600b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50366/0 (socket says 172.21.15.185:50366) 2020-11-16T22:14:52.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.615+0000 7f2764dda700 1 --2- 172.21.15.185:0/2821332722 >> v2:172.21.15.96:3300/0 conn(0x7f27600ba800 0x7f27600b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2753fff700 1 -- 172.21.15.185:0/2821332722 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952915386 0 0) 0x7f27600b7290 con 0x7f27600b5d70 2020-11-16T22:14:52.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2765ddc700 1 -- 172.21.15.185:0/2821332722 >> v2:172.21.15.96:3300/0 conn(0x7f27600ba800 msgr2=0x7f27600b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2765ddc700 1 --2- 172.21.15.185:0/2821332722 >> v2:172.21.15.96:3300/0 conn(0x7f27600ba800 0x7f27600b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2765ddc700 1 -- 172.21.15.185:0/2821332722 >> v1:172.21.15.185:6789/0 conn(0x7f27600b5d70 legacy=0x7f27600b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2765ddc700 1 -- 172.21.15.185:0/2821332722 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600b74c0 con 0x7f2760143470 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2765ddc700 1 --2- 172.21.15.185:0/2821332722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2758009d50 tx=0x7f2758009290).ready entity=mon.2 client_cookie=e3090125bc34405b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2753fff700 1 -- 172.21.15.185:0/2821332722 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800bbc0 con 0x7f2760143470 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2753fff700 1 -- 172.21.15.185:0/2821332722 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275800bd20 con 0x7f2760143470 2020-11-16T22:14:52.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f2753fff700 1 -- 172.21.15.185:0/2821332722 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758011610 con 0x7f2760143470 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- 172.21.15.185:0/2821332722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 msgr2=0x7f2760143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 --2- 172.21.15.185:0/2821332722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2758009d50 tx=0x7f2758009290).stop 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- 172.21.15.185:0/2821332722 shutdown_connections 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 --2- 172.21.15.185:0/2821332722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2760143470 0x7f2760143920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 --2- 172.21.15.185:0/2821332722 >> v2:172.21.15.96:3300/0 conn(0x7f27600ba800 0x7f27600b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- 172.21.15.185:0/2821332722 >> 172.21.15.185:0/2821332722 conn(0x7f276005ba80 msgr2=0x7f276005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- 172.21.15.185:0/2821332722 shutdown_connections 2020-11-16T22:14:52.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- 172.21.15.185:0/2821332722 wait complete. 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 Processor -- start 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.619+0000 7f27665dd700 1 -- start start 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27600ba800 0x7f27601420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27600afef0 con 0x7f27600ba800 2020-11-16T22:14:52.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760153040 con 0x7f2760143470 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27601570e0 con 0x7f27600b5d70 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27655db700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56936/0 (socket says 172.21.15.185:56936) 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 -- 172.21.15.185:0/294125499 learned_addr learned my addr 172.21.15.185:0/294125499 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27655db700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50376/0 (socket says 172.21.15.185:50376) 2020-11-16T22:14:52.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2764dda700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27600ba800 0x7f27601420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 -- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 msgr2=0x7f276014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 -- 172.21.15.185:0/294125499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27600ba800 msgr2=0x7f27601420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27600ba800 0x7f27601420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 -- 172.21.15.185:0/294125499 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760157340 con 0x7f2760143470 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27655db700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:52.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f2765ddc700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f27580092d0 tx=0x7f2758016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800be90 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2758011e20 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27600af890 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758010cd0 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.623+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276014b0a0 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2758017070 con 0x7f2760143470 2020-11-16T22:14:52.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2760142be0 con 0x7f2760143470 2020-11-16T22:14:52.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27517fa700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 0x7f2738057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f275806ff20 con 0x7f2760143470 2020-11-16T22:14:52.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27655db700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 0x7f2738057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.627+0000 7f27655db700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 0x7f2738057550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f275c0070b0 tx=0x7f275c007960).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.631+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2758046070 con 0x7f2760143470 2020-11-16T22:14:52.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f276005d200 con 0x7f2760143470 2020-11-16T22:14:52.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27517fa700 1 -- 172.21.15.185:0/294125499 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2758015020 con 0x7f2760143470 2020-11-16T22:14:52.765 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 msgr2=0x7f2738057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 0x7f2738057550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f275c0070b0 tx=0x7f275c007960).stop 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 msgr2=0x7f27601425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f27580092d0 tx=0x7f2758016000).stop 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 shutdown_connections 2020-11-16T22:14:52.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2738055090 0x7f2738057550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2760143470 0x7f27601425d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27600b5d70 0x7f276014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 --2- 172.21.15.185:0/294125499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27600ba800 0x7f27601420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 >> 172.21.15.185:0/294125499 conn(0x7f276005ba80 msgr2=0x7f27600b9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 shutdown_connections 2020-11-16T22:14:52.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.779+0000 7f27665dd700 1 -- 172.21.15.185:0/294125499 wait complete. 2020-11-16T22:14:52.779 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:14:52.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 Processor -- start 2020-11-16T22:14:52.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 -- start start 2020-11-16T22:14:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe368063df0 0x7fe3680641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe368061240 0x7fe368062b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680b4360 con 0x7fe368062ff0 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680b4600 con 0x7fe368061240 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36e367700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680b48a0 con 0x7fe368063df0 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe368063df0 0x7fe3680641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe368063df0 0x7fe3680641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56940/0 (socket says 172.21.15.185:56940) 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 -- 172.21.15.185:0/812008330 learned_addr learned my addr 172.21.15.185:0/812008330 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36cb64700 1 --2- 172.21.15.185:0/812008330 >> v2:172.21.15.96:3300/0 conn(0x7fe368061240 0x7fe368062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 -- 172.21.15.185:0/812008330 >> v2:172.21.15.96:3300/0 conn(0x7fe368061240 msgr2=0x7fe368062b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 --2- 172.21.15.185:0/812008330 >> v2:172.21.15.96:3300/0 conn(0x7fe368061240 0x7fe368062b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 -- 172.21.15.185:0/812008330 >> v1:172.21.15.96:6789/0 conn(0x7fe368062ff0 legacy=0x7fe3680b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.923+0000 7fe36d365700 1 -- 172.21.15.185:0/812008330 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680b5f70 con 0x7fe368063df0 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36d365700 1 --2- 172.21.15.185:0/812008330 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368063df0 0x7fe3680641d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fe358006fd0 tx=0x7fe358006eb0).ready entity=mon.1 client_cookie=b115d001d9c1e96a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe367fff700 1 -- 172.21.15.185:0/812008330 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358009890 con 0x7fe368063df0 2020-11-16T22:14:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe367fff700 1 -- 172.21.15.185:0/812008330 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580099f0 con 0x7fe368063df0 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe367fff700 1 -- 172.21.15.185:0/812008330 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358009d70 con 0x7fe368063df0 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- 172.21.15.185:0/812008330 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368063df0 msgr2=0x7fe3680641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- 172.21.15.185:0/812008330 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368063df0 0x7fe3680641d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fe358006fd0 tx=0x7fe358006eb0).stop 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- 172.21.15.185:0/812008330 shutdown_connections 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- 172.21.15.185:0/812008330 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368063df0 0x7fe3680641d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- 172.21.15.185:0/812008330 >> v2:172.21.15.96:3300/0 conn(0x7fe368061240 0x7fe368062b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- 172.21.15.185:0/812008330 >> 172.21.15.185:0/812008330 conn(0x7fe368067940 msgr2=0x7fe368067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:52.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- 172.21.15.185:0/812008330 shutdown_connections 2020-11-16T22:14:52.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- 172.21.15.185:0/812008330 wait complete. 2020-11-16T22:14:52.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 Processor -- start 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- start start 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe368061240 0x7fe36814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe368063df0 0x7fe36814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36814b850 con 0x7fe368061240 2020-11-16T22:14:52.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815b260 con 0x7fe368062ff0 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36e367700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815b500 con 0x7fe368063df0 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56948/0 (socket says 172.21.15.185:56948) 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 -- 172.21.15.185:0/2226241010 learned_addr learned my addr 172.21.15.185:0/2226241010 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36db66700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe368063df0 0x7fe36814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36d365700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe368061240 0x7fe36814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 -- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe368063df0 msgr2=0x7fe36814b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe368063df0 0x7fe36814b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 -- 172.21.15.185:0/2226241010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe368061240 msgr2=0x7fe36814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe368061240 0x7fe36814dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.927+0000 7fe36cb64700 1 -- 172.21.15.185:0/2226241010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36815b7a0 con 0x7fe368062ff0 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36cb64700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe35c00db60 tx=0x7fe35c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c009790 con 0x7fe368062ff0 2020-11-16T22:14:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c0098f0 con 0x7fe368062ff0 2020-11-16T22:14:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36815ba40 con 0x7fe368062ff0 2020-11-16T22:14:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c009c90 con 0x7fe368062ff0 2020-11-16T22:14:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36814e230 con 0x7fe368062ff0 2020-11-16T22:14:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe35c013480 con 0x7fe368062ff0 2020-11-16T22:14:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36805d200 con 0x7fe368062ff0 2020-11-16T22:14:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 0x7fe340057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe35c06e190 con 0x7fe368062ff0 2020-11-16T22:14:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36d365700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 0x7fe340057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.931+0000 7fe36d365700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 0x7fe340057520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe35800b1b0 tx=0x7fe358015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:52.939+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe35c044390 con 0x7fe368062ff0 2020-11-16T22:14:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe368145cf0 con 0x7fe368062ff0 2020-11-16T22:14:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe3657fa700 1 -- 172.21.15.185:0/2226241010 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe35c0479b0 con 0x7fe368062ff0 2020-11-16T22:14:53.077 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 msgr2=0x7fe340057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 0x7fe340057520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe35800b1b0 tx=0x7fe358015000).stop 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 msgr2=0x7fe36814ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe35c00db60 tx=0x7fe35c008040).stop 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 shutdown_connections 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe340055060 0x7fe340057520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe368063df0 0x7fe36814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe368061240 0x7fe36814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 --2- 172.21.15.185:0/2226241010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe368062ff0 0x7fe36814ae70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 >> 172.21.15.185:0/2226241010 conn(0x7fe368067940 msgr2=0x7fe36814ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 shutdown_connections 2020-11-16T22:14:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.091+0000 7fe36e367700 1 -- 172.21.15.185:0/2226241010 wait complete. 2020-11-16T22:14:53.091 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:14:53.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 Processor -- start 2020-11-16T22:14:53.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- start start 2020-11-16T22:14:53.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 0x7f06e80ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f06e80b4470 0x7f06e80b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e80b7290 con 0x7f06e80b5d70 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e80b4bf0 con 0x7f06e80b4470 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80b74c0 con 0x7f06e81475d0 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50402/0 (socket says 172.21.15.185:50402) 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 -- 172.21.15.185:0/1745617091 learned_addr learned my addr 172.21.15.185:0/1745617091 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:53.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e67fc700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.185:3300/0 conn(0x7f06e80b4470 0x7f06e80b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e6ffd700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 0x7f06e80ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 -- 172.21.15.185:0/1745617091 >> v2:172.21.15.185:3300/0 conn(0x7f06e80b4470 msgr2=0x7f06e80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.185:3300/0 conn(0x7f06e80b4470 0x7f06e80b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 -- 172.21.15.185:0/1745617091 >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 msgr2=0x7f06e80ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 0x7f06e80ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 -- 172.21.15.185:0/1745617091 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80b7760 con 0x7f06e81475d0 2020-11-16T22:14:53.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e6ffd700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 0x7f06e80ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:53.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e77fe700 1 --2- 172.21.15.185:0/1745617091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f06e8080bf0 tx=0x7f06e0009290).ready entity=mon.2 client_cookie=a946bcaaa100839c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e5ffb700 1 -- 172.21.15.185:0/1745617091 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000bbc0 con 0x7f06e81475d0 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e5ffb700 1 -- 172.21.15.185:0/1745617091 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e000bd20 con 0x7f06e81475d0 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e5ffb700 1 -- 172.21.15.185:0/1745617091 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0011610 con 0x7f06e81475d0 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- 172.21.15.185:0/1745617091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 msgr2=0x7f06e8147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- 172.21.15.185:0/1745617091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f06e8080bf0 tx=0x7f06e0009290).stop 2020-11-16T22:14:53.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- 172.21.15.185:0/1745617091 shutdown_connections 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- 172.21.15.185:0/1745617091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e81475d0 0x7f06e8147a80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.96:3300/0 conn(0x7f06e80b5d70 0x7f06e80ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 --2- 172.21.15.185:0/1745617091 >> v2:172.21.15.185:3300/0 conn(0x7f06e80b4470 0x7f06e80b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- 172.21.15.185:0/1745617091 >> 172.21.15.185:0/1745617091 conn(0x7f06e805ba80 msgr2=0x7f06e805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- 172.21.15.185:0/1745617091 shutdown_connections 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.235+0000 7f06e7fff700 1 -- 172.21.15.185:0/1745617091 wait complete. 2020-11-16T22:14:53.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 Processor -- start 2020-11-16T22:14:53.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 -- start start 2020-11-16T22:14:53.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 0x7f06e80b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06e81475d0 0x7f06e80af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e8143f10 con 0x7f06e80b5d70 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81441b0 con 0x7f06e81475d0 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e8144450 con 0x7f06e80b4470 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06e81475d0 0x7f06e80af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50406/0 (socket says 172.21.15.185:50406) 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e6ffd700 1 -- 172.21.15.185:0/4167363182 learned_addr learned my addr 172.21.15.185:0/4167363182 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.239+0000 7f06e67fc700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 0x7f06e80b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 -- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06e81475d0 msgr2=0x7f06e80af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06e81475d0 0x7f06e80af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 -- 172.21.15.185:0/4167363182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 msgr2=0x7f06e80b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 0x7f06e80b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 -- 172.21.15.185:0/4167363182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e81446f0 con 0x7f06e80b4470 2020-11-16T22:14:53.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e67fc700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 0x7f06e80b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e6ffd700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f06d800db60 tx=0x7f06d8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8009790 con 0x7f06e80b4470 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d80098f0 con 0x7f06e80b4470 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e80aed50 con 0x7f06e80b4470 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8009cb0 con 0x7f06e80b4470 2020-11-16T22:14:53.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e80baa40 con 0x7f06e80b4470 2020-11-16T22:14:53.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e80aeee0 con 0x7f06e80b4470 2020-11-16T22:14:53.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f06d8013690 con 0x7f06e80b4470 2020-11-16T22:14:53.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.243+0000 7f06ceffd700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 0x7f06c4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.247+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f06d806e560 con 0x7f06e80b4470 2020-11-16T22:14:53.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.247+0000 7f06e67fc700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 0x7f06c4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.247+0000 7f06e67fc700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 0x7f06c4057520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f06e8147030 tx=0x7f06dc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.251+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f06d8044650 con 0x7f06e80b4470 2020-11-16T22:14:53.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.403+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06e805d200 con 0x7f06e80b4470 2020-11-16T22:14:53.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.403+0000 7f06ceffd700 1 -- 172.21.15.185:0/4167363182 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f06d8047c70 con 0x7f06e80b4470 2020-11-16T22:14:53.391 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:53.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 msgr2=0x7f06c4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 0x7f06c4057520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f06e8147030 tx=0x7f06dc008040).stop 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 msgr2=0x7f06e80ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f06d800db60 tx=0x7f06d8008040).stop 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 shutdown_connections 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f06c4055060 0x7f06c4057520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06e81475d0 0x7f06e80af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06e80b4470 0x7f06e80ae7b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 --2- 172.21.15.185:0/4167363182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e80b5d70 0x7f06e80b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 >> 172.21.15.185:0/4167363182 conn(0x7f06e805ba80 msgr2=0x7f06e8140520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 shutdown_connections 2020-11-16T22:14:53.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.407+0000 7f06e7fff700 1 -- 172.21.15.185:0/4167363182 wait complete. 2020-11-16T22:14:53.411 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:14:53.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 Processor -- start 2020-11-16T22:14:53.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- start start 2020-11-16T22:14:53.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7faac00b7440 0x7faac00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00ad490 con 0x7faac00afc60 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac00ad730 con 0x7faac00b7440 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00ad9d0 con 0x7faac0144860 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabf7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7faac00b7440 0x7faac00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50416/0 (socket says 172.21.15.185:50416) 2020-11-16T22:14:53.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 -- 172.21.15.185:0/4177309118 learned_addr learned my addr 172.21.15.185:0/4177309118 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 -- 172.21.15.185:0/4177309118 >> v2:172.21.15.185:3300/0 conn(0x7faac00b7440 msgr2=0x7faac00b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 --2- 172.21.15.185:0/4177309118 >> v2:172.21.15.185:3300/0 conn(0x7faac00b7440 0x7faac00b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 -- 172.21.15.185:0/4177309118 >> v1:172.21.15.96:6789/0 conn(0x7faac00afc60 legacy=0x7faac013dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 -- 172.21.15.185:0/4177309118 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac013e5b0 con 0x7faac0144860 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabffff700 1 --2- 172.21.15.185:0/4177309118 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faab8009d50 tx=0x7faab8009290).ready entity=mon.2 client_cookie=68eea4f873688d36 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabe7fc700 1 -- 172.21.15.185:0/4177309118 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800bbc0 con 0x7faac0144860 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabe7fc700 1 -- 172.21.15.185:0/4177309118 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800bd20 con 0x7faac0144860 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faabe7fc700 1 -- 172.21.15.185:0/4177309118 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8011610 con 0x7faac0144860 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- 172.21.15.185:0/4177309118 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 msgr2=0x7faac00ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 --2- 172.21.15.185:0/4177309118 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faab8009d50 tx=0x7faab8009290).stop 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- 172.21.15.185:0/4177309118 shutdown_connections 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 --2- 172.21.15.185:0/4177309118 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac00ade70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 --2- 172.21.15.185:0/4177309118 >> v2:172.21.15.185:3300/0 conn(0x7faac00b7440 0x7faac00b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.547+0000 7faac61f7700 1 -- 172.21.15.185:0/4177309118 >> 172.21.15.185:0/4177309118 conn(0x7faac0067860 msgr2=0x7faac0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- 172.21.15.185:0/4177309118 shutdown_connections 2020-11-16T22:14:53.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- 172.21.15.185:0/4177309118 wait complete. 2020-11-16T22:14:53.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 Processor -- start 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- start start 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 0x7faac013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac013c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac013cd90 con 0x7faac00b7440 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b2bf0 con 0x7faac00afc60 2020-11-16T22:14:53.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faac61f7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac01438d0 con 0x7faac0144860 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56974/0 (socket says 172.21.15.185:56974) 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 -- 172.21.15.185:0/1605940057 learned_addr learned my addr 172.21.15.185:0/1605940057 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabffff700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac013c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabeffd700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 0x7faac013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 -- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 msgr2=0x7faac013c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac013c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 -- 172.21.15.185:0/1605940057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 msgr2=0x7faac013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 0x7faac013c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 -- 172.21.15.185:0/1605940057 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0143b70 con 0x7faac00afc60 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabeffd700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 0x7faac013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.551+0000 7faabf7fe700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7faab000eba0 tx=0x7faab0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000c9b0 con 0x7faac00afc60 2020-11-16T22:14:53.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab000cb10 con 0x7faac00afc60 2020-11-16T22:14:53.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac014b380 con 0x7faac00afc60 2020-11-16T22:14:53.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000ce90 con 0x7faac00afc60 2020-11-16T22:14:53.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac00b1a40 con 0x7faac00afc60 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac005d4e0 con 0x7faac00afc60 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faab0012600 con 0x7faac00afc60 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 0x7faa98057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faab006e130 con 0x7faac00afc60 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faabeffd700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 0x7faa98057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.555+0000 7faabeffd700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 0x7faa98057520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7faac00b9190 tx=0x7faab4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.563+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faab0044330 con 0x7faac00afc60 2020-11-16T22:14:53.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faac0144cf0 con 0x7faac00afc60 2020-11-16T22:14:53.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faa9f7fe700 1 -- 172.21.15.185:0/1605940057 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7faab0047950 con 0x7faac00afc60 2020-11-16T22:14:53.705 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:14:53.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 msgr2=0x7faa98057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 0x7faa98057520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7faac00b9190 tx=0x7faab4009040).stop 2020-11-16T22:14:53.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 msgr2=0x7faac014ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7faab000eba0 tx=0x7faab0006df0).stop 2020-11-16T22:14:53.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 shutdown_connections 2020-11-16T22:14:53.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa98055060 0x7faa98057520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faac00afc60 0x7faac014ae90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faac0144860 0x7faac013c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 --2- 172.21.15.185:0/1605940057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac00b7440 0x7faac013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 >> 172.21.15.185:0/1605940057 conn(0x7faac0067860 msgr2=0x7faac00b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 shutdown_connections 2020-11-16T22:14:53.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.719+0000 7faac61f7700 1 -- 172.21.15.185:0/1605940057 wait complete. 2020-11-16T22:14:53.720 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:14:53.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 Processor -- start 2020-11-16T22:14:53.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 -- start start 2020-11-16T22:14:53.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f628814cd00 0x7f628814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62880b82e0 con 0x7f6288144b90 2020-11-16T22:14:53.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62880b7650 con 0x7f62880b4720 2020-11-16T22:14:53.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628ebfe700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62880b78f0 con 0x7f628814cd00 2020-11-16T22:14:53.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628dbfc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f62880b4720 0x7f62880b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50420/0 (socket says 172.21.15.185:50420) 2020-11-16T22:14:53.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628dbfc700 1 -- 172.21.15.185:0/1479811837 learned_addr learned my addr 172.21.15.185:0/1479811837 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:53.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628e3fd700 1 --2- 172.21.15.185:0/1479811837 >> v2:172.21.15.96:3300/0 conn(0x7f628814cd00 0x7f628814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628cbfa700 1 -- 172.21.15.185:0/1479811837 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642857275 0 0) 0x7f62880b7650 con 0x7f62880b4720 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628cbfa700 1 -- 172.21.15.185:0/1479811837 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6274003240 con 0x7f62880b4720 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628e3fd700 1 -- 172.21.15.185:0/1479811837 >> v1:172.21.15.185:6789/0 conn(0x7f62880b4720 legacy=0x7f62880b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.851+0000 7f628e3fd700 1 -- 172.21.15.185:0/1479811837 >> v1:172.21.15.96:6789/0 conn(0x7f6288144b90 legacy=0x7f62880b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628e3fd700 1 -- 172.21.15.185:0/1479811837 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880b1ee0 con 0x7f628814cd00 2020-11-16T22:14:53.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628e3fd700 1 --2- 172.21.15.185:0/1479811837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f628814f0f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6280009c90 tx=0x7f62800091d0).ready entity=mon.0 client_cookie=8924ef418b1e05a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628cbfa700 1 -- 172.21.15.185:0/1479811837 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000ba70 con 0x7f628814cd00 2020-11-16T22:14:53.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628cbfa700 1 -- 172.21.15.185:0/1479811837 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000bbd0 con 0x7f628814cd00 2020-11-16T22:14:53.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628cbfa700 1 -- 172.21.15.185:0/1479811837 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280011400 con 0x7f628814cd00 2020-11-16T22:14:53.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- 172.21.15.185:0/1479811837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 msgr2=0x7f628814f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 --2- 172.21.15.185:0/1479811837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f628814f0f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6280009c90 tx=0x7f62800091d0).stop 2020-11-16T22:14:53.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- 172.21.15.185:0/1479811837 shutdown_connections 2020-11-16T22:14:53.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 --2- 172.21.15.185:0/1479811837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f628814f0f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- 172.21.15.185:0/1479811837 >> 172.21.15.185:0/1479811837 conn(0x7f6288067860 msgr2=0x7f6288067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:53.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- 172.21.15.185:0/1479811837 shutdown_connections 2020-11-16T22:14:53.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- 172.21.15.185:0/1479811837 wait complete. 2020-11-16T22:14:53.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 Processor -- start 2020-11-16T22:14:53.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.855+0000 7f628ebfe700 1 -- start start 2020-11-16T22:14:53.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6288144b90 0x7f6288153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f6288153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288153490 con 0x7f628814cd00 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62880ba7a0 con 0x7f62880b4720 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62880ad1f0 con 0x7f6288144b90 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:56988/0 (socket says 172.21.15.185:56988) 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 -- 172.21.15.185:0/2948921010 learned_addr learned my addr 172.21.15.185:0/2948921010 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628d3fb700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6288144b90 0x7f6288153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628e3fd700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f6288153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 -- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6288144b90 msgr2=0x7f6288153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6288144b90 0x7f6288153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 -- 172.21.15.185:0/2948921010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 msgr2=0x7f6288153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f6288153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:53.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 -- 172.21.15.185:0/2948921010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288153680 con 0x7f62880b4720 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628dbfc700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f627800c6e0 tx=0x7f6278007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780072e0 con 0x7f62880b4720 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278007440 con 0x7f62880b4720 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62880ba500 con 0x7f62880b4720 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278005800 con 0x7f62880b4720 2020-11-16T22:14:53.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62880b6750 con 0x7f62880b4720 2020-11-16T22:14:53.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6278010070 con 0x7f62880b4720 2020-11-16T22:14:53.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6288153840 con 0x7f62880b4720 2020-11-16T22:14:53.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.859+0000 7f6285ffb700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 0x7f6260057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:53.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.863+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f627806d490 con 0x7f62880b4720 2020-11-16T22:14:53.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.863+0000 7f628d3fb700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 0x7f6260057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:53.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.863+0000 7f628d3fb700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 0x7f6260057520 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6288127740 tx=0x7f627c007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:53.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:53.867+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6288153840 con 0x7f62880b4720 2020-11-16T22:14:54.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f628805d200 con 0x7f62880b4720 2020-11-16T22:14:54.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f6285ffb700 1 -- 172.21.15.185:0/2948921010 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2511 (secure 0 0 0) 0x7f62780436f0 con 0x7f62880b4720 2020-11-16T22:14:54.053 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:54.054 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:14:07.542849+0000","modified":"2020-11-16T22:14:11.589550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4278,"mds_1":4273,"mds_2":4217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4217":{"gid":4217,"name":"a","rank":2,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/1427562659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1427562659},{"type":"v1","addr":"172.21.15.185:6835","nonce":1427562659}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4273":{"gid":4273,"name":"c","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6833/93135634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":93135634},{"type":"v1","addr":"172.21.15.96:6833","nonce":93135634}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3128886787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3128886787},{"type":"v1","addr":"172.21.15.96:6835","nonce":3128886787}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-16T22:14:54.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 msgr2=0x7f6260057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 0x7f6260057520 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6288127740 tx=0x7f627c007b90).stop 2020-11-16T22:14:54.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 msgr2=0x7f628812c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f627800c6e0 tx=0x7f6278007a40).stop 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 shutdown_connections 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6260055060 0x7f6260057520 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f62880b4720 0x7f628812c280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f628814cd00 0x7f6288153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 --2- 172.21.15.185:0/2948921010 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6288144b90 0x7f6288153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 >> 172.21.15.185:0/2948921010 conn(0x7f6288067860 msgr2=0x7f628812a650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:54.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 shutdown_connections 2020-11-16T22:14:54.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.067+0000 7f628ebfe700 1 -- 172.21.15.185:0/2948921010 wait complete. 2020-11-16T22:14:54.057 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 10 2020-11-16T22:14:54.067 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:14:54.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 Processor -- start 2020-11-16T22:14:54.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 -- start start 2020-11-16T22:14:54.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 0x7f11fc066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11fc0acc10 0x7f11fc147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0abf90 con 0x7f11fc061e60 2020-11-16T22:14:54.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0ac230 con 0x7f11fc0acc10 2020-11-16T22:14:54.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202e86700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0ac4d0 con 0x7f11fc0669a0 2020-11-16T22:14:54.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202685700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f11fc061e60 0x7f11fc147a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50434/0 (socket says 172.21.15.185:50434) 2020-11-16T22:14:54.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 0x7f11fc066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1202685700 1 -- 172.21.15.185:0/2679343570 learned_addr learned my addr 172.21.15.185:0/2679343570 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:54.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201683700 1 --2- 172.21.15.185:0/2679343570 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0acc10 0x7f11fc147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1200e82700 1 -- 172.21.15.185:0/2679343570 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824667417 0 0) 0x7f11fc0abf90 con 0x7f11fc061e60 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 -- 172.21.15.185:0/2679343570 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0acc10 msgr2=0x7f11fc147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 --2- 172.21.15.185:0/2679343570 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0acc10 0x7f11fc147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 -- 172.21.15.185:0/2679343570 >> v1:172.21.15.185:6789/0 conn(0x7f11fc061e60 legacy=0x7f11fc147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 -- 172.21.15.185:0/2679343570 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0ac770 con 0x7f11fc0669a0 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.207+0000 7f1201e84700 1 --2- 172.21.15.185:0/2679343570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 0x7f11fc066d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f11f4009c90 tx=0x7f11f40091d0).ready entity=mon.2 client_cookie=adb1c387ac739b06 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1200e82700 1 -- 172.21.15.185:0/2679343570 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400ba70 con 0x7f11fc0669a0 2020-11-16T22:14:54.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1200e82700 1 -- 172.21.15.185:0/2679343570 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f400bbd0 con 0x7f11fc0669a0 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1200e82700 1 -- 172.21.15.185:0/2679343570 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4011400 con 0x7f11fc0669a0 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- 172.21.15.185:0/2679343570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 msgr2=0x7f11fc066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- 172.21.15.185:0/2679343570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 0x7f11fc066d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f11f4009c90 tx=0x7f11f40091d0).stop 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- 172.21.15.185:0/2679343570 shutdown_connections 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- 172.21.15.185:0/2679343570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0669a0 0x7f11fc066d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- 172.21.15.185:0/2679343570 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0acc10 0x7f11fc147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- 172.21.15.185:0/2679343570 >> 172.21.15.185:0/2679343570 conn(0x7f11fc060130 msgr2=0x7f11fc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:54.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- 172.21.15.185:0/2679343570 shutdown_connections 2020-11-16T22:14:54.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- 172.21.15.185:0/2679343570 wait complete. 2020-11-16T22:14:54.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 Processor -- start 2020-11-16T22:14:54.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- start start 2020-11-16T22:14:54.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc061e60 0x7f11fc13ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc0ad500 0x7f11fc147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1537a0 con 0x7f11fc061e60 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc153a40 con 0x7f11fc0ad500 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.211+0000 7f1202e86700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc153ce0 con 0x7f11fc0adcd0 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1202685700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc0ad500 0x7f11fc147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50444/0 (socket says 172.21.15.185:50444) 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 -- 172.21.15.185:0/2607527468 learned_addr learned my addr 172.21.15.185:0/2607527468 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201e84700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc061e60 0x7f11fc13ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 -- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc0ad500 msgr2=0x7f11fc147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc0ad500 0x7f11fc147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 -- 172.21.15.185:0/2607527468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc061e60 msgr2=0x7f11fc13ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc061e60 0x7f11fc13ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 -- 172.21.15.185:0/2607527468 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc153f80 con 0x7f11fc0adcd0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1201683700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f11e400eb20 tx=0x7f11e4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400b340 con 0x7f11fc0adcd0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e400c820 con 0x7f11fc0adcd0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc13ef30 con 0x7f11fc0adcd0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400cbc0 con 0x7f11fc0adcd0 2020-11-16T22:14:54.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc146440 con 0x7f11fc0adcd0 2020-11-16T22:14:54.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc05d200 con 0x7f11fc0adcd0 2020-11-16T22:14:54.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.215+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f11e401c460 con 0x7f11fc0adcd0 2020-11-16T22:14:54.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.219+0000 7f11f9ffb700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.219+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11e406e040 con 0x7f11fc0adcd0 2020-11-16T22:14:54.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.219+0000 7f1201e84700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.219+0000 7f1201e84700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 0x7f11d8057520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f11fc0b2ec0 tx=0x7f11f4016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.223+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f11e4044240 con 0x7f11fc0adcd0 2020-11-16T22:14:54.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.375+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11fc067270 con 0x7f11fc0adcd0 2020-11-16T22:14:54.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.375+0000 7f11f9ffb700 1 -- 172.21.15.185:0/2607527468 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f11e4047860 con 0x7f11fc0adcd0 2020-11-16T22:14:54.364 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:54.364 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":22,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:08.547687+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:14:05.836112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-16T22:14:06.559613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:14:54.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 msgr2=0x7f11d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 0x7f11d8057520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f11fc0b2ec0 tx=0x7f11f4016000).stop 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 msgr2=0x7f11fc147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f11e400eb20 tx=0x7f11e4006df0).stop 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 shutdown_connections 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d8055060 0x7f11d8057520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc0ad500 0x7f11fc147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc061e60 0x7f11fc13ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 --2- 172.21.15.185:0/2607527468 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0adcd0 0x7f11fc147ad0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 >> 172.21.15.185:0/2607527468 conn(0x7f11fc060130 msgr2=0x7f11fc13df70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:54.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 shutdown_connections 2020-11-16T22:14:54.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.379+0000 7f1202e86700 1 -- 172.21.15.185:0/2607527468 wait complete. 2020-11-16T22:14:54.379 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:14:54.380 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:14:54.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.519+0000 7f691300a700 1 Processor -- start 2020-11-16T22:14:54.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.519+0000 7f691300a700 1 -- start start 2020-11-16T22:14:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f691300a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f690c099d80 0x7f690c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f691300a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f690c0b2980 0x7f690c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f691300a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c068060 con 0x7f690c0b4140 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f691300a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c067260 con 0x7f690c099d80 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f691300a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0b2e30 con 0x7f690c0b2980 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f690c0b2980 0x7f690c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f690c0b2980 0x7f690c1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57012/0 (socket says 172.21.15.185:57012) 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 -- 172.21.15.185:0/652332515 learned_addr learned my addr 172.21.15.185:0/652332515 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f6910da6700 1 --1- 172.21.15.185:0/652332515 >> v1:172.21.15.185:6789/0 conn(0x7f690c0b4140 0x7f690c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50446/0 (socket says 172.21.15.185:50446) 2020-11-16T22:14:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f690bfff700 1 --2- 172.21.15.185:0/652332515 >> v2:172.21.15.96:3300/0 conn(0x7f690c099d80 0x7f690c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 -- 172.21.15.185:0/652332515 >> v2:172.21.15.96:3300/0 conn(0x7f690c099d80 msgr2=0x7f690c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 --2- 172.21.15.185:0/652332515 >> v2:172.21.15.96:3300/0 conn(0x7f690c099d80 0x7f690c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f690b7fe700 1 -- 172.21.15.185:0/652332515 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28703510 0 0) 0x7f690c068060 con 0x7f690c0b4140 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 -- 172.21.15.185:0/652332515 >> v1:172.21.15.185:6789/0 conn(0x7f690c0b4140 legacy=0x7f690c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 -- 172.21.15.185:0/652332515 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c067ce0 con 0x7f690c0b2980 2020-11-16T22:14:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f69115a7700 1 --2- 172.21.15.185:0/652332515 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c1445b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f690c0b5190 tx=0x7f6904006eb0).ready entity=mon.1 client_cookie=b12523b33603ada3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f690b7fe700 1 -- 172.21.15.185:0/652332515 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904009810 con 0x7f690c0b2980 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f690b7fe700 1 -- 172.21.15.185:0/652332515 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904009970 con 0x7f690c0b2980 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.523+0000 7f690b7fe700 1 -- 172.21.15.185:0/652332515 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904009cf0 con 0x7f690c0b2980 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- 172.21.15.185:0/652332515 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 msgr2=0x7f690c1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- 172.21.15.185:0/652332515 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c1445b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f690c0b5190 tx=0x7f6904006eb0).stop 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- 172.21.15.185:0/652332515 shutdown_connections 2020-11-16T22:14:54.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- 172.21.15.185:0/652332515 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c1445b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- 172.21.15.185:0/652332515 >> v2:172.21.15.96:3300/0 conn(0x7f690c099d80 0x7f690c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- 172.21.15.185:0/652332515 >> 172.21.15.185:0/652332515 conn(0x7f690c061240 msgr2=0x7f690c061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- 172.21.15.185:0/652332515 shutdown_connections 2020-11-16T22:14:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- 172.21.15.185:0/652332515 wait complete. 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 Processor -- start 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- start start 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f690c099d80 0x7f690c063de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f690c0b4140 0x7f690c0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0ad7f0 con 0x7f690c099d80 2020-11-16T22:14:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c152d10 con 0x7f690c0b2980 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f691300a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c152fb0 con 0x7f690c0b4140 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f690bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f690bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57016/0 (socket says 172.21.15.185:57016) 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f690bfff700 1 -- 172.21.15.185:0/1328901807 learned_addr learned my addr 172.21.15.185:0/1328901807 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f69115a7700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f690c0b4140 0x7f690c0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.527+0000 7f6910da6700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f690c099d80 0x7f690c063de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 -- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f690c0b4140 msgr2=0x7f690c0ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f690c0b4140 0x7f690c0ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 -- 172.21.15.185:0/1328901807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f690c099d80 msgr2=0x7f690c063de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f690c099d80 0x7f690c063de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 -- 172.21.15.185:0/1328901807 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0ada20 con 0x7f690c0b2980 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f690bfff700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f690000ebd0 tx=0x7f6900006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000c8e0 con 0x7f690c0b2980 2020-11-16T22:14:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690000ca40 con 0x7f690c0b2980 2020-11-16T22:14:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c0642d0 con 0x7f690c0b2980 2020-11-16T22:14:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000cdc0 con 0x7f690c0b2980 2020-11-16T22:14:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c0af9d0 con 0x7f690c0b2980 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6900012600 con 0x7f690c0b2980 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0afbc0 con 0x7f690c0b2980 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 0x7f68e4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f690006e090 con 0x7f690c0b2980 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6910da6700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 0x7f68e4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:54.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.531+0000 7f6910da6700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 0x7f68e4057550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f68fc004b80 tx=0x7f68fc007ce0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:54.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.539+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6900044290 con 0x7f690c0b2980 2020-11-16T22:14:54.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.739+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f690c0adbb0 con 0x7f690c0b2980 2020-11-16T22:14:54.959 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:14:54.974+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:14:54.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.987+0000 7f6908ff9700 1 -- 172.21.15.185:0/1328901807 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f69000478b0 con 0x7f690c0b2980 2020-11-16T22:14:54.975 INFO:tasks.ceph.mds.b.smithi096.stderr: -1836> 2020-11-16T22:14:10.150+0000 7fdcc80a3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:14:54.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 msgr2=0x7f68e4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 0x7f68e4057550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f68fc004b80 tx=0x7f68fc007ce0).stop 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 msgr2=0x7f690c0adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f690000ebd0 tx=0x7f6900006df0).stop 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 shutdown_connections 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f68e4055090 0x7f68e4057550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f690c0b4140 0x7f690c0ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f690c099d80 0x7f690c063de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 --2- 172.21.15.185:0/1328901807 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f690c0b2980 0x7f690c0adec0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:54.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 >> 172.21.15.185:0/1328901807 conn(0x7f690c061240 msgr2=0x7f690c09a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:54.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 shutdown_connections 2020-11-16T22:14:54.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:54.991+0000 7f691300a700 1 -- 172.21.15.185:0/1328901807 wait complete. 2020-11-16T22:14:54.979 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:14:54.994 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:14:55.000 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:14:55.000 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:14:55.001 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:14:55.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.127+0000 7f9b5cd4a700 1 Processor -- start 2020-11-16T22:14:55.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.127+0000 7f9b5cd4a700 1 -- start start 2020-11-16T22:14:55.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b580675e0 0x7f9b580679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:55.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b58142f70 con 0x7f9b580b45a0 2020-11-16T22:14:55.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b58068680 con 0x7f9b580b2ec0 2020-11-16T22:14:55.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b580b33d0 con 0x7f9b580675e0 2020-11-16T22:14:55.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b55d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f9b580b2ec0 0x7f9b5813bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50472/0 (socket says 172.21.15.185:50472) 2020-11-16T22:14:55.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b55d9b700 1 -- 172.21.15.185:0/4225153551 learned_addr learned my addr 172.21.15.185:0/4225153551 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:55.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5659c700 1 --2- 172.21.15.185:0/4225153551 >> v2:172.21.15.96:3300/0 conn(0x7f9b580675e0 0x7f9b580679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:55.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5559a700 1 -- 172.21.15.185:0/4225153551 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648305873 0 0) 0x7f9b58068680 con 0x7f9b580b2ec0 2020-11-16T22:14:55.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5559a700 1 -- 172.21.15.185:0/4225153551 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b3c003240 con 0x7f9b580b2ec0 2020-11-16T22:14:55.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5659c700 1 -- 172.21.15.185:0/4225153551 >> v1:172.21.15.185:6789/0 conn(0x7f9b580b2ec0 legacy=0x7f9b5813bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5659c700 1 -- 172.21.15.185:0/4225153551 >> v1:172.21.15.96:6789/0 conn(0x7f9b580b45a0 legacy=0x7f9b58145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5659c700 1 -- 172.21.15.185:0/4225153551 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580b3670 con 0x7f9b580675e0 2020-11-16T22:14:55.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5659c700 1 --2- 172.21.15.185:0/4225153551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b580679c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9b4800b9a0 tx=0x7f9b48006df0).ready entity=mon.0 client_cookie=a27a39cc02eaccd2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:55.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5559a700 1 -- 172.21.15.185:0/4225153551 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480097b0 con 0x7f9b580675e0 2020-11-16T22:14:55.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5559a700 1 -- 172.21.15.185:0/4225153551 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48009910 con 0x7f9b580675e0 2020-11-16T22:14:55.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5559a700 1 -- 172.21.15.185:0/4225153551 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48009c90 con 0x7f9b580675e0 2020-11-16T22:14:55.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/4225153551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 msgr2=0x7f9b580679c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/4225153551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b580679c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f9b4800b9a0 tx=0x7f9b48006df0).stop 2020-11-16T22:14:55.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/4225153551 shutdown_connections 2020-11-16T22:14:55.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/4225153551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b580679c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/4225153551 >> 172.21.15.185:0/4225153551 conn(0x7f9b58061240 msgr2=0x7f9b58061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:55.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/4225153551 shutdown_connections 2020-11-16T22:14:55.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.131+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/4225153551 wait complete. 2020-11-16T22:14:55.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 Processor -- start 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- start start 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b5813d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 0x7f9b5813b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580aed70 con 0x7f9b580675e0 2020-11-16T22:14:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b581491e0 con 0x7f9b580b45a0 2020-11-16T22:14:55.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58149480 con 0x7f9b580b2ec0 2020-11-16T22:14:55.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:55.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b55d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 0x7f9b5813b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:55.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57044/0 (socket says 172.21.15.185:57044) 2020-11-16T22:14:55.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 -- 172.21.15.185:0/3187318940 learned_addr learned my addr 172.21.15.185:0/3187318940 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5659c700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b5813d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 -- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 msgr2=0x7f9b5813b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 0x7f9b5813b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 -- 172.21.15.185:0/3187318940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 msgr2=0x7f9b5813d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b5813d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 -- 172.21.15.185:0/3187318940 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58149720 con 0x7f9b580b45a0 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b55d9b700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 0x7f9b5813b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b56d9d700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9b5000ca60 tx=0x7f9b50007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000a730 con 0x7f9b580b45a0 2020-11-16T22:14:55.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5000a890 con 0x7f9b580b45a0 2020-11-16T22:14:55.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5813bac0 con 0x7f9b580b45a0 2020-11-16T22:14:55.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000ac10 con 0x7f9b580b45a0 2020-11-16T22:14:55.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.135+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b58147bb0 con 0x7f9b580b45a0 2020-11-16T22:14:55.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5805d4e0 con 0x7f9b580b45a0 2020-11-16T22:14:55.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9b50019030 con 0x7f9b580b45a0 2020-11-16T22:14:55.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b467fc700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 0x7f9b2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:55.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f9b5006d420 con 0x7f9b580b45a0 2020-11-16T22:14:55.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b5659c700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 0x7f9b2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:55.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.139+0000 7f9b5659c700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 0x7f9b2c057520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9b48006e30 tx=0x7f9b48015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:55.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.143+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9b50043630 con 0x7f9b580b45a0 2020-11-16T22:14:55.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.339+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9b58062880 con 0x7f9b580b45a0 2020-11-16T22:14:55.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.987+0000 7f9b467fc700 1 -- 172.21.15.185:0/3187318940 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b50046c50 con 0x7f9b580b45a0 2020-11-16T22:14:55.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 msgr2=0x7f9b2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 0x7f9b2c057520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9b48006e30 tx=0x7f9b48015000).stop 2020-11-16T22:14:55.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 msgr2=0x7f9b5813b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:55.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9b5000ca60 tx=0x7f9b50007eb0).stop 2020-11-16T22:14:55.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 shutdown_connections 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9b2c055060 0x7f9b2c057520 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9b580b45a0 0x7f9b5813b5d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b580675e0 0x7f9b5813d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 --2- 172.21.15.185:0/3187318940 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9b580b2ec0 0x7f9b5813b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 >> 172.21.15.185:0/3187318940 conn(0x7f9b58061240 msgr2=0x7f9b58063290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 shutdown_connections 2020-11-16T22:14:55.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:55.991+0000 7f9b5cd4a700 1 -- 172.21.15.185:0/3187318940 wait complete. 2020-11-16T22:14:55.997 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:14:56.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 Processor -- start 2020-11-16T22:14:56.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 -- start start 2020-11-16T22:14:56.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2d080b1d30 0x7f2d080b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2d080ad110 0x7f2d081477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d08061fe0 con 0x7f2d080b3090 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b2b60 con 0x7f2d080ad110 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d0cb26700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0814c060 con 0x7f2d080b1d30 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d06ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2d080b1d30 0x7f2d080b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d077fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2d080b3090 0x7f2d0814b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50490/0 (socket says 172.21.15.185:50490) 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d077fe700 1 -- 172.21.15.185:0/2900218922 learned_addr learned my addr 172.21.15.185:0/2900218922 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.187+0000 7f2d07fff700 1 --2- 172.21.15.185:0/2900218922 >> v2:172.21.15.96:3300/0 conn(0x7f2d080ad110 0x7f2d081477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d06ffd700 1 -- 172.21.15.185:0/2900218922 >> v2:172.21.15.96:3300/0 conn(0x7f2d080ad110 msgr2=0x7f2d081477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d06ffd700 1 --2- 172.21.15.185:0/2900218922 >> v2:172.21.15.96:3300/0 conn(0x7f2d080ad110 0x7f2d081477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d06ffd700 1 -- 172.21.15.185:0/2900218922 >> v1:172.21.15.185:6789/0 conn(0x7f2d080b3090 legacy=0x7f2d0814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d067fc700 1 -- 172.21.15.185:0/2900218922 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319918375 0 0) 0x7f2d08061fe0 con 0x7f2d080b3090 2020-11-16T22:14:56.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d06ffd700 1 -- 172.21.15.185:0/2900218922 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b0fe0 con 0x7f2d080b1d30 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d06ffd700 1 --2- 172.21.15.185:0/2900218922 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080b1d30 0x7f2d080b1720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f2d0814c1a0 tx=0x7f2cf0006df0).ready entity=mon.1 client_cookie=28d99c94aa07eda0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d067fc700 1 -- 172.21.15.185:0/2900218922 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00097b0 con 0x7f2d080b1d30 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d067fc700 1 -- 172.21.15.185:0/2900218922 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0009910 con 0x7f2d080b1d30 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d067fc700 1 -- 172.21.15.185:0/2900218922 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0009c90 con 0x7f2d080b1d30 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- 172.21.15.185:0/2900218922 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080b1d30 msgr2=0x7f2d080b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/2900218922 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080b1d30 0x7f2d080b1720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f2d0814c1a0 tx=0x7f2cf0006df0).stop 2020-11-16T22:14:56.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- 172.21.15.185:0/2900218922 shutdown_connections 2020-11-16T22:14:56.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/2900218922 >> v2:172.21.15.96:3300/0 conn(0x7f2d080ad110 0x7f2d081477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:56.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/2900218922 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080b1d30 0x7f2d080b1720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:56.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- 172.21.15.185:0/2900218922 >> 172.21.15.185:0/2900218922 conn(0x7f2d08060130 msgr2=0x7f2d08060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:56.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- 172.21.15.185:0/2900218922 shutdown_connections 2020-11-16T22:14:56.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- 172.21.15.185:0/2900218922 wait complete. 2020-11-16T22:14:56.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 Processor -- start 2020-11-16T22:14:56.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.191+0000 7f2d0cb26700 1 -- start start 2020-11-16T22:14:56.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d080b1d30 0x7f2d08112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 0x7f2d081106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08113140 con 0x7f2d080b3090 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b0230 con 0x7f2d080ad110 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08158610 con 0x7f2d080b1d30 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57056/0 (socket says 172.21.15.185:57056) 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 -- 172.21.15.185:0/3087842512 learned_addr learned my addr 172.21.15.185:0/3087842512 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d06ffd700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d080b1d30 0x7f2d08112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d07fff700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 0x7f2d081106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 -- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d080b1d30 msgr2=0x7f2d08112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d080b1d30 0x7f2d08112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 -- 172.21.15.185:0/3087842512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 msgr2=0x7f2d081106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:56.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 0x7f2d081106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 -- 172.21.15.185:0/3087842512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d081588b0 con 0x7f2d080ad110 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d07fff700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 0x7f2d081106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d077fe700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f2cf800cac0 tx=0x7f2cf8007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8005b80 con 0x7f2d080ad110 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d08158310 con 0x7f2d080ad110 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080af0d0 con 0x7f2d080ad110 2020-11-16T22:14:56.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005ce0 con 0x7f2d080ad110 2020-11-16T22:14:56.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8007320 con 0x7f2d080ad110 2020-11-16T22:14:56.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d080b11c0 con 0x7f2d080ad110 2020-11-16T22:14:56.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.195+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2cf8010030 con 0x7f2d080ad110 2020-11-16T22:14:56.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.199+0000 7f2cff7fe700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 0x7f2ce4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:56.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.199+0000 7f2d06ffd700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 0x7f2ce4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:56.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.199+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f2cf806dff0 con 0x7f2d080ad110 2020-11-16T22:14:56.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.199+0000 7f2d06ffd700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 0x7f2ce4057520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2d0814c1a0 tx=0x7f2cf0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:56.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.203+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2cf8044150 con 0x7f2d080ad110 2020-11-16T22:14:56.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:56.351+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d0805d200 con 0x7f2d080ad110 2020-11-16T22:14:57.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.055+0000 7f2cff7fe700 1 -- 172.21.15.185:0/3087842512 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2cf8047770 con 0x7f2d080ad110 2020-11-16T22:14:57.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 msgr2=0x7f2ce4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 0x7f2ce4057520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2d0814c1a0 tx=0x7f2cf0015000).stop 2020-11-16T22:14:57.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 msgr2=0x7f2d08112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f2cf800cac0 tx=0x7f2cf8007e40).stop 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 shutdown_connections 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ce4055060 0x7f2ce4057520 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d080ad110 0x7f2d08112760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d080b3090 0x7f2d081106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 --2- 172.21.15.185:0/3087842512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d080b1d30 0x7f2d08112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 >> 172.21.15.185:0/3087842512 conn(0x7f2d08060130 msgr2=0x7f2d0814ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:57.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 shutdown_connections 2020-11-16T22:14:57.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.059+0000 7f2d0cb26700 1 -- 172.21.15.185:0/3087842512 wait complete. 2020-11-16T22:14:57.048 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:14:57.059 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:14:57.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.199+0000 7fa896fbb700 1 Processor -- start 2020-11-16T22:14:57.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.199+0000 7fa896fbb700 1 -- start start 2020-11-16T22:14:57.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa8900b1d30 0x7fa8900b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:57.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa890061fe0 con 0x7fa8900ad110 2020-11-16T22:14:57.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8900b0fe0 con 0x7fa8900b3090 2020-11-16T22:14:57.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8900b2b60 con 0x7fa8900b1d30 2020-11-16T22:14:57.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa895fb9700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa8900b3090 0x7fa89014b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50502/0 (socket says 172.21.15.185:50502) 2020-11-16T22:14:57.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa895fb9700 1 -- 172.21.15.185:0/1915993151 learned_addr learned my addr 172.21.15.185:0/1915993151 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:57.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa8957b8700 1 --2- 172.21.15.185:0/1915993151 >> v2:172.21.15.96:3300/0 conn(0x7fa8900b1d30 0x7fa8900b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:57.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866458719 0 0) 0x7fa8900b0fe0 con 0x7fa8900b3090 2020-11-16T22:14:57.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa870003240 con 0x7fa8900b3090 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1870321944 0 0) 0x7fa870003240 con 0x7fa8900b3090 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa8957b8700 1 -- 172.21.15.185:0/1915993151 >> v1:172.21.15.185:6789/0 conn(0x7fa8900b3090 legacy=0x7fa89014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa8957b8700 1 -- 172.21.15.185:0/1915993151 >> v1:172.21.15.96:6789/0 conn(0x7fa8900ad110 legacy=0x7fa8901477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa8957b8700 1 -- 172.21.15.185:0/1915993151 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900b1280 con 0x7fa8900b1d30 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa8957b8700 1 --2- 172.21.15.185:0/1915993151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa8900b1720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa880009c10 tx=0x7fa8800091d0).ready entity=mon.0 client_cookie=700a187c8d7484a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880014e0 con 0x7fa8900b1d30 2020-11-16T22:14:57.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88000b790 con 0x7fa8900b1d30 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa894fb7700 1 -- 172.21.15.185:0/1915993151 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000bb10 con 0x7fa8900b1d30 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- 172.21.15.185:0/1915993151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 msgr2=0x7fa8900b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1915993151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa8900b1720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa880009c10 tx=0x7fa8800091d0).stop 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- 172.21.15.185:0/1915993151 shutdown_connections 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1915993151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa8900b1720 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- 172.21.15.185:0/1915993151 >> 172.21.15.185:0/1915993151 conn(0x7fa890060130 msgr2=0x7fa890060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:57.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- 172.21.15.185:0/1915993151 shutdown_connections 2020-11-16T22:14:57.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.203+0000 7fa896fbb700 1 -- 172.21.15.185:0/1915993151 wait complete. 2020-11-16T22:14:57.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 Processor -- start 2020-11-16T22:14:57.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 -- start start 2020-11-16T22:14:57.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa890157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa8900b3090 0x7fa890158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890112380 con 0x7fa8900b1d30 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890111080 con 0x7fa8900b3090 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa890111320 con 0x7fa8900ad110 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:57.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50514/0 (socket says 172.21.15.185:50514) 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 -- 172.21.15.185:0/1398441393 learned_addr learned my addr 172.21.15.185:0/1398441393 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa8967ba700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa8900b3090 0x7fa890158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa8957b8700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa890157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 -- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa8900b3090 msgr2=0x7fa890158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa8900b3090 0x7fa890158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 -- 172.21.15.185:0/1398441393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 msgr2=0x7fa890157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:57.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa890157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:57.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 -- 172.21.15.185:0/1398441393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89011db50 con 0x7fa8900ad110 2020-11-16T22:14:57.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa895fb9700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa88800cae0 tx=0x7fa888001280).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:57.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888007170 con 0x7fa8900ad110 2020-11-16T22:14:57.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8880072f0 con 0x7fa8900ad110 2020-11-16T22:14:57.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.207+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa890157840 con 0x7fa8900ad110 2020-11-16T22:14:57.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800a980 con 0x7fa8900ad110 2020-11-16T22:14:57.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa890110b70 con 0x7fa8900ad110 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa888010070 con 0x7fa8900ad110 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8901121a0 con 0x7fa8900ad110 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa88dffb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 0x7fa86c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5893+0+0 (secure 0 0 0) 0x7fa88806d970 con 0x7fa8900ad110 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa8957b8700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 0x7fa86c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:57.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.211+0000 7fa8957b8700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 0x7fa86c057520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fa88000a010 tx=0x7fa880016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:57.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.215+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa888043c80 con 0x7fa8900ad110 2020-11-16T22:14:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:57.375+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa89005d200 con 0x7fa8900ad110 2020-11-16T22:14:58.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa88dffb700 1 -- 172.21.15.185:0/1398441393 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa8880472a0 con 0x7fa8900ad110 2020-11-16T22:14:58.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 msgr2=0x7fa86c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 0x7fa86c057520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fa88000a010 tx=0x7fa880016000).stop 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 msgr2=0x7fa8900af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fa88800cae0 tx=0x7fa888001280).stop 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 shutdown_connections 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055060 0x7fa86c057520 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa8900b3090 0x7fa890158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa8900ad110 0x7fa8900af690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 --2- 172.21.15.185:0/1398441393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8900b1d30 0x7fa890157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 >> 172.21.15.185:0/1398441393 conn(0x7fa890060130 msgr2=0x7fa890060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 shutdown_connections 2020-11-16T22:14:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.063+0000 7fa896fbb700 1 -- 172.21.15.185:0/1398441393 wait complete. 2020-11-16T22:14:58.054 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:14:58.066 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:14:58.067 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:14:58.067 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:14:58.067 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:14:58.067 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:14:58.068 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:14:58.068 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:14:58.068 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:14:58.068 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:14:58.068 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:14:58.069 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:14:58.069 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:14:58.079 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:14:58.079 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:14:58.081 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:14:58.082 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:14:58.083 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:14:58.084 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:14:58.121 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:14:58.121 DEBUG:teuthology.parallel:result is None 2020-11-16T22:14:58.128 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:14:58.128 DEBUG:teuthology.parallel:result is None 2020-11-16T22:14:58.129 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:14:58.130 DEBUG:teuthology.parallel:result is None 2020-11-16T22:14:58.130 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:14:58.197 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:14:58.203 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:14:58.206 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:14:58.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 Processor -- start 2020-11-16T22:14:58.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 -- start start 2020-11-16T22:14:58.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f5878062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:58.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5878067930 con 0x7f58781446c0 2020-11-16T22:14:58.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58780632e0 con 0x7f5878062900 2020-11-16T22:14:58.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587fa8a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5878066ae0 con 0x7f5878067de0 2020-11-16T22:14:58.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587d826700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5878062900 0x7f5878067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50528/0 (socket says 172.21.15.185:50528) 2020-11-16T22:14:58.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587e027700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f5878062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:58.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587d826700 1 -- 172.21.15.185:0/820028007 learned_addr learned my addr 172.21.15.185:0/820028007 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:58.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071045172 0 0) 0x7f58780632e0 con 0x7f5878062900 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5860003240 con 0x7f5878062900 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587e027700 1 -- 172.21.15.185:0/820028007 >> v1:172.21.15.185:6789/0 conn(0x7f5878062900 legacy=0x7f5878067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587e027700 1 -- 172.21.15.185:0/820028007 >> v1:172.21.15.96:6789/0 conn(0x7f58781446c0 legacy=0x7f58781408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48794281 0 0) 0x7f5878067930 con 0x7f58781446c0 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587e027700 1 -- 172.21.15.185:0/820028007 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878066d80 con 0x7f5878067de0 2020-11-16T22:14:58.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.275+0000 7f587e027700 1 --2- 172.21.15.185:0/820028007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f5878062ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f587400b9b0 tx=0x7f5874006de0).ready entity=mon.2 client_cookie=40299e97d833fc3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874006030 con 0x7f5878067de0 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874006190 con 0x7f5878067de0 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587c824700 1 -- 172.21.15.185:0/820028007 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874006510 con 0x7f5878067de0 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- 172.21.15.185:0/820028007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 msgr2=0x7f5878062ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 --2- 172.21.15.185:0/820028007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f5878062ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f587400b9b0 tx=0x7f5874006de0).stop 2020-11-16T22:14:58.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- 172.21.15.185:0/820028007 shutdown_connections 2020-11-16T22:14:58.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 --2- 172.21.15.185:0/820028007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f5878062ce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- 172.21.15.185:0/820028007 >> 172.21.15.185:0/820028007 conn(0x7f58780b4410 msgr2=0x7f5878060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:58.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- 172.21.15.185:0/820028007 shutdown_connections 2020-11-16T22:14:58.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- 172.21.15.185:0/820028007 wait complete. 2020-11-16T22:14:58.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 Processor -- start 2020-11-16T22:14:58.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.279+0000 7f587fa8a700 1 -- start start 2020-11-16T22:14:58.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5878062900 0x7f58780ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:58.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 0x7f58780afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878154010 con 0x7f5878062900 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58780ae6c0 con 0x7f58781446c0 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58780ae960 con 0x7f5878067de0 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 0x7f58780afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d025700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:58.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d025700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50544/0 (socket says 172.21.15.185:50544) 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d025700 1 -- 172.21.15.185:0/1417681415 learned_addr learned my addr 172.21.15.185:0/1417681415 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d826700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5878062900 0x7f58780ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 -- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 msgr2=0x7f58780b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 -- 172.21.15.185:0/1417681415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5878062900 msgr2=0x7f58780ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:58.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5878062900 0x7f58780ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 -- 172.21.15.185:0/1417681415 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58781541f0 con 0x7f58781446c0 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d025700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587e027700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 0x7f58780afed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f587400b9b0 tx=0x7f5874015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874021070 con 0x7f58781446c0 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874005e60 con 0x7f58781446c0 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58780b0eb0 con 0x7f58781446c0 2020-11-16T22:14:58.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874006030 con 0x7f58781446c0 2020-11-16T22:14:58.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58780b0560 con 0x7f58781446c0 2020-11-16T22:14:58.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878064540 con 0x7f58781446c0 2020-11-16T22:14:58.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f587400f070 con 0x7f58781446c0 2020-11-16T22:14:58.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 0x7f5850057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:58.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f587406f4f0 con 0x7f58781446c0 2020-11-16T22:14:58.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.283+0000 7f587d826700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 0x7f5850057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:58.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.287+0000 7f587d826700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 0x7f5850057520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f58780abfa0 tx=0x7f5870009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:58.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.291+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5878064540 con 0x7f58781446c0 2020-11-16T22:14:58.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:58.459+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5878064540 con 0x7f58781446c0 2020-11-16T22:14:59.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f5865ffb700 1 -- 172.21.15.185:0/1417681415 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5878064540 con 0x7f58781446c0 2020-11-16T22:14:59.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 msgr2=0x7f5850057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 0x7f5850057520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f58780abfa0 tx=0x7f5870009040).stop 2020-11-16T22:14:59.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 msgr2=0x7f58780afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 0x7f58780afed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f587400b9b0 tx=0x7f5874015000).stop 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 shutdown_connections 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5850055060 0x7f5850057520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f58781446c0 0x7f58780afed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5878062900 0x7f58780ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 --2- 172.21.15.185:0/1417681415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5878067de0 0x7f58780b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 >> 172.21.15.185:0/1417681415 conn(0x7f58780b4410 msgr2=0x7f5878060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:59.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 shutdown_connections 2020-11-16T22:14:59.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.243+0000 7f587fa8a700 1 -- 172.21.15.185:0/1417681415 wait complete. 2020-11-16T22:14:59.233 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:14:59.246 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:14:59.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.371+0000 7f3964cab700 1 Processor -- start 2020-11-16T22:14:59.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.371+0000 7f3964cab700 1 -- start start 2020-11-16T22:14:59.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f3960144a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600b7cb0 con 0x7f3960143a30 2020-11-16T22:14:59.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3960143e60 con 0x7f39600b7570 2020-11-16T22:14:59.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b7a10 con 0x7f39600602d0 2020-11-16T22:14:59.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f3960144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3956ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f39600b7570 0x7f3960143320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50544/0 (socket says 172.21.15.185:50544) 2020-11-16T22:14:59.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3956ffd700 1 -- 172.21.15.185:0/1339732480 learned_addr learned my addr 172.21.15.185:0/1339732480 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:14:59.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395effd700 1 -- 172.21.15.185:0/1339732480 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346429101 0 0) 0x7f3960143e60 con 0x7f39600b7570 2020-11-16T22:14:59.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395effd700 1 -- 172.21.15.185:0/1339732480 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3940003240 con 0x7f39600b7570 2020-11-16T22:14:59.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395f7fe700 1 -- 172.21.15.185:0/1339732480 >> v1:172.21.15.185:6789/0 conn(0x7f39600b7570 legacy=0x7f3960143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395f7fe700 1 -- 172.21.15.185:0/1339732480 >> v1:172.21.15.96:6789/0 conn(0x7f3960143a30 legacy=0x7f3960067bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395f7fe700 1 -- 172.21.15.185:0/1339732480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960067470 con 0x7f39600602d0 2020-11-16T22:14:59.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395f7fe700 1 --2- 172.21.15.185:0/1339732480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f3960144a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f395000b9d0 tx=0x7f3950006df0).ready entity=mon.2 client_cookie=90eacde80b3f9d41 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395effd700 1 -- 172.21.15.185:0/1339732480 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39500097b0 con 0x7f39600602d0 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395effd700 1 -- 172.21.15.185:0/1339732480 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3950009910 con 0x7f39600602d0 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f395effd700 1 -- 172.21.15.185:0/1339732480 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950009c90 con 0x7f39600602d0 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- 172.21.15.185:0/1339732480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 msgr2=0x7f3960144a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 --2- 172.21.15.185:0/1339732480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f3960144a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f395000b9d0 tx=0x7f3950006df0).stop 2020-11-16T22:14:59.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- 172.21.15.185:0/1339732480 shutdown_connections 2020-11-16T22:14:59.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 --2- 172.21.15.185:0/1339732480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f3960144a10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- 172.21.15.185:0/1339732480 >> 172.21.15.185:0/1339732480 conn(0x7f396005ba80 msgr2=0x7f396005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:59.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- 172.21.15.185:0/1339732480 shutdown_connections 2020-11-16T22:14:59.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.375+0000 7f3964cab700 1 -- 172.21.15.185:0/1339732480 wait complete. 2020-11-16T22:14:59.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 Processor -- start 2020-11-16T22:14:59.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- start start 2020-11-16T22:14:59.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39600b7570 0x7f39600afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3960143a30 0x7f39600b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601409e0 con 0x7f39600b7570 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b0a80 con 0x7f3960143a30 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b07e0 con 0x7f39600602d0 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3960143a30 0x7f39600b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50554/0 (socket says 172.21.15.185:50554) 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 -- 172.21.15.185:0/3638568018 learned_addr learned my addr 172.21.15.185:0/3638568018 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3956ffd700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39600b7570 0x7f39600afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 -- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3960143a30 msgr2=0x7f39600b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3960143a30 0x7f39600b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 -- 172.21.15.185:0/3638568018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39600b7570 msgr2=0x7f39600afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39600b7570 0x7f39600afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 -- 172.21.15.185:0/3638568018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960140740 con 0x7f39600602d0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f395f7fe700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f39500098e0 tx=0x7f3950015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950011d50 con 0x7f39600602d0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39600b0c80 con 0x7f39600602d0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395001e460 con 0x7f39600602d0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395001e820 con 0x7f39600602d0 2020-11-16T22:14:59.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.379+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3960145570 con 0x7f39600602d0 2020-11-16T22:14:59.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396005d200 con 0x7f39600602d0 2020-11-16T22:14:59.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f395001a040 con 0x7f39600602d0 2020-11-16T22:14:59.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3957fff700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3956ffd700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f395006ed80 con 0x7f39600602d0 2020-11-16T22:14:59.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.383+0000 7f3956ffd700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f39601478c0 tx=0x7f3948009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.391+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f39500452b0 con 0x7f39600602d0 2020-11-16T22:14:59.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f396005c720 con 0x7f39600602d0 2020-11-16T22:14:59.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3957fff700 1 -- 172.21.15.185:0/3638568018 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f3950023070 con 0x7f39600602d0 2020-11-16T22:14:59.582 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:14:59.582 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.583 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:14:59.584 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.585 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.586 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:14:59.587 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.588 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.589 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.590 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.591 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:14:59.592 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.593 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.594 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.595 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:14:59.596 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.597 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:14:59.598 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:14:59.599 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:14:59.600 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.601 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-mds", 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJf05RsCBAAjcDJfG8oWRmgGHZWtDkkyQ==", 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-mds" 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.602 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-mgr", 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJfmatsCBAAQwtw3vFPYziuCVtyyOLEfw==", 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-mgr" 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.603 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-osd", 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJfU79sCBAAaNBwzLiuhmiOJXfmB5qEeQ==", 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-osd" 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.604 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-rbd", 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJfntJsCBAAetoM1g4pe4IoQKx0MGS2pw==", 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-rbd" 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.605 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJfe+ZsCBAAaNBGJsielcQrrkmsKpOGSw==", 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.606 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.bootstrap-rgw", 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCb+bJfYQNtCBAAHXPCKjpll3QKk4tnrTeTEw==", 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile bootstrap-rgw" 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.607 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:14:59.608 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:14:59.609 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:14:59.610 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:14:59.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 msgr2=0x7f393c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f39601478c0 tx=0x7f3948009040).stop 2020-11-16T22:14:59.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 msgr2=0x7f39601462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f39500098e0 tx=0x7f3950015000).stop 2020-11-16T22:14:59.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 shutdown_connections 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3960143a30 0x7f39600b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600602d0 0x7f39601462b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 --2- 172.21.15.185:0/3638568018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39600b7570 0x7f39600afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.595+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 >> 172.21.15.185:0/3638568018 conn(0x7f396005ba80 msgr2=0x7f396005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.599+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 shutdown_connections 2020-11-16T22:14:59.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.599+0000 7f3964cab700 1 -- 172.21.15.185:0/3638568018 wait complete. 2020-11-16T22:14:59.614 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2020-11-16T22:14:59.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 Processor -- start 2020-11-16T22:14:59.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- start start 2020-11-16T22:14:59.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbf100b2160 0x7fbf10067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 0x7fbf10065630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf10098cc0 con 0x7fbf100b17e0 2020-11-16T22:14:59.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1014c4f0 con 0x7fbf100b2160 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100b3950 con 0x7fbf10066240 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf16bb0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 0x7fbf10065630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbf100b2160 0x7fbf10067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf16bb0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 0x7fbf10065630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50562/0 (socket says 172.21.15.185:50562) 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf16bb0700 1 -- 172.21.15.185:0/1738903677 learned_addr learned my addr 172.21.15.185:0/1738903677 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 -- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 msgr2=0x7fbf10065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:14:59.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 --2- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 0x7fbf10065630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 -- 172.21.15.185:0/1738903677 >> v1:172.21.15.96:6789/0 conn(0x7fbf100b17e0 legacy=0x7fbf1014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 -- 172.21.15.185:0/1738903677 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100b3bf0 con 0x7fbf100b2160 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf15bae700 1 --2- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b2160 0x7fbf10067fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fbf0000b9a0 tx=0x7fbf00006df0).ready entity=mon.1 client_cookie=1ee76c295b3ee1d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf153ad700 1 -- 172.21.15.185:0/1738903677 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000097b0 con 0x7fbf100b2160 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf153ad700 1 -- 172.21.15.185:0/1738903677 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00009910 con 0x7fbf100b2160 2020-11-16T22:14:59.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf153ad700 1 -- 172.21.15.185:0/1738903677 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00009c90 con 0x7fbf100b2160 2020-11-16T22:14:59.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b2160 msgr2=0x7fbf10067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 --2- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b2160 0x7fbf10067fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fbf0000b9a0 tx=0x7fbf00006df0).stop 2020-11-16T22:14:59.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- 172.21.15.185:0/1738903677 shutdown_connections 2020-11-16T22:14:59.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 --2- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf10066240 0x7fbf10065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 --2- 172.21.15.185:0/1738903677 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b2160 0x7fbf10067fb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- 172.21.15.185:0/1738903677 >> 172.21.15.185:0/1738903677 conn(0x7fbf100604a0 msgr2=0x7fbf100608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- 172.21.15.185:0/1738903677 shutdown_connections 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.755+0000 7fbf18613700 1 -- 172.21.15.185:0/1738903677 wait complete. 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 Processor -- start 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- start start 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 0x7fbf10147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 0x7fbf100acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10147c50 con 0x7fbf10066240 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100b0810 con 0x7fbf100b03b0 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100adb90 con 0x7fbf100b17e0 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf16bb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 0x7fbf100acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50568/0 (socket says 172.21.15.185:50568) 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 -- 172.21.15.185:0/1175185690 learned_addr learned my addr 172.21.15.185:0/1175185690 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf163af700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 0x7fbf10147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 -- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 msgr2=0x7fbf100acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 0x7fbf100acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 -- 172.21.15.185:0/1175185690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 msgr2=0x7fbf10147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 0x7fbf10147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 -- 172.21.15.185:0/1175185690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100ac340 con 0x7fbf100b17e0 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf16bb0700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 0x7fbf100acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf163af700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 0x7fbf10147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf15bae700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbf0000b150 tx=0x7fbf00015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00011d00 con 0x7fbf100b17e0 2020-11-16T22:14:59.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00011e60 con 0x7fbf100b17e0 2020-11-16T22:14:59.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf100aca40 con 0x7fbf100b17e0 2020-11-16T22:14:59.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0001e670 con 0x7fbf100b17e0 2020-11-16T22:14:59.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.759+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1014a4c0 con 0x7fbf100b17e0 2020-11-16T22:14:59.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1014a6b0 con 0x7fbf100b17e0 2020-11-16T22:14:59.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbf00025030 con 0x7fbf100b17e0 2020-11-16T22:14:59.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbefe7fc700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 0x7fbee8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:14:59.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbf0006f100 con 0x7fbf100b17e0 2020-11-16T22:14:59.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbf163af700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 0x7fbee8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:14:59.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.763+0000 7fbf163af700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 0x7fbee8057520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fbf0c007f10 tx=0x7fbf0c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:14:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.771+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbf00045740 con 0x7fbf100b17e0 2020-11-16T22:14:59.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.975+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbf1005d4e0 con 0x7fbf100b17e0 2020-11-16T22:14:59.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbefe7fc700 1 -- 172.21.15.185:0/1175185690 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf00023070 con 0x7fbf100b17e0 2020-11-16T22:14:59.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 msgr2=0x7fbee8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 0x7fbee8057520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fbf0c007f10 tx=0x7fbf0c009040).stop 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 msgr2=0x7fbf100afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fbf0000b150 tx=0x7fbf00015000).stop 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 shutdown_connections 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbee8055060 0x7fbee8057520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf100b03b0 0x7fbf100acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf10066240 0x7fbf10147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 --2- 172.21.15.185:0/1175185690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf100b17e0 0x7fbf100afec0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 >> 172.21.15.185:0/1175185690 conn(0x7fbf100604a0 msgr2=0x7fbf1014aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 shutdown_connections 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:14:59.987+0000 7fbf18613700 1 -- 172.21.15.185:0/1175185690 wait complete. 2020-11-16T22:14:59.976 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:14:59.990 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2020-11-16T22:15:00.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 Processor -- start 2020-11-16T22:15:00.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 -- start start 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe550144fa0 0x7fe5500b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5501445c0 con 0x7fe5500b2ee0 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe550144860 con 0x7fe5500abef0 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5500ad980 con 0x7fe550144fa0 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe5500abef0 0x7fe5501412c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50572/0 (socket says 172.21.15.185:50572) 2020-11-16T22:15:00.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54f7fe700 1 -- 172.21.15.185:0/3941219204 learned_addr learned my addr 172.21.15.185:0/3941219204 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:00.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54e7fc700 1 --2- 172.21.15.185:0/3941219204 >> v2:172.21.15.96:3300/0 conn(0x7fe550144fa0 0x7fe5500b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226143696 0 0) 0x7fe550144860 con 0x7fe5500abef0 2020-11-16T22:15:00.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe530003240 con 0x7fe5500abef0 2020-11-16T22:15:00.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481179556 0 0) 0x7fe5501445c0 con 0x7fe5500b2ee0 2020-11-16T22:15:00.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe550144860 con 0x7fe5500b2ee0 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3941219204 >> v1:172.21.15.185:6789/0 conn(0x7fe5500abef0 legacy=0x7fe5501412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3941219204 >> v1:172.21.15.96:6789/0 conn(0x7fe5500b2ee0 legacy=0x7fe5500b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 613935192 0 0) 0x7fe530003240 con 0x7fe5500abef0 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3941219204 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe550144b00 con 0x7fe550144fa0 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54e7fc700 1 --2- 172.21.15.185:0/3941219204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe550144fa0 0x7fe5500b32c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe548006fd0 tx=0x7fe5480091a0).ready entity=mon.0 client_cookie=931d48c702b740f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380014e0 con 0x7fe550144fa0 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548005a50 con 0x7fe550144fa0 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54dffb700 1 -- 172.21.15.185:0/3941219204 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548005dd0 con 0x7fe550144fa0 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 -- 172.21.15.185:0/3941219204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe550144fa0 msgr2=0x7fe5500b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.123+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3941219204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe550144fa0 0x7fe5500b32c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe548006fd0 tx=0x7fe5480091a0).stop 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- 172.21.15.185:0/3941219204 shutdown_connections 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3941219204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe550144fa0 0x7fe5500b32c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- 172.21.15.185:0/3941219204 >> 172.21.15.185:0/3941219204 conn(0x7fe550065e30 msgr2=0x7fe550066240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- 172.21.15.185:0/3941219204 shutdown_connections 2020-11-16T22:15:00.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- 172.21.15.185:0/3941219204 wait complete. 2020-11-16T22:15:00.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 Processor -- start 2020-11-16T22:15:00.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- start start 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5500abef0 0x7fe550151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 0x7fe5500ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55013fa40 con 0x7fe5500abef0 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe550140150 con 0x7fe5500b2ee0 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe550154ad0 con 0x7fe550144fa0 2020-11-16T22:15:00.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 0x7fe5500ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50586/0 (socket says 172.21.15.185:50586) 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54f7fe700 1 -- 172.21.15.185:0/3763608726 learned_addr learned my addr 172.21.15.185:0/3763608726 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54effd700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5500abef0 0x7fe550151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 msgr2=0x7fe550154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3763608726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5500abef0 msgr2=0x7fe550151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5500abef0 0x7fe550151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 -- 172.21.15.185:0/3763608726 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe550114170 con 0x7fe5500b2ee0 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54f7fe700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54e7fc700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 0x7fe5500ac360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe548006fd0 tx=0x7fe548014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548006310 con 0x7fe5500b2ee0 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548006490 con 0x7fe5500b2ee0 2020-11-16T22:15:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.127+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55010ffb0 con 0x7fe5500b2ee0 2020-11-16T22:15:00.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548006850 con 0x7fe5500b2ee0 2020-11-16T22:15:00.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5500aca70 con 0x7fe5500b2ee0 2020-11-16T22:15:00.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe550140320 con 0x7fe5500b2ee0 2020-11-16T22:15:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe548027070 con 0x7fe5500b2ee0 2020-11-16T22:15:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe546ffd700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 0x7fe52c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe548018030 con 0x7fe5500b2ee0 2020-11-16T22:15:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe54effd700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 0x7fe52c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.131+0000 7fe54effd700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 0x7fe52c057520 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe5380011a0 tx=0x7fe538008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.139+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe548044d00 con 0x7fe5500b2ee0 2020-11-16T22:15:00.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.343+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe5500b2a10 con 0x7fe5500b2ee0 2020-11-16T22:15:00.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe546ffd700 1 -- 172.21.15.185:0/3763608726 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe548025070 con 0x7fe5500b2ee0 2020-11-16T22:15:00.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 msgr2=0x7fe52c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 0x7fe52c057520 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe5380011a0 tx=0x7fe538008040).stop 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 msgr2=0x7fe5500ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 0x7fe5500ac360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe548006fd0 tx=0x7fe548014000).stop 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 shutdown_connections 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe52c055060 0x7fe52c057520 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe550144fa0 0x7fe550154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5500abef0 0x7fe550151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 --2- 172.21.15.185:0/3763608726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5500b2ee0 0x7fe5500ac360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 >> 172.21.15.185:0/3763608726 conn(0x7fe550065e30 msgr2=0x7fe5500b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:00.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 shutdown_connections 2020-11-16T22:15:00.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.351+0000 7fe54ffff700 1 -- 172.21.15.185:0/3763608726 wait complete. 2020-11-16T22:15:00.342 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:15:00.355 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2020-11-16T22:15:00.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 Processor -- start 2020-11-16T22:15:00.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 -- start start 2020-11-16T22:15:00.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f39780b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f39780b1d30 0x7f397814b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3978061fe0 con 0x7f39780ad110 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39780b2b60 con 0x7f39780b1d30 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397f4ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39780b0fe0 con 0x7f39780b3090 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397eced700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f39780ad110 0x7f39781477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50586/0 (socket says 172.21.15.185:50586) 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397e4ec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f39780b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397eced700 1 -- 172.21.15.185:0/1890395748 learned_addr learned my addr 172.21.15.185:0/1890395748 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397dceb700 1 --2- 172.21.15.185:0/1890395748 >> v2:172.21.15.185:3300/0 conn(0x7f39780b1d30 0x7f397814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397d4ea700 1 -- 172.21.15.185:0/1890395748 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698467009 0 0) 0x7f3978061fe0 con 0x7f39780ad110 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.491+0000 7f397d4ea700 1 -- 172.21.15.185:0/1890395748 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3964003240 con 0x7f39780ad110 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397e4ec700 1 -- 172.21.15.185:0/1890395748 >> v2:172.21.15.185:3300/0 conn(0x7f39780b1d30 msgr2=0x7f397814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397e4ec700 1 --2- 172.21.15.185:0/1890395748 >> v2:172.21.15.185:3300/0 conn(0x7f39780b1d30 0x7f397814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397e4ec700 1 -- 172.21.15.185:0/1890395748 >> v1:172.21.15.185:6789/0 conn(0x7f39780ad110 legacy=0x7f39781477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397e4ec700 1 -- 172.21.15.185:0/1890395748 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780b1280 con 0x7f39780b3090 2020-11-16T22:15:00.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397e4ec700 1 --2- 172.21.15.185:0/1890395748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f39780b1720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f397000b9a0 tx=0x7f3970006df0).ready entity=mon.2 client_cookie=851d2b0de778ae7c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397d4ea700 1 -- 172.21.15.185:0/1890395748 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39700097b0 con 0x7f39780b3090 2020-11-16T22:15:00.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397d4ea700 1 -- 172.21.15.185:0/1890395748 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970009910 con 0x7f39780b3090 2020-11-16T22:15:00.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397d4ea700 1 -- 172.21.15.185:0/1890395748 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970009c90 con 0x7f39780b3090 2020-11-16T22:15:00.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- 172.21.15.185:0/1890395748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 msgr2=0x7f39780b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- 172.21.15.185:0/1890395748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f39780b1720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f397000b9a0 tx=0x7f3970006df0).stop 2020-11-16T22:15:00.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- 172.21.15.185:0/1890395748 shutdown_connections 2020-11-16T22:15:00.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- 172.21.15.185:0/1890395748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f39780b1720 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- 172.21.15.185:0/1890395748 >> v2:172.21.15.185:3300/0 conn(0x7f39780b1d30 0x7f397814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- 172.21.15.185:0/1890395748 >> 172.21.15.185:0/1890395748 conn(0x7f3978060130 msgr2=0x7f3978060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- 172.21.15.185:0/1890395748 shutdown_connections 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- 172.21.15.185:0/1890395748 wait complete. 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 Processor -- start 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- start start 2020-11-16T22:15:00.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39780ad110 0x7f397814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 0x7f3978145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978112260 con 0x7f39780ad110 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978112b10 con 0x7f39780b1d30 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.495+0000 7f397f4ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397814c360 con 0x7f39780b3090 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397dceb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 0x7f3978145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50598/0 (socket says 172.21.15.185:50598) 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 -- 172.21.15.185:0/2001459698 learned_addr learned my addr 172.21.15.185:0/2001459698 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397dceb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 0x7f3978145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57150/0 (socket says 172.21.15.185:57150) 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397e4ec700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39780ad110 0x7f397814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 -- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 msgr2=0x7f3978145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 0x7f3978145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 -- 172.21.15.185:0/2001459698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39780ad110 msgr2=0x7f397814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39780ad110 0x7f397814b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 -- 172.21.15.185:0/2001459698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978112490 con 0x7f39780b3090 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397eced700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f396800cac0 tx=0x7f3968007d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968005b80 con 0x7f39780b3090 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968005ce0 con 0x7f39780b3090 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3978112f70 con 0x7f39780b3090 2020-11-16T22:15:00.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968007320 con 0x7f39780b3090 2020-11-16T22:15:00.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3978111e60 con 0x7f39780b3090 2020-11-16T22:15:00.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397814c4d0 con 0x7f39780b3090 2020-11-16T22:15:00.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3968010030 con 0x7f39780b3090 2020-11-16T22:15:00.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 0x7f3950057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f396806da70 con 0x7f39780b3090 2020-11-16T22:15:00.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.499+0000 7f397e4ec700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 0x7f3950057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.503+0000 7f397e4ec700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 0x7f3950057520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f3970006e30 tx=0x7f3970015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.511+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f397814c4d0 con 0x7f39780b3090 2020-11-16T22:15:00.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.719+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f397814c4a0 con 0x7f39780b3090 2020-11-16T22:15:00.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f39767fc700 1 -- 172.21.15.185:0/2001459698 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f396806de60 con 0x7f39780b3090 2020-11-16T22:15:00.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 msgr2=0x7f3950057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 0x7f3950057520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f3970006e30 tx=0x7f3970015000).stop 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 msgr2=0x7f3978157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f396800cac0 tx=0x7f3968007d60).stop 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 shutdown_connections 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3950055060 0x7f3950057520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39780b3090 0x7f3978157370 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39780ad110 0x7f397814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 --2- 172.21.15.185:0/2001459698 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39780b1d30 0x7f3978145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 >> 172.21.15.185:0/2001459698 conn(0x7f3978060130 msgr2=0x7f3978148740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:00.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 shutdown_connections 2020-11-16T22:15:00.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.727+0000 7f397f4ee700 1 -- 172.21.15.185:0/2001459698 wait complete. 2020-11-16T22:15:00.716 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:15:00.730 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2020-11-16T22:15:00.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 Processor -- start 2020-11-16T22:15:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- start start 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee40b30e0 0x7f2ee40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2ee4062970 0x7f2ee4062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40632b0 con 0x7f2ee40b86d0 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee40b4060 con 0x7f2ee4062970 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b4430 con 0x7f2ee40b30e0 2020-11-16T22:15:00.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee9ea7700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2ee40b86d0 0x7f2ee40b38b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50596/0 (socket says 172.21.15.185:50596) 2020-11-16T22:15:00.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee9ea7700 1 -- 172.21.15.185:0/2413967264 learned_addr learned my addr 172.21.15.185:0/2413967264 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:00.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/2413967264 >> v2:172.21.15.185:3300/0 conn(0x7f2ee4062970 0x7f2ee4062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee96a6700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee40b30e0 0x7f2ee40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee8ea5700 1 -- 172.21.15.185:0/2413967264 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126267216 0 0) 0x7f2ed4001450 con 0x7f2ee40b86d0 2020-11-16T22:15:00.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 -- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee40b30e0 msgr2=0x7f2ee40b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:00.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee40b30e0 0x7f2ee40b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 -- 172.21.15.185:0/2413967264 >> v1:172.21.15.185:6789/0 conn(0x7f2ee40b86d0 legacy=0x7f2ee40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 -- 172.21.15.185:0/2413967264 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40b46d0 con 0x7f2ee4062970 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee4062970 0x7f2ee4062dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f2edc00b9a0 tx=0x7f2edc006df0).ready entity=mon.1 client_cookie=acad9b97c0d03e9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee8ea5700 1 -- 172.21.15.185:0/2413967264 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0097b0 con 0x7f2ee4062970 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee8ea5700 1 -- 172.21.15.185:0/2413967264 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc009910 con 0x7f2ee4062970 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2ee8ea5700 1 -- 172.21.15.185:0/2413967264 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc009c90 con 0x7f2ee4062970 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee4062970 msgr2=0x7f2ee4062dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee4062970 0x7f2ee4062dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f2edc00b9a0 tx=0x7f2edc006df0).stop 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- 172.21.15.185:0/2413967264 shutdown_connections 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee4062970 0x7f2ee4062dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/2413967264 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee40b30e0 0x7f2ee40b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- 172.21.15.185:0/2413967264 >> 172.21.15.185:0/2413967264 conn(0x7f2ee40645d0 msgr2=0x7f2ee40649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.867+0000 7f2eeaea9700 1 -- 172.21.15.185:0/2413967264 shutdown_connections 2020-11-16T22:15:00.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- 172.21.15.185:0/2413967264 wait complete. 2020-11-16T22:15:00.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 Processor -- start 2020-11-16T22:15:00.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- start start 2020-11-16T22:15:00.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 0x7f2ee41409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 0x7f2ee4140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41413b0 con 0x7f2ee40b30e0 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee413f070 con 0x7f2ee40b86d0 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4156b90 con 0x7f2ee4062970 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee9ea7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 0x7f2ee41409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57168/0 (socket says 172.21.15.185:57168) 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 -- 172.21.15.185:0/1609218808 learned_addr learned my addr 172.21.15.185:0/1609218808 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:00.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee96a6700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 0x7f2ee4140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 -- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 msgr2=0x7f2ee41409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 0x7f2ee41409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 -- 172.21.15.185:0/1609218808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 msgr2=0x7f2ee4140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 0x7f2ee4140ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 -- 172.21.15.185:0/1609218808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee41568f0 con 0x7f2ee40b86d0 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee96a6700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 0x7f2ee4140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:00.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee9ea7700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 0x7f2ee41409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eea6a8700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f2ee4065460 tx=0x7f2edc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01e460 con 0x7f2ee40b86d0 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc01e5c0 con 0x7f2ee40b86d0 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee413f330 con 0x7f2ee40b86d0 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01e940 con 0x7f2ee40b86d0 2020-11-16T22:15:00.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.871+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee4141730 con 0x7f2ee40b86d0 2020-11-16T22:15:00.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee413f4d0 con 0x7f2ee40b86d0 2020-11-16T22:15:00.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2edc025030 con 0x7f2ee40b86d0 2020-11-16T22:15:00.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2ee1ffb700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 0x7f2ec0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:00.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2edc06f150 con 0x7f2ee40b86d0 2020-11-16T22:15:00.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2ee9ea7700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 0x7f2ec0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:00.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.875+0000 7f2ee9ea7700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 0x7f2ec0057520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2ed4007fd0 tx=0x7f2ed4007be0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:00.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:00.879+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ee413f4d0 con 0x7f2ee40b86d0 2020-11-16T22:15:01.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.087+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2ee405d200 con 0x7f2ee40b86d0 2020-11-16T22:15:01.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2ee1ffb700 1 -- 172.21.15.185:0/1609218808 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2edc023020 con 0x7f2ee40b86d0 2020-11-16T22:15:01.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 msgr2=0x7f2ec0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 0x7f2ec0057520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2ed4007fd0 tx=0x7f2ed4007be0).stop 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 msgr2=0x7f2ee413d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f2ee4065460 tx=0x7f2edc015000).stop 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 shutdown_connections 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ec0055060 0x7f2ec0057520 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ee40b86d0 0x7f2ee413d310 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ee4062970 0x7f2ee41409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 --2- 172.21.15.185:0/1609218808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ee40b30e0 0x7f2ee4140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 >> 172.21.15.185:0/1609218808 conn(0x7f2ee40645d0 msgr2=0x7f2ee40b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 shutdown_connections 2020-11-16T22:15:01.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.095+0000 7f2eeaea9700 1 -- 172.21.15.185:0/1609218808 wait complete. 2020-11-16T22:15:01.084 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:15:01.095 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2020-11-16T22:15:01.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 Processor -- start 2020-11-16T22:15:01.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 -- start start 2020-11-16T22:15:01.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa36813de70 0x7fa3681448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3680b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 0x7fa3680b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3680b27a0 con 0x7fa368068060 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa368144e70 con 0x7fa36813de70 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36f2f2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa368067680 con 0x7fa368060530 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36e2f0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa36813de70 0x7fa3681448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36e2f0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa36813de70 0x7fa3681448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57172/0 (socket says 172.21.15.185:57172) 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36daef700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3680b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36e2f0700 1 -- 172.21.15.185:0/2649958362 learned_addr learned my addr 172.21.15.185:0/2649958362 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:01.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.227+0000 7fa36eaf1700 1 --2- 172.21.15.185:0/2649958362 >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 0x7fa3680b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 -- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 msgr2=0x7fa3680b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3680b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 -- 172.21.15.185:0/2649958362 >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 msgr2=0x7fa3680b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/2649958362 >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 0x7fa3680b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 -- 172.21.15.185:0/2649958362 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368067920 con 0x7fa36813de70 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36eaf1700 1 --2- 172.21.15.185:0/2649958362 >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 0x7fa3680b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:01.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa36813de70 0x7fa3681448a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa368062290 tx=0x7fa364006eb0).ready entity=mon.1 client_cookie=26991f2103318d6e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36d2ee700 1 -- 172.21.15.185:0/2649958362 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364009810 con 0x7fa36813de70 2020-11-16T22:15:01.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36d2ee700 1 -- 172.21.15.185:0/2649958362 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364009970 con 0x7fa36813de70 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36d2ee700 1 -- 172.21.15.185:0/2649958362 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364009cf0 con 0x7fa36813de70 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa36813de70 msgr2=0x7fa3681448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa36813de70 0x7fa3681448a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa368062290 tx=0x7fa364006eb0).stop 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- 172.21.15.185:0/2649958362 shutdown_connections 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/2649958362 >> v2:172.21.15.96:3300/0 conn(0x7fa368068060 0x7fa3680b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa36813de70 0x7fa3681448a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/2649958362 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3680b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- 172.21.15.185:0/2649958362 >> 172.21.15.185:0/2649958362 conn(0x7fa36805ba80 msgr2=0x7fa36805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- 172.21.15.185:0/2649958362 shutdown_connections 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- 172.21.15.185:0/2649958362 wait complete. 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 Processor -- start 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.231+0000 7fa36f2f2700 1 -- start start 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa368068060 0x7fa3680b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 0x7fa368157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa368156c40 con 0x7fa36813de70 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680ae660 con 0x7fa368068060 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa368156e70 con 0x7fa368060530 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50622/0 (socket says 172.21.15.185:50622) 2020-11-16T22:15:01.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 -- 172.21.15.185:0/1884610011 learned_addr learned my addr 172.21.15.185:0/1884610011 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36daef700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa368068060 0x7fa3680b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36eaf1700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 0x7fa368157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 -- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa368068060 msgr2=0x7fa3680b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa368068060 0x7fa3680b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 -- 172.21.15.185:0/1884610011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 msgr2=0x7fa368157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 0x7fa368157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 -- 172.21.15.185:0/1884610011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680b6fe0 con 0x7fa368060530 2020-11-16T22:15:01.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36eaf1700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 0x7fa368157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36e2f0700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa364006fd0 tx=0x7fa364015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364006890 con 0x7fa368060530 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364011d50 con 0x7fa368060530 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3680ae900 con 0x7fa368060530 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36401e670 con 0x7fa368060530 2020-11-16T22:15:01.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa368141780 con 0x7fa368060530 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36805d200 con 0x7fa368060530 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.235+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa364025030 con 0x7fa368060530 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.239+0000 7fa3567fc700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 0x7fa340057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.239+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa36406efd0 con 0x7fa368060530 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.239+0000 7fa36daef700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 0x7fa340057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.239+0000 7fa36daef700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 0x7fa340057520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa368143550 tx=0x7fa358006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.243+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa3640455b0 con 0x7fa368060530 2020-11-16T22:15:01.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.455+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa3680ae440 con 0x7fa368060530 2020-11-16T22:15:01.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.463+0000 7fa3567fc700 1 -- 172.21.15.185:0/1884610011 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa364023030 con 0x7fa368060530 2020-11-16T22:15:01.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 msgr2=0x7fa340057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 0x7fa340057520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa368143550 tx=0x7fa358006df0).stop 2020-11-16T22:15:01.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 msgr2=0x7fa3681535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa364006fd0 tx=0x7fa364015000).stop 2020-11-16T22:15:01.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 shutdown_connections 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa340055060 0x7fa340057520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa36813de70 0x7fa368157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa368060530 0x7fa3681535d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 --2- 172.21.15.185:0/1884610011 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa368068060 0x7fa3680b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 >> 172.21.15.185:0/1884610011 conn(0x7fa36805ba80 msgr2=0x7fa36805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 shutdown_connections 2020-11-16T22:15:01.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.467+0000 7fa36f2f2700 1 -- 172.21.15.185:0/1884610011 wait complete. 2020-11-16T22:15:01.456 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:15:01.470 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2020-11-16T22:15:01.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 Processor -- start 2020-11-16T22:15:01.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 -- start start 2020-11-16T22:15:01.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99d40acc10 0x7f99d41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f99d4061e60 0x7f99d41437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d40abf90 con 0x7f99d40669a0 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d40ac230 con 0x7f99d40acc10 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d40ac4d0 con 0x7f99d4061e60 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d2d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f99d4061e60 0x7f99d41437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d259c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f99d40669a0 0x7f99d4066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50624/0 (socket says 172.21.15.185:50624) 2020-11-16T22:15:01.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99d259c700 1 -- 172.21.15.185:0/2950592678 learned_addr learned my addr 172.21.15.185:0/2950592678 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:01.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.603+0000 7f99c9d9b700 1 --2- 172.21.15.185:0/2950592678 >> v2:172.21.15.96:3300/0 conn(0x7f99d40acc10 0x7f99d41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d1d9b700 1 -- 172.21.15.185:0/2950592678 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790367440 0 0) 0x7f99d40abf90 con 0x7f99d40669a0 2020-11-16T22:15:01.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d1d9b700 1 -- 172.21.15.185:0/2950592678 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b4003240 con 0x7f99d40669a0 2020-11-16T22:15:01.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d2d9d700 1 -- 172.21.15.185:0/2950592678 >> v2:172.21.15.96:3300/0 conn(0x7f99d40acc10 msgr2=0x7f99d41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:01.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d2d9d700 1 --2- 172.21.15.185:0/2950592678 >> v2:172.21.15.96:3300/0 conn(0x7f99d40acc10 0x7f99d41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d2d9d700 1 -- 172.21.15.185:0/2950592678 >> v1:172.21.15.185:6789/0 conn(0x7f99d40669a0 legacy=0x7f99d4066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d2d9d700 1 -- 172.21.15.185:0/2950592678 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d40ac770 con 0x7f99d4061e60 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d2d9d700 1 --2- 172.21.15.185:0/2950592678 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4061e60 0x7f99d41437d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f99d4066500 tx=0x7f99cc0091d0).ready entity=mon.1 client_cookie=e2ba410dd924711a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d1d9b700 1 -- 172.21.15.185:0/2950592678 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00b9c0 con 0x7f99d4061e60 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d1d9b700 1 -- 172.21.15.185:0/2950592678 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc00bb20 con 0x7f99d4061e60 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d1d9b700 1 -- 172.21.15.185:0/2950592678 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00bea0 con 0x7f99d4061e60 2020-11-16T22:15:01.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- 172.21.15.185:0/2950592678 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4061e60 msgr2=0x7f99d41437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 --2- 172.21.15.185:0/2950592678 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4061e60 0x7f99d41437d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f99d4066500 tx=0x7f99cc0091d0).stop 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- 172.21.15.185:0/2950592678 shutdown_connections 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 --2- 172.21.15.185:0/2950592678 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4061e60 0x7f99d41437d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 --2- 172.21.15.185:0/2950592678 >> v2:172.21.15.96:3300/0 conn(0x7f99d40acc10 0x7f99d41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- 172.21.15.185:0/2950592678 >> 172.21.15.185:0/2950592678 conn(0x7f99d4060130 msgr2=0x7f99d4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- 172.21.15.185:0/2950592678 shutdown_connections 2020-11-16T22:15:01.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- 172.21.15.185:0/2950592678 wait complete. 2020-11-16T22:15:01.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 Processor -- start 2020-11-16T22:15:01.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.607+0000 7f99d359e700 1 -- start start 2020-11-16T22:15:01.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 0x7f99d41415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 0x7f99d4113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d4154530 con 0x7f99d40669a0 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d41538b0 con 0x7f99d4113fb0 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d4153b50 con 0x7f99d4061e60 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 0x7f99d41415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57196/0 (socket says 172.21.15.185:57196) 2020-11-16T22:15:01.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 0x7f99d41415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50636/0 (socket says 172.21.15.185:50636) 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 -- 172.21.15.185:0/3420649396 learned_addr learned my addr 172.21.15.185:0/3420649396 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99c9d9b700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 0x7f99d4113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 -- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 msgr2=0x7f99d41415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 0x7f99d41415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 -- 172.21.15.185:0/3420649396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 msgr2=0x7f99d4113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 0x7f99d4113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 -- 172.21.15.185:0/3420649396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4153df0 con 0x7f99d4113fb0 2020-11-16T22:15:01.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99c9d9b700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 0x7f99d4113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:01.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d2d9d700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f99cc011da0 tx=0x7f99cc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc01c5f0 con 0x7f99d4113fb0 2020-11-16T22:15:01.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc01c750 con 0x7f99d4113fb0 2020-11-16T22:15:01.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d4154090 con 0x7f99d4113fb0 2020-11-16T22:15:01.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc01caf0 con 0x7f99d4113fb0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.611+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d4141ae0 con 0x7f99d4113fb0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d4154220 con 0x7f99d4113fb0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f99cc017070 con 0x7f99d4113fb0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99caffd700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 0x7f99b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f99cc06fe40 con 0x7f99d4113fb0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99d259c700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 0x7f99b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.615+0000 7f99d259c700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 0x7f99b0057520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f99c40055a0 tx=0x7f99c4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.623+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f99cc046410 con 0x7f99d4113fb0 2020-11-16T22:15:01.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.831+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f99d41536b0 con 0x7f99d4113fb0 2020-11-16T22:15:01.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.839+0000 7f99caffd700 1 -- 172.21.15.185:0/3420649396 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v14) v1 ==== 97+0+0 (secure 0 0 0) 0x7f99cc015030 con 0x7f99d4113fb0 2020-11-16T22:15:01.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 msgr2=0x7f99b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 0x7f99b0057520 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f99c40055a0 tx=0x7f99c4009040).stop 2020-11-16T22:15:01.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 msgr2=0x7f99d41151b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f99cc011da0 tx=0x7f99cc016000).stop 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 shutdown_connections 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f99b0055060 0x7f99b0057520 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f99d4113fb0 0x7f99d41151b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f99d4061e60 0x7f99d41415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 --2- 172.21.15.185:0/3420649396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d40669a0 0x7f99d4113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 >> 172.21.15.185:0/3420649396 conn(0x7f99d4060130 msgr2=0x7f99d4142630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 shutdown_connections 2020-11-16T22:15:01.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.843+0000 7f99d359e700 1 -- 172.21.15.185:0/3420649396 wait complete. 2020-11-16T22:15:01.832 INFO:teuthology.orchestra.run.smithi185.stderr:updated 2020-11-16T22:15:01.844 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:15:01.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 Processor -- start 2020-11-16T22:15:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 -- start start 2020-11-16T22:15:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f762c0b6aa0 0x7f762c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762c147f90 con 0x7f762c0b64b0 2020-11-16T22:15:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762c0af020 con 0x7f762c0b6aa0 2020-11-16T22:15:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0b4810 con 0x7f762c0b72e0 2020-11-16T22:15:01.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50646/0 (socket says 172.21.15.185:50646) 2020-11-16T22:15:01.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f762c0b6aa0 0x7f762c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 -- 172.21.15.185:0/912510373 learned_addr learned my addr 172.21.15.185:0/912510373 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 -- 172.21.15.185:0/912510373 >> v2:172.21.15.185:3300/0 conn(0x7f762c0b6aa0 msgr2=0x7f762c0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 --2- 172.21.15.185:0/912510373 >> v2:172.21.15.185:3300/0 conn(0x7f762c0b6aa0 0x7f762c0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 -- 172.21.15.185:0/912510373 >> v1:172.21.15.96:6789/0 conn(0x7f762c0b64b0 legacy=0x7f762c144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 -- 172.21.15.185:0/912510373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c148460 con 0x7f762c0b72e0 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.983+0000 7f762affd700 1 --2- 172.21.15.185:0/912510373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f7614006fd0 tx=0x7f7614006e50).ready entity=mon.2 client_cookie=bb81f183b22929af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762a7fc700 1 -- 172.21.15.185:0/912510373 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614006030 con 0x7f762c0b72e0 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762a7fc700 1 -- 172.21.15.185:0/912510373 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614006190 con 0x7f762c0b72e0 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762a7fc700 1 -- 172.21.15.185:0/912510373 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614006510 con 0x7f762c0b72e0 2020-11-16T22:15:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- 172.21.15.185:0/912510373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 msgr2=0x7f762c0b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- 172.21.15.185:0/912510373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f7614006fd0 tx=0x7f7614006e50).stop 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- 172.21.15.185:0/912510373 shutdown_connections 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- 172.21.15.185:0/912510373 >> v2:172.21.15.185:3300/0 conn(0x7f762c0b6aa0 0x7f762c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- 172.21.15.185:0/912510373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c0b72e0 0x7f762c0b76c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- 172.21.15.185:0/912510373 >> 172.21.15.185:0/912510373 conn(0x7f762c067860 msgr2=0x7f762c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- 172.21.15.185:0/912510373 shutdown_connections 2020-11-16T22:15:01.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- 172.21.15.185:0/912510373 wait complete. 2020-11-16T22:15:01.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 Processor -- start 2020-11-16T22:15:01.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- start start 2020-11-16T22:15:01.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c0b64b0 0x7f762c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 0x7f762c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c157750 con 0x7f762c0b64b0 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c1579f0 con 0x7f762c0b6aa0 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0b0770 con 0x7f762c157260 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f7623fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 0x7f762c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f7623fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57208/0 (socket says 172.21.15.185:57208) 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f7623fff700 1 -- 172.21.15.185:0/3524814444 learned_addr learned my addr 172.21.15.185:0/3524814444 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.987+0000 7f762affd700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c0b64b0 0x7f762c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 -- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 msgr2=0x7f762c156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 -- 172.21.15.185:0/3524814444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c0b64b0 msgr2=0x7f762c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c0b64b0 0x7f762c13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 -- 172.21.15.185:0/3524814444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0b0a10 con 0x7f762c157260 2020-11-16T22:15:01.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7623fff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:01.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762b7fe700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 0x7f762c0b1b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f762400ef20 tx=0x7f76240091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76240095f0 con 0x7f762c157260 2020-11-16T22:15:01.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762400cb00 con 0x7f762c157260 2020-11-16T22:15:01.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c141120 con 0x7f762c157260 2020-11-16T22:15:01.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400cea0 con 0x7f762c157260 2020-11-16T22:15:01.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c0b0120 con 0x7f762c157260 2020-11-16T22:15:01.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762c0b0ba0 con 0x7f762c157260 2020-11-16T22:15:01.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7624022070 con 0x7f762c157260 2020-11-16T22:15:01.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 0x7f75fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:01.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f762406e120 con 0x7f762c157260 2020-11-16T22:15:01.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.991+0000 7f762affd700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 0x7f75fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:01.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.995+0000 7f762affd700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 0x7f75fc057520 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f762c080a30 tx=0x7f7614015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:01.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:01.995+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f76240446b0 con 0x7f762c157260 2020-11-16T22:15:02.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f762c05d200 con 0x7f762c157260 2020-11-16T22:15:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f7622ffd700 1 -- 172.21.15.185:0/3524814444 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+13675 (secure 0 0 0) 0x7f7624016020 con 0x7f762c157260 2020-11-16T22:15:02.138 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:02.138 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:14:59.236511+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:15:02.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 msgr2=0x7f75fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 0x7f75fc057520 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f762c080a30 tx=0x7f7614015000).stop 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 msgr2=0x7f762c0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 0x7f762c0b1b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f762400ef20 tx=0x7f76240091d0).stop 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 shutdown_connections 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75fc055060 0x7f75fc057520 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f762c157260 0x7f762c0b1b90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c0b64b0 0x7f762c13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 --2- 172.21.15.185:0/3524814444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f762c0b6aa0 0x7f762c156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 >> 172.21.15.185:0/3524814444 conn(0x7f762c067860 msgr2=0x7f762c143620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.151+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 shutdown_connections 2020-11-16T22:15:02.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.155+0000 7f762bfff700 1 -- 172.21.15.185:0/3524814444 wait complete. 2020-11-16T22:15:02.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:15:02.153 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:15:02.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.283+0000 7fdcece37700 1 Processor -- start 2020-11-16T22:15:02.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.283+0000 7fdcece37700 1 -- start start 2020-11-16T22:15:02.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdce80b1a20 0x7fdce81411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdce8147fb0 0x7fdce81416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce80b2f30 con 0x7fdce80b1a20 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce8143f90 con 0x7fdce8147fb0 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8060fe0 con 0x7fdce805ba80 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdce8147fb0 0x7fdce81416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50662/0 (socket says 172.21.15.185:50662) 2020-11-16T22:15:02.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdce8147fb0 0x7fdce81416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57216/0 (socket says 172.21.15.185:57216) 2020-11-16T22:15:02.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce7fff700 1 -- 172.21.15.185:0/2478021660 learned_addr learned my addr 172.21.15.185:0/2478021660 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce77fe700 1 --2- 172.21.15.185:0/2478021660 >> v2:172.21.15.96:3300/0 conn(0x7fdce80b1a20 0x7fdce81411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 -- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 msgr2=0x7fdce81443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 -- 172.21.15.185:0/2478021660 >> v2:172.21.15.96:3300/0 conn(0x7fdce80b1a20 msgr2=0x7fdce81411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/2478021660 >> v2:172.21.15.96:3300/0 conn(0x7fdce80b1a20 0x7fdce81411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 -- 172.21.15.185:0/2478021660 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8061280 con 0x7fdce8147fb0 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce7fff700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce8147fb0 0x7fdce81416a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fdce80b1490 tx=0x7fdcdc006df0).ready entity=mon.1 client_cookie=a2a3e73362b2a1c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce67fc700 1 -- 172.21.15.185:0/2478021660 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0096d0 con 0x7fdce8147fb0 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce67fc700 1 -- 172.21.15.185:0/2478021660 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc009830 con 0x7fdce8147fb0 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdce67fc700 1 -- 172.21.15.185:0/2478021660 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc009bb0 con 0x7fdce8147fb0 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce8147fb0 msgr2=0x7fdce81416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce8147fb0 0x7fdce81416a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fdce80b1490 tx=0x7fdcdc006df0).stop 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- 172.21.15.185:0/2478021660 shutdown_connections 2020-11-16T22:15:02.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce805ba80 0x7fdce81443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- 172.21.15.185:0/2478021660 >> v2:172.21.15.96:3300/0 conn(0x7fdce80b1a20 0x7fdce81411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 --2- 172.21.15.185:0/2478021660 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce8147fb0 0x7fdce81416a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- 172.21.15.185:0/2478021660 >> 172.21.15.185:0/2478021660 conn(0x7fdce8063df0 msgr2=0x7fdce80641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- 172.21.15.185:0/2478021660 shutdown_connections 2020-11-16T22:15:02.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.287+0000 7fdcece37700 1 -- 172.21.15.185:0/2478021660 wait complete. 2020-11-16T22:15:02.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 Processor -- start 2020-11-16T22:15:02.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 -- start start 2020-11-16T22:15:02.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce80b1a20 0x7fdce8142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce81335f0 con 0x7fdce80b1a20 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8140770 con 0x7fdce805ba80 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdcece37700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8153c80 con 0x7fdce8147fb0 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50668/0 (socket says 172.21.15.185:50668) 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57220/0 (socket says 172.21.15.185:57220) 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce7fff700 1 -- 172.21.15.185:0/1167705232 learned_addr learned my addr 172.21.15.185:0/1167705232 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.291+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce80b1a20 0x7fdce8142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 -- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 msgr2=0x7fdce813e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 -- 172.21.15.185:0/1167705232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce80b1a20 msgr2=0x7fdce8142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce80b1a20 0x7fdce8142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 -- 172.21.15.185:0/1167705232 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8153f20 con 0x7fdce805ba80 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce7fff700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce77fe700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdce000ef50 tx=0x7fdce00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000cb30 con 0x7fdce805ba80 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce000cc90 con 0x7fdce805ba80 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce81541c0 con 0x7fdce805ba80 2020-11-16T22:15:02.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0012600 con 0x7fdce805ba80 2020-11-16T22:15:02.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce81403f0 con 0x7fdce805ba80 2020-11-16T22:15:02.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce805d200 con 0x7fdce805ba80 2020-11-16T22:15:02.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdce0022070 con 0x7fdce805ba80 2020-11-16T22:15:02.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 0x7fdcc0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 0x7fdcc0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdce006e1f0 con 0x7fdce805ba80 2020-11-16T22:15:02.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.295+0000 7fdce6ffd700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 0x7fdcc0057520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fdce81483f0 tx=0x7fdcdc015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.303+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdce0044780 con 0x7fdce805ba80 2020-11-16T22:15:02.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdce8060330 con 0x7fdce805ba80 2020-11-16T22:15:02.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcc77fe700 1 -- 172.21.15.185:0/1167705232 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v27) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdce0016070 con 0x7fdce805ba80 2020-11-16T22:15:02.441 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:15:02.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 msgr2=0x7fdcc0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 0x7fdcc0057520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fdce81483f0 tx=0x7fdcdc015000).stop 2020-11-16T22:15:02.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 msgr2=0x7fdce8154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdce000ef50 tx=0x7fdce00091d0).stop 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 shutdown_connections 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdcc0055060 0x7fdcc0057520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdce805ba80 0x7fdce8154660 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdce8147fb0 0x7fdce813e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 --2- 172.21.15.185:0/1167705232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdce80b1a20 0x7fdce8142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 >> 172.21.15.185:0/1167705232 conn(0x7fdce8063df0 msgr2=0x7fdce813ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 shutdown_connections 2020-11-16T22:15:02.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.455+0000 7fdcece37700 1 -- 172.21.15.185:0/1167705232 wait complete. 2020-11-16T22:15:02.455 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:15:02.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.595+0000 7f7dff59e700 1 Processor -- start 2020-11-16T22:15:02.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.595+0000 7f7dff59e700 1 -- start start 2020-11-16T22:15:02.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e000acdf0 con 0x7f7e000b3020 2020-11-16T22:15:02.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e000ad2c0 con 0x7f7e000abef0 2020-11-16T22:15:02.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001462b0 con 0x7f7e000b17c0 2020-11-16T22:15:02.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50676/0 (socket says 172.21.15.185:50676) 2020-11-16T22:15:02.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 -- 172.21.15.185:0/1327470340 learned_addr learned my addr 172.21.15.185:0/1327470340 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 -- 172.21.15.185:0/1327470340 >> v1:172.21.15.185:6789/0 conn(0x7f7e000abef0 legacy=0x7f7e00141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 -- 172.21.15.185:0/1327470340 >> v1:172.21.15.96:6789/0 conn(0x7f7e000b3020 legacy=0x7f7e00145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:02.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 -- 172.21.15.185:0/1327470340 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000ad020 con 0x7f7e000b17c0 2020-11-16T22:15:02.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfed9d700 1 --2- 172.21.15.185:0/1327470340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f7df8009d80 tx=0x7f7df8009290).ready entity=mon.2 client_cookie=7fd4211e8045f9ae server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfd59a700 1 -- 172.21.15.185:0/1327470340 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800bc40 con 0x7f7e000b17c0 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfd59a700 1 -- 172.21.15.185:0/1327470340 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df800bda0 con 0x7f7e000b17c0 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dfd59a700 1 -- 172.21.15.185:0/1327470340 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8011610 con 0x7f7e000b17c0 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- 172.21.15.185:0/1327470340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 msgr2=0x7f7e00141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 --2- 172.21.15.185:0/1327470340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f7df8009d80 tx=0x7f7df8009290).stop 2020-11-16T22:15:02.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- 172.21.15.185:0/1327470340 shutdown_connections 2020-11-16T22:15:02.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 --2- 172.21.15.185:0/1327470340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000b17c0 0x7f7e00141860 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- 172.21.15.185:0/1327470340 >> 172.21.15.185:0/1327470340 conn(0x7f7e00060130 msgr2=0x7f7e00060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- 172.21.15.185:0/1327470340 shutdown_connections 2020-11-16T22:15:02.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- 172.21.15.185:0/1327470340 wait complete. 2020-11-16T22:15:02.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 Processor -- start 2020-11-16T22:15:02.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.599+0000 7f7dff59e700 1 -- start start 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 0x7f7e0010f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 0x7f7e00151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 0x7f7e00151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00111120 con 0x7f7e000b3020 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00111350 con 0x7f7e000b17c0 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001115f0 con 0x7f7e000abef0 2020-11-16T22:15:02.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfe59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 0x7f7e0010f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfe59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 0x7f7e0010f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50678/0 (socket says 172.21.15.185:50678) 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfe59c700 1 -- 172.21.15.185:0/3136873659 learned_addr learned my addr 172.21.15.185:0/3136873659 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 0x7f7e00151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfed9d700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 0x7f7e00151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 -- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 msgr2=0x7f7e0010f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 0x7f7e0010f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 -- 172.21.15.185:0/3136873659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 msgr2=0x7f7e00151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 0x7f7e00151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 -- 172.21.15.185:0/3136873659 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00151d30 con 0x7f7e000b17c0 2020-11-16T22:15:02.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfed9d700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 0x7f7e00151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:02.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dfdd9b700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 0x7f7e00151350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f7de800c710 tx=0x7f7de8007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8007020 con 0x7f7e000b17c0 2020-11-16T22:15:02.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de8007180 con 0x7f7e000b17c0 2020-11-16T22:15:02.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e000aede0 con 0x7f7e000b17c0 2020-11-16T22:15:02.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de80064f0 con 0x7f7e000b17c0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0013d5c0 con 0x7f7e000b17c0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.603+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7de8010070 con 0x7f7e000b17c0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.607+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e001117b0 con 0x7f7e000b17c0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.607+0000 7f7df67fc700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 0x7f7dd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.607+0000 7f7dfe59c700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 0x7f7dd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.607+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7de806cec0 con 0x7f7e000b17c0 2020-11-16T22:15:02.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.607+0000 7f7dfe59c700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 0x7f7dd0057520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f7df0003370 tx=0x7f7df0007c30).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.611+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7de8043450 con 0x7f7e000b17c0 2020-11-16T22:15:02.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.763+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e0005d200 con 0x7f7e000b17c0 2020-11-16T22:15:02.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7df67fc700 1 -- 172.21.15.185:0/3136873659 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7de8016020 con 0x7f7e000b17c0 2020-11-16T22:15:02.752 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 msgr2=0x7f7dd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 0x7f7dd0057520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f7df0003370 tx=0x7f7df0007c30).stop 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 msgr2=0x7f7e00151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 0x7f7e00151350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f7de800c710 tx=0x7f7de8007b90).stop 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 shutdown_connections 2020-11-16T22:15:02.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7dd0055060 0x7f7dd0057520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e000b3020 0x7f7e00151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7e000abef0 0x7f7e0010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 --2- 172.21.15.185:0/3136873659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7e000b17c0 0x7f7e00151350 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 >> 172.21.15.185:0/3136873659 conn(0x7f7e00060130 msgr2=0x7f7e000b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 shutdown_connections 2020-11-16T22:15:02.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.767+0000 7f7dff59e700 1 -- 172.21.15.185:0/3136873659 wait complete. 2020-11-16T22:15:02.768 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:15:02.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.899+0000 7f7606c3e700 1 Processor -- start 2020-11-16T22:15:02.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.899+0000 7f7606c3e700 1 -- start start 2020-11-16T22:15:02.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f76000b17e0 0x7f7600067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7600066240 0x7f7600065630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7600098cc0 con 0x7f76000b2160 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760014c4f0 con 0x7f76000b17e0 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000b3950 con 0x7f7600066240 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7600066240 0x7f7600065630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7600066240 0x7f7600065630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57242/0 (socket says 172.21.15.185:57242) 2020-11-16T22:15:02.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 -- 172.21.15.185:0/3034420244 learned_addr learned my addr 172.21.15.185:0/3034420244 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76049da700 1 --2- 172.21.15.185:0/3034420244 >> v2:172.21.15.96:3300/0 conn(0x7f76000b17e0 0x7f7600067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 -- 172.21.15.185:0/3034420244 >> v2:172.21.15.96:3300/0 conn(0x7f76000b17e0 msgr2=0x7f7600067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 --2- 172.21.15.185:0/3034420244 >> v2:172.21.15.96:3300/0 conn(0x7f76000b17e0 0x7f7600067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 -- 172.21.15.185:0/3034420244 >> v1:172.21.15.96:6789/0 conn(0x7f76000b2160 legacy=0x7f760014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 -- 172.21.15.185:0/3034420244 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000b3bf0 con 0x7f7600066240 2020-11-16T22:15:02.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f76051db700 1 --2- 172.21.15.185:0/3034420244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f7600065630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f76000b0fe0 tx=0x7f75f8009290).ready entity=mon.1 client_cookie=e1cdfc992dd108fb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f75ff7fe700 1 -- 172.21.15.185:0/3034420244 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800bb80 con 0x7f7600066240 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f75ff7fe700 1 -- 172.21.15.185:0/3034420244 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800bce0 con 0x7f7600066240 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f75ff7fe700 1 -- 172.21.15.185:0/3034420244 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8011610 con 0x7f7600066240 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- 172.21.15.185:0/3034420244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 msgr2=0x7f7600065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3034420244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f7600065630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f76000b0fe0 tx=0x7f75f8009290).stop 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- 172.21.15.185:0/3034420244 shutdown_connections 2020-11-16T22:15:02.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3034420244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f7600065630 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3034420244 >> v2:172.21.15.96:3300/0 conn(0x7f76000b17e0 0x7f7600067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- 172.21.15.185:0/3034420244 >> 172.21.15.185:0/3034420244 conn(0x7f76000604a0 msgr2=0x7f76000608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:02.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- 172.21.15.185:0/3034420244 shutdown_connections 2020-11-16T22:15:02.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.903+0000 7f7606c3e700 1 -- 172.21.15.185:0/3034420244 wait complete. 2020-11-16T22:15:02.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 Processor -- start 2020-11-16T22:15:02.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 -- start start 2020-11-16T22:15:02.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f760008ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 0x7f760008f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000aefa0 con 0x7f76000b17e0 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b0410 con 0x7f7600066240 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000aed00 con 0x7f76000b2160 2020-11-16T22:15:02.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76049da700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f760008ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50694/0 (socket says 172.21.15.185:50694) 2020-11-16T22:15:02.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 -- 172.21.15.185:0/3884092494 learned_addr learned my addr 172.21.15.185:0/3884092494 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:02.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76049da700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f760008ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57248/0 (socket says 172.21.15.185:57248) 2020-11-16T22:15:02.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f75fffff700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 0x7f760008f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 -- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 msgr2=0x7f760008ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f760008ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 -- 172.21.15.185:0/3884092494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 msgr2=0x7f760008f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 0x7f760008f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 -- 172.21.15.185:0/3884092494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000af4b0 con 0x7f76000b2160 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f75fffff700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 0x7f760008f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f76051db700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f75f80092d0 tx=0x7f75f8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800bb80 con 0x7f76000b2160 2020-11-16T22:15:02.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800be50 con 0x7f76000b2160 2020-11-16T22:15:02.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.907+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76000b0100 con 0x7f76000b2160 2020-11-16T22:15:02.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8010da0 con 0x7f76000b2160 2020-11-16T22:15:02.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000ae880 con 0x7f76000b2160 2020-11-16T22:15:02.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760005d4e0 con 0x7f76000b2160 2020-11-16T22:15:02.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f75f8017070 con 0x7f76000b2160 2020-11-16T22:15:02.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f75fcff9700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 0x7f75d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:02.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f75f806fc80 con 0x7f76000b2160 2020-11-16T22:15:02.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f76049da700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 0x7f75d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:02.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.911+0000 7f76049da700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 0x7f75d8057520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f76000b4530 tx=0x7f75f0006f90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:02.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:02.919+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f75f8046100 con 0x7f76000b2160 2020-11-16T22:15:03.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.075+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7600063240 con 0x7f76000b2160 2020-11-16T22:15:03.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.075+0000 7f75fcff9700 1 -- 172.21.15.185:0/3884092494 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+143 (secure 0 0 0) 0x7f75f8015030 con 0x7f76000b2160 2020-11-16T22:15:03.062 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:03.062 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:15:03.062 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:15:03.062 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:03.063 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:15:03.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 msgr2=0x7f75d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 0x7f75d8057520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f76000b4530 tx=0x7f75f0006f90).stop 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 msgr2=0x7f76000ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f75f80092d0 tx=0x7f75f8016000).stop 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 shutdown_connections 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75d8055060 0x7f75d8057520 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7600066240 0x7f760008ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76000b2160 0x7f76000ad530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 --2- 172.21.15.185:0/3884092494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76000b17e0 0x7f760008f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 >> 172.21.15.185:0/3884092494 conn(0x7f76000604a0 msgr2=0x7f76000b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:03.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 shutdown_connections 2020-11-16T22:15:03.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.079+0000 7f7606c3e700 1 -- 172.21.15.185:0/3884092494 wait complete. 2020-11-16T22:15:03.084 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:15:03.085 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:15:03.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 Processor -- start 2020-11-16T22:15:03.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 -- start start 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 0x7fcba0143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba00b7290 con 0x7fcba00b4470 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba00b4bf0 con 0x7fcba0143470 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba6158700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00b74c0 con 0x7fcba00b5d70 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5957700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 0x7fcba0143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5957700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 0x7fcba0143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57260/0 (socket says 172.21.15.185:57260) 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50706/0 (socket says 172.21.15.185:50706) 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5957700 1 -- 172.21.15.185:0/2482680444 learned_addr learned my addr 172.21.15.185:0/2482680444 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 -- 172.21.15.185:0/2482680444 >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 msgr2=0x7fcba0143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 --2- 172.21.15.185:0/2482680444 >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 0x7fcba0143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 -- 172.21.15.185:0/2482680444 >> v1:172.21.15.96:6789/0 conn(0x7fcba00b4470 legacy=0x7fcba00b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 -- 172.21.15.185:0/2482680444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00b7760 con 0x7fcba00b5d70 2020-11-16T22:15:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcba5156700 1 --2- 172.21.15.185:0/2482680444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcb9c00b9a0 tx=0x7fcb9c006df0).ready entity=mon.2 client_cookie=d23a210483b640ee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.215+0000 7fcb93fff700 1 -- 172.21.15.185:0/2482680444 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0097b0 con 0x7fcba00b5d70 2020-11-16T22:15:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcb93fff700 1 -- 172.21.15.185:0/2482680444 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c009910 con 0x7fcba00b5d70 2020-11-16T22:15:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcb93fff700 1 -- 172.21.15.185:0/2482680444 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c009c90 con 0x7fcba00b5d70 2020-11-16T22:15:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- 172.21.15.185:0/2482680444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 msgr2=0x7fcba00ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- 172.21.15.185:0/2482680444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcb9c00b9a0 tx=0x7fcb9c006df0).stop 2020-11-16T22:15:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- 172.21.15.185:0/2482680444 shutdown_connections 2020-11-16T22:15:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- 172.21.15.185:0/2482680444 >> v2:172.21.15.185:3300/0 conn(0x7fcba0143470 0x7fcba0143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- 172.21.15.185:0/2482680444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b5d70 0x7fcba00ba800 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- 172.21.15.185:0/2482680444 >> 172.21.15.185:0/2482680444 conn(0x7fcba005ba80 msgr2=0x7fcba005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- 172.21.15.185:0/2482680444 shutdown_connections 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- 172.21.15.185:0/2482680444 wait complete. 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 Processor -- start 2020-11-16T22:15:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- start start 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 0x7fcba0149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0143470 0x7fcba0148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00ae0b0 con 0x7fcba0143470 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00b0130 con 0x7fcba00b5d70 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba6158700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00ae510 con 0x7fcba00b4470 2020-11-16T22:15:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba5156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.219+0000 7fcba5156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50708/0 (socket says 172.21.15.185:50708) 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 -- 172.21.15.185:0/3235594062 learned_addr learned my addr 172.21.15.185:0/3235594062 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba4955700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 0x7fcba0149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5957700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0143470 0x7fcba0148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 -- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 msgr2=0x7fcba0149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 0x7fcba0149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 -- 172.21.15.185:0/3235594062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0143470 msgr2=0x7fcba0148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0143470 0x7fcba0148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 -- 172.21.15.185:0/3235594062 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00ad000 con 0x7fcba00b4470 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba4955700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 0x7fcba0149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba5156700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fcb9c006e30 tx=0x7fcb9c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c011d50 con 0x7fcba00b4470 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c01e460 con 0x7fcba00b4470 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00aff00 con 0x7fcba00b4470 2020-11-16T22:15:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c01e7e0 con 0x7fcba00b4470 2020-11-16T22:15:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0140a70 con 0x7fcba00b4470 2020-11-16T22:15:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcb9c025030 con 0x7fcba00b4470 2020-11-16T22:15:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba005d200 con 0x7fcba00b4470 2020-11-16T22:15:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 0x7fcb78057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcb9c06efe0 con 0x7fcba00b4470 2020-11-16T22:15:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba4955700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 0x7fcb78057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.223+0000 7fcba4955700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 0x7fcb78057520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fcba00b1150 tx=0x7fcb94009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.231+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcb9c045620 con 0x7fcba00b4470 2020-11-16T22:15:03.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:03.387+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcba005c720 con 0x7fcba00b4470 2020-11-16T22:15:04.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.247+0000 7fcb917fa700 1 -- 172.21.15.185:0/3235594062 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v28) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcb9c023030 con 0x7fcba00b4470 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.247+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 msgr2=0x7fcb78057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.247+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 0x7fcb78057520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fcba00b1150 tx=0x7fcb94009040).stop 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.247+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 msgr2=0x7fcba0149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.247+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fcb9c006e30 tx=0x7fcb9c015000).stop 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 shutdown_connections 2020-11-16T22:15:04.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcb78055060 0x7fcb78057520 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0143470 0x7fcba0148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcba00b4470 0x7fcba0149670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 --2- 172.21.15.185:0/3235594062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcba00b5d70 0x7fcba0149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 >> 172.21.15.185:0/3235594062 conn(0x7fcba005ba80 msgr2=0x7fcba0142440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 shutdown_connections 2020-11-16T22:15:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.251+0000 7fcba6158700 1 -- 172.21.15.185:0/3235594062 wait complete. 2020-11-16T22:15:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:15:04.251 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:15:04.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.395+0000 7ff74a6bc700 1 Processor -- start 2020-11-16T22:15:04.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.395+0000 7ff74a6bc700 1 -- start start 2020-11-16T22:15:04.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff74413de10 0x7ff7440666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff74413e1f0 0x7ff744143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff7440b3570 0x7ff7440b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b3a80 con 0x7ff74413e1f0 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b3d20 con 0x7ff7440b3570 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff74a6bc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440b3fc0 con 0x7ff74413de10 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff7440b3570 0x7ff7440b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff7496ba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff74413de10 0x7ff7440666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff7440b3570 0x7ff7440b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57276/0 (socket says 172.21.15.185:57276) 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 -- 172.21.15.185:0/3657888496 learned_addr learned my addr 172.21.15.185:0/3657888496 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff748eb9700 1 --2- 172.21.15.185:0/3657888496 >> v2:172.21.15.96:3300/0 conn(0x7ff74413e1f0 0x7ff744143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 -- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff74413de10 msgr2=0x7ff7440666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 --2- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff74413de10 0x7ff7440666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 -- 172.21.15.185:0/3657888496 >> v2:172.21.15.96:3300/0 conn(0x7ff74413e1f0 msgr2=0x7ff744143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 --2- 172.21.15.185:0/3657888496 >> v2:172.21.15.96:3300/0 conn(0x7ff74413e1f0 0x7ff744143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 -- 172.21.15.185:0/3657888496 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440b4260 con 0x7ff7440b3570 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff749ebb700 1 --2- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff7440b3570 0x7ff7440b7510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff74406a070 tx=0x7ff73c009290).ready entity=mon.1 client_cookie=6431b299e4c3dbe3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff733fff700 1 -- 172.21.15.185:0/3657888496 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00bbc0 con 0x7ff7440b3570 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff733fff700 1 -- 172.21.15.185:0/3657888496 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00bd20 con 0x7ff7440b3570 2020-11-16T22:15:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.399+0000 7ff733fff700 1 -- 172.21.15.185:0/3657888496 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011610 con 0x7ff7440b3570 2020-11-16T22:15:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff7440b3570 msgr2=0x7ff7440b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff7440b3570 0x7ff7440b7510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff74406a070 tx=0x7ff73c009290).stop 2020-11-16T22:15:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3657888496 shutdown_connections 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff7440b3570 0x7ff7440b7510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3657888496 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff74413de10 0x7ff7440666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3657888496 >> v2:172.21.15.96:3300/0 conn(0x7ff74413e1f0 0x7ff744143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3657888496 >> 172.21.15.185:0/3657888496 conn(0x7ff744064b90 msgr2=0x7ff744063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3657888496 shutdown_connections 2020-11-16T22:15:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3657888496 wait complete. 2020-11-16T22:15:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 Processor -- start 2020-11-16T22:15:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.403+0000 7ff74a6bc700 1 -- start start 2020-11-16T22:15:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 0x7ff7440aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 0x7ff7441429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440afb60 con 0x7ff7441433b0 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff744154120 con 0x7ff74413de10 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7441543c0 con 0x7ff7440b3570 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7496ba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 0x7ff7440aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7496ba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 0x7ff7440aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50722/0 (socket says 172.21.15.185:50722) 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57280/0 (socket says 172.21.15.185:57280) 2020-11-16T22:15:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7496ba700 1 -- 172.21.15.185:0/3186651748 learned_addr learned my addr 172.21.15.185:0/3186651748 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff749ebb700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 0x7ff7441429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 -- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 msgr2=0x7ff7440aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 0x7ff7440aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 -- 172.21.15.185:0/3186651748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 msgr2=0x7ff7441429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 0x7ff7441429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 -- 172.21.15.185:0/3186651748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440afd50 con 0x7ff74413de10 2020-11-16T22:15:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff749ebb700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 0x7ff7441429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff748eb9700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7ff7440b19f0 tx=0x7ff734008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73400f5e0 con 0x7ff74413de10 2020-11-16T22:15:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73400f740 con 0x7ff74413de10 2020-11-16T22:15:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73400fae0 con 0x7ff74413de10 2020-11-16T22:15:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7440af190 con 0x7ff74413de10 2020-11-16T22:15:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7440af7a0 con 0x7ff74413de10 2020-11-16T22:15:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.407+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff7340123e0 con 0x7ff74413de10 2020-11-16T22:15:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.411+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744154580 con 0x7ff74413de10 2020-11-16T22:15:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.411+0000 7ff7317fa700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.411+0000 7ff7496ba700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.411+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5607+0+0 (secure 0 0 0) 0x7ff73406d510 con 0x7ff74413de10 2020-11-16T22:15:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.411+0000 7ff7496ba700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 0x7ff71c057520 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff744143b30 tx=0x7ff740006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:04.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.419+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff734043990 con 0x7ff74413de10 2020-11-16T22:15:04.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:04.571+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff7440afee0 con 0x7ff74413de10 2020-11-16T22:15:05.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.259+0000 7ff7317fa700 1 -- 172.21.15.185:0/3186651748 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v29) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff734046fb0 con 0x7ff74413de10 2020-11-16T22:15:05.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 msgr2=0x7ff71c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 0x7ff71c057520 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff744143b30 tx=0x7ff740006df0).stop 2020-11-16T22:15:05.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 msgr2=0x7ff7441424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7ff7440b19f0 tx=0x7ff734008040).stop 2020-11-16T22:15:05.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 shutdown_connections 2020-11-16T22:15:05.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff71c055060 0x7ff71c057520 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7441433b0 0x7ff7441429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff7440b3570 0x7ff7440aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 --2- 172.21.15.185:0/3186651748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff74413de10 0x7ff7441424f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 >> 172.21.15.185:0/3186651748 conn(0x7ff744064b90 msgr2=0x7ff7440ac1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 shutdown_connections 2020-11-16T22:15:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.263+0000 7ff74a6bc700 1 -- 172.21.15.185:0/3186651748 wait complete. 2020-11-16T22:15:05.253 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:15:05.267 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:15:05.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 Processor -- start 2020-11-16T22:15:05.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 -- start start 2020-11-16T22:15:05.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f74080b7550 0x7f7408148160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080ad370 con 0x7f74080b7170 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7408144170 con 0x7f74080acee0 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d932700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7408144f80 con 0x7f74080b7550 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f7407fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f74080b7550 0x7f7408148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d131700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f74080acee0 0x7f740813dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50728/0 (socket says 172.21.15.185:50728) 2020-11-16T22:15:05.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.407+0000 7f740d131700 1 -- 172.21.15.185:0/2035263654 learned_addr learned my addr 172.21.15.185:0/2035263654 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:05.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f7407fff700 1 -- 172.21.15.185:0/2035263654 >> v1:172.21.15.185:6789/0 conn(0x7f74080acee0 legacy=0x7f740813dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f7407fff700 1 -- 172.21.15.185:0/2035263654 >> v1:172.21.15.96:6789/0 conn(0x7f74080b7170 legacy=0x7f7408144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f7407fff700 1 -- 172.21.15.185:0/2035263654 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080b6ed0 con 0x7f74080b7550 2020-11-16T22:15:05.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f7407fff700 1 --2- 172.21.15.185:0/2035263654 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f7408148160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f73f800b9a0 tx=0x7f73f8006df0).ready entity=mon.1 client_cookie=4a3051ce2bd5623e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f74077fe700 1 -- 172.21.15.185:0/2035263654 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80097b0 con 0x7f74080b7550 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f74077fe700 1 -- 172.21.15.185:0/2035263654 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8009910 con 0x7f74080b7550 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f74077fe700 1 -- 172.21.15.185:0/2035263654 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8009c90 con 0x7f74080b7550 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 -- 172.21.15.185:0/2035263654 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 msgr2=0x7f7408148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 --2- 172.21.15.185:0/2035263654 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f7408148160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f73f800b9a0 tx=0x7f73f8006df0).stop 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 -- 172.21.15.185:0/2035263654 shutdown_connections 2020-11-16T22:15:05.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 --2- 172.21.15.185:0/2035263654 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f7408148160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 -- 172.21.15.185:0/2035263654 >> 172.21.15.185:0/2035263654 conn(0x7f7408067860 msgr2=0x7f7408067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:05.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 -- 172.21.15.185:0/2035263654 shutdown_connections 2020-11-16T22:15:05.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.411+0000 7f740d932700 1 -- 172.21.15.185:0/2035263654 wait complete. 2020-11-16T22:15:05.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 Processor -- start 2020-11-16T22:15:05.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- start start 2020-11-16T22:15:05.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74080b7170 0x7f74080b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f74080b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080b0e20 con 0x7f74080b7170 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408147210 con 0x7f74080b7550 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74081479c0 con 0x7f74080acee0 2020-11-16T22:15:05.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d131700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f74080b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50736/0 (socket says 172.21.15.185:50736) 2020-11-16T22:15:05.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 -- 172.21.15.185:0/3122518784 learned_addr learned my addr 172.21.15.185:0/3122518784 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:05.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f7407fff700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74080b7170 0x7f74080b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 -- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 msgr2=0x7f74080b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f74080b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 -- 172.21.15.185:0/3122518784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74080b7170 msgr2=0x7f74080b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74080b7170 0x7f74080b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 -- 172.21.15.185:0/3122518784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408147df0 con 0x7f74080acee0 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740c930700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f73fc00c710 tx=0x7f73fc007bf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00e490 con 0x7f74080acee0 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc00e5f0 con 0x7f74080acee0 2020-11-16T22:15:05.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74080b0b80 con 0x7f74080acee0 2020-11-16T22:15:05.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00e970 con 0x7f74080acee0 2020-11-16T22:15:05.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.415+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740813a1e0 con 0x7f74080acee0 2020-11-16T22:15:05.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080b2b70 con 0x7f74080acee0 2020-11-16T22:15:05.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f73fc019030 con 0x7f74080acee0 2020-11-16T22:15:05.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f7404ff9700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 0x7f73e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f7407fff700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 0x7f73e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f73fc06cfe0 con 0x7f74080acee0 2020-11-16T22:15:05.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.419+0000 7f7407fff700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 0x7f73e0057520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f73f800b9a0 tx=0x7f73f8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.427+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f73fc043320 con 0x7f74080acee0 2020-11-16T22:15:05.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.627+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f74081468e0 con 0x7f74080acee0 2020-11-16T22:15:05.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.639+0000 7f7404ff9700 1 -- 172.21.15.185:0/3122518784 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v15) v1 ==== 187+0+0 (secure 0 0 0) 0x7f74081468e0 con 0x7f74080acee0 2020-11-16T22:15:05.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 msgr2=0x7f73e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 0x7f73e0057520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f73f800b9a0 tx=0x7f73f8015000).stop 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 msgr2=0x7f7408139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f73fc00c710 tx=0x7f73fc007bf0).stop 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 shutdown_connections 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f73e0055060 0x7f73e0057520 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0x7f73f800b9a0 tx=0x7f73f8015000).stop 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f74080b7550 0x7f74080b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f74080acee0 0x7f7408139cf0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0x7f73fc00c710 tx=0x7f73fc007bf0).stop 2020-11-16T22:15:05.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 --2- 172.21.15.185:0/3122518784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74080b7170 0x7f74080b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 >> 172.21.15.185:0/3122518784 conn(0x7f7408067860 msgr2=0x7f74080b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:05.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.643+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 shutdown_connections 2020-11-16T22:15:05.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.647+0000 7f740d932700 1 -- 172.21.15.185:0/3122518784 wait complete. 2020-11-16T22:15:05.640 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 5 and data pool 6 2020-11-16T22:15:05.652 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:15:05.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 Processor -- start 2020-11-16T22:15:05.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 -- start start 2020-11-16T22:15:05.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 0x7f2b900b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 0x7f2b900acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b90147f90 con 0x7f2b900b72e0 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900af020 con 0x7f2b900b6aa0 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b96fe8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900b4810 con 0x7f2b900b64b0 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b95fe6700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2b900b72e0 0x7f2b90144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50738/0 (socket says 172.21.15.185:50738) 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b95fe6700 1 -- 172.21.15.185:0/2229302106 learned_addr learned my addr 172.21.15.185:0/2229302106 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:05.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 --2- 172.21.15.185:0/2229302106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 0x7f2b900b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b967e7700 1 --2- 172.21.15.185:0/2229302106 >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 0x7f2b900acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b94fe4700 1 -- 172.21.15.185:0/2229302106 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470316024 0 0) 0x7f2b90147f90 con 0x7f2b900b72e0 2020-11-16T22:15:05.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b94fe4700 1 -- 172.21.15.185:0/2229302106 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b74003240 con 0x7f2b900b72e0 2020-11-16T22:15:05.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 -- 172.21.15.185:0/2229302106 >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 msgr2=0x7f2b900acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:05.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 --2- 172.21.15.185:0/2229302106 >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 0x7f2b900acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 -- 172.21.15.185:0/2229302106 >> v1:172.21.15.185:6789/0 conn(0x7f2b900b72e0 legacy=0x7f2b90144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 -- 172.21.15.185:0/2229302106 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90148460 con 0x7f2b900b64b0 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b967e7700 1 --2- 172.21.15.185:0/2229302106 >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 0x7f2b900acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b957e5700 1 --2- 172.21.15.185:0/2229302106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 0x7f2b900b76c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2b8000b9a0 tx=0x7f2b80006df0).ready entity=mon.2 client_cookie=66a1fa1c0dd0519d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b94fe4700 1 -- 172.21.15.185:0/2229302106 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80009680 con 0x7f2b900b64b0 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b94fe4700 1 -- 172.21.15.185:0/2229302106 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b800097e0 con 0x7f2b900b64b0 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.779+0000 7f2b94fe4700 1 -- 172.21.15.185:0/2229302106 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80009b60 con 0x7f2b900b64b0 2020-11-16T22:15:05.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- 172.21.15.185:0/2229302106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 msgr2=0x7f2b900b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/2229302106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 0x7f2b900b76c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2b8000b9a0 tx=0x7f2b80006df0).stop 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- 172.21.15.185:0/2229302106 shutdown_connections 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/2229302106 >> v2:172.21.15.185:3300/0 conn(0x7f2b900b6aa0 0x7f2b900acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/2229302106 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b64b0 0x7f2b900b76c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- 172.21.15.185:0/2229302106 >> 172.21.15.185:0/2229302106 conn(0x7f2b90067860 msgr2=0x7f2b90067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- 172.21.15.185:0/2229302106 shutdown_connections 2020-11-16T22:15:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- 172.21.15.185:0/2229302106 wait complete. 2020-11-16T22:15:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 Processor -- start 2020-11-16T22:15:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.783+0000 7f2b96fe8700 1 -- start start 2020-11-16T22:15:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 0x7f2b901574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b72e0 0x7f2b901533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900b2cf0 con 0x7f2b900b64b0 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90156ad0 con 0x7f2b900b6aa0 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90156d70 con 0x7f2b900b72e0 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57308/0 (socket says 172.21.15.185:57308) 2020-11-16T22:15:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 -- 172.21.15.185:0/1589387625 learned_addr learned my addr 172.21.15.185:0/1589387625 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b967e7700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b72e0 0x7f2b901533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b95fe6700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 0x7f2b901574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 -- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b72e0 msgr2=0x7f2b901533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b72e0 0x7f2b901533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 -- 172.21.15.185:0/1589387625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 msgr2=0x7f2b901574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 0x7f2b901574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 -- 172.21.15.185:0/1589387625 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90157010 con 0x7f2b900b6aa0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b95fe6700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 0x7f2b901574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b957e5700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f2b80006e30 tx=0x7f2b80015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80011c40 con 0x7f2b900b6aa0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b80011da0 con 0x7f2b900b6aa0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b900b2650 con 0x7f2b900b6aa0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001e690 con 0x7f2b900b6aa0 2020-11-16T22:15:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b90140950 con 0x7f2b900b6aa0 2020-11-16T22:15:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b901571a0 con 0x7f2b900b6aa0 2020-11-16T22:15:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.787+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2b80025030 con 0x7f2b900b6aa0 2020-11-16T22:15:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.791+0000 7f2b85ffb700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 0x7f2b70057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.791+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b8006f280 con 0x7f2b900b6aa0 2020-11-16T22:15:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.791+0000 7f2b95fe6700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 0x7f2b70057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.791+0000 7f2b95fe6700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 0x7f2b70057550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2b880016a0 tx=0x7f2b88009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.795+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2b800454e0 con 0x7f2b900b6aa0 2020-11-16T22:15:05.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.947+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b9005d200 con 0x7f2b900b6aa0 2020-11-16T22:15:05.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.947+0000 7f2b85ffb700 1 -- 172.21.15.185:0/1589387625 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16852 (secure 0 0 0) 0x7f2b80023090 con 0x7f2b900b6aa0 2020-11-16T22:15:05.936 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:05.936 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":30,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:05.634943+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:15:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 msgr2=0x7f2b70057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 0x7f2b70057550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2b880016a0 tx=0x7f2b88009040).stop 2020-11-16T22:15:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 msgr2=0x7f2b90143290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f2b80006e30 tx=0x7f2b80015000).stop 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 shutdown_connections 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b70055090 0x7f2b70057550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2b900b72e0 0x7f2b901533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b900b64b0 0x7f2b901574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 --2- 172.21.15.185:0/1589387625 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2b900b6aa0 0x7f2b90143290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 >> 172.21.15.185:0/1589387625 conn(0x7f2b90067860 msgr2=0x7f2b900b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 shutdown_connections 2020-11-16T22:15:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:05.951+0000 7f2b96fe8700 1 -- 172.21.15.185:0/1589387625 wait complete. 2020-11-16T22:15:06.141 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:15:06.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 Processor -- start 2020-11-16T22:15:06.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- start start 2020-11-16T22:15:06.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f18f40b30e0 0x7f18f40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40af130 con 0x7f18f40b8820 2020-11-16T22:15:06.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40af600 con 0x7f18f4141b30 2020-11-16T22:15:06.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40af360 con 0x7f18f40b30e0 2020-11-16T22:15:06.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fa59d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f18f40b30e0 0x7f18f40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18f959b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f18f4141b30 0x7f18f41454e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50752/0 (socket says 172.21.15.185:50752) 2020-11-16T22:15:06.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18f959b700 1 -- 172.21.15.185:0/918951618 learned_addr learned my addr 172.21.15.185:0/918951618 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:06.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fa59d700 1 -- 172.21.15.185:0/918951618 >> v1:172.21.15.185:6789/0 conn(0x7f18f4141b30 legacy=0x7f18f41454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fa59d700 1 -- 172.21.15.185:0/918951618 >> v1:172.21.15.96:6789/0 conn(0x7f18f40b8820 legacy=0x7f18f4141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fa59d700 1 -- 172.21.15.185:0/918951618 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40b3dc0 con 0x7f18f40b30e0 2020-11-16T22:15:06.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fa59d700 1 --2- 172.21.15.185:0/918951618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b30e0 0x7f18f40af7e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f18ec009c90 tx=0x7f18ec0091d0).ready entity=mon.1 client_cookie=808061b3d71d67bd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18f8d9a700 1 -- 172.21.15.185:0/918951618 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00bbc0 con 0x7f18f40b30e0 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18f8d9a700 1 -- 172.21.15.185:0/918951618 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00bd20 con 0x7f18f40b30e0 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18f8d9a700 1 -- 172.21.15.185:0/918951618 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011610 con 0x7f18f40b30e0 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- 172.21.15.185:0/918951618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b30e0 msgr2=0x7f18f40af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 --2- 172.21.15.185:0/918951618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b30e0 0x7f18f40af7e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f18ec009c90 tx=0x7f18ec0091d0).stop 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- 172.21.15.185:0/918951618 shutdown_connections 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 --2- 172.21.15.185:0/918951618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b30e0 0x7f18f40af7e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- 172.21.15.185:0/918951618 >> 172.21.15.185:0/918951618 conn(0x7f18f40645d0 msgr2=0x7f18f40649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:06.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- 172.21.15.185:0/918951618 shutdown_connections 2020-11-16T22:15:06.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.287+0000 7f18fad9e700 1 -- 172.21.15.185:0/918951618 wait complete. 2020-11-16T22:15:06.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 Processor -- start 2020-11-16T22:15:06.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- start start 2020-11-16T22:15:06.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 0x7f18f40b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 0x7f18f413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 0x7f18f413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41119d0 con 0x7f18f40b1890 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4111c70 con 0x7f18f40b8820 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4111f10 con 0x7f18f40b30e0 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f9d9c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 0x7f18f40b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 0x7f18f413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f9d9c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 0x7f18f40b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50764/0 (socket says 172.21.15.185:50764) 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f9d9c700 1 -- 172.21.15.185:0/2290199547 learned_addr learned my addr 172.21.15.185:0/2290199547 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:06.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fa59d700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 0x7f18f413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 -- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 msgr2=0x7f18f40b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 0x7f18f40b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 -- 172.21.15.185:0/2290199547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 msgr2=0x7f18f413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 0x7f18f413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 -- 172.21.15.185:0/2290199547 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f4155750 con 0x7f18f40b8820 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fa59d700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 0x7f18f413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:06.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18f959b700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 0x7f18f413d1f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f18e400c6e0 tx=0x7f18e4007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40072e0 con 0x7f18f40b8820 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4007440 con 0x7f18f40b8820 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f40abf80 con 0x7f18f40b8820 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4005800 con 0x7f18f40b8820 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.291+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f40acbf0 con 0x7f18f40b8820 2020-11-16T22:15:06.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f405d200 con 0x7f18f40b8820 2020-11-16T22:15:06.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f18e4010070 con 0x7f18f40b8820 2020-11-16T22:15:06.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18e1ffb700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 0x7f18cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f18e406d4d0 con 0x7f18f40b8820 2020-11-16T22:15:06.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18f9d9c700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 0x7f18cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.295+0000 7f18f9d9c700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 0x7f18cc057520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f18f0003300 tx=0x7f18f0000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.299+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f18e4043620 con 0x7f18f40b8820 2020-11-16T22:15:06.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.451+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18f4068500 con 0x7f18f40b8820 2020-11-16T22:15:06.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18e1ffb700 1 -- 172.21.15.185:0/2290199547 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16852 (secure 0 0 0) 0x7f18e4046c40 con 0x7f18f40b8820 2020-11-16T22:15:06.441 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:06.441 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":30,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:05.634943+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:15:06.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 msgr2=0x7f18cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 0x7f18cc057520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f18f0003300 tx=0x7f18f0000b80).stop 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 msgr2=0x7f18f413d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 0x7f18f413d1f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f18e400c6e0 tx=0x7f18e4007b80).stop 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 shutdown_connections 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f18cc055060 0x7f18cc057520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f40b1890 0x7f18f413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f18f40b30e0 0x7f18f40b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 --2- 172.21.15.185:0/2290199547 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f18f40b8820 0x7f18f413d1f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 >> 172.21.15.185:0/2290199547 conn(0x7f18f40645d0 msgr2=0x7f18f41411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:06.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 shutdown_connections 2020-11-16T22:15:06.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.455+0000 7f18fad9e700 1 -- 172.21.15.185:0/2290199547 wait complete. 2020-11-16T22:15:06.646 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:15:06.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 Processor -- start 2020-11-16T22:15:06.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 -- start start 2020-11-16T22:15:06.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 0x7fbf900b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf90144490 con 0x7fbf90144000 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf901446e0 con 0x7fbf900b5c40 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf9492d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90144980 con 0x7fbf900ba800 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 0x7fbf900b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbf900b5c40 0x7fbf9013dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50768/0 (socket says 172.21.15.185:50768) 2020-11-16T22:15:06.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8f7fe700 1 -- 172.21.15.185:0/3832209055 learned_addr learned my addr 172.21.15.185:0/3832209055 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8e7fc700 1 -- 172.21.15.185:0/3832209055 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193402924 0 0) 0x7fbf901446e0 con 0x7fbf900b5c40 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8e7fc700 1 -- 172.21.15.185:0/3832209055 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf70003240 con 0x7fbf900b5c40 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8effd700 1 -- 172.21.15.185:0/3832209055 >> v1:172.21.15.185:6789/0 conn(0x7fbf900b5c40 legacy=0x7fbf9013dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8effd700 1 -- 172.21.15.185:0/3832209055 >> v1:172.21.15.96:6789/0 conn(0x7fbf90144000 legacy=0x7fbf900b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8effd700 1 -- 172.21.15.185:0/3832209055 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90144c20 con 0x7fbf900ba800 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8effd700 1 --2- 172.21.15.185:0/3832209055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 0x7fbf900b6020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbf8000b9d0 tx=0x7fbf80006df0).ready entity=mon.2 client_cookie=da00e186f7acb621 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8e7fc700 1 -- 172.21.15.185:0/3832209055 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80009770 con 0x7fbf900ba800 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.787+0000 7fbf8e7fc700 1 -- 172.21.15.185:0/3832209055 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf800098d0 con 0x7fbf900ba800 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8e7fc700 1 -- 172.21.15.185:0/3832209055 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80009c50 con 0x7fbf900ba800 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/3832209055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 msgr2=0x7fbf900b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 --2- 172.21.15.185:0/3832209055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 0x7fbf900b6020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbf8000b9d0 tx=0x7fbf80006df0).stop 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/3832209055 shutdown_connections 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 --2- 172.21.15.185:0/3832209055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf900ba800 0x7fbf900b6020 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/3832209055 >> 172.21.15.185:0/3832209055 conn(0x7fbf9005ba80 msgr2=0x7fbf9005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:06.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/3832209055 shutdown_connections 2020-11-16T22:15:06.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/3832209055 wait complete. 2020-11-16T22:15:06.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 Processor -- start 2020-11-16T22:15:06.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- start start 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 0x7fbf90156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 0x7fbf900b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900af5d0 con 0x7fbf900b5c40 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901576c0 con 0x7fbf900ba800 2020-11-16T22:15:06.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901462c0 con 0x7fbf90144000 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 0x7fbf900b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57336/0 (socket says 172.21.15.185:57336) 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 0x7fbf900b8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50782/0 (socket says 172.21.15.185:50782) 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 -- 172.21.15.185:0/1357883235 learned_addr learned my addr 172.21.15.185:0/1357883235 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8f7fe700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 0x7fbf90156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 -- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 msgr2=0x7fbf900b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 0x7fbf900b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 -- 172.21.15.185:0/1357883235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 msgr2=0x7fbf90156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 0x7fbf90156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 -- 172.21.15.185:0/1357883235 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90146560 con 0x7fbf900ba800 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8f7fe700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 0x7fbf90156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf8effd700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf8000b150 tx=0x7fbf80015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80011d50 con 0x7fbf900ba800 2020-11-16T22:15:06.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8001e460 con 0x7fbf900ba800 2020-11-16T22:15:06.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8001e7e0 con 0x7fbf900ba800 2020-11-16T22:15:06.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf900affc0 con 0x7fbf900ba800 2020-11-16T22:15:06.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.791+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900aefe0 con 0x7fbf900ba800 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf900b0150 con 0x7fbf900ba800 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbf80025030 con 0x7fbf900ba800 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf877fe700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 0x7fbf6c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf8006f520 con 0x7fbf900ba800 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf8f7fe700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 0x7fbf6c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:06.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.795+0000 7fbf8f7fe700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 0x7fbf6c057550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fbf780055a0 tx=0x7fbf78009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:06.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.803+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbf80045830 con 0x7fbf900ba800 2020-11-16T22:15:06.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:06.999+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbf9005d200 con 0x7fbf900ba800 2020-11-16T22:15:07.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.943+0000 7fbf877fe700 1 -- 172.21.15.185:0/1357883235 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbf80023070 con 0x7fbf900ba800 2020-11-16T22:15:07.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 msgr2=0x7fbf6c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 0x7fbf6c057550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fbf780055a0 tx=0x7fbf78009040).stop 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 msgr2=0x7fbf90145c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fbf8000b150 tx=0x7fbf80015000).stop 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 shutdown_connections 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbf6c055090 0x7fbf6c057550 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:07.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbf90144000 0x7fbf900b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:07.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf900b5c40 0x7fbf90156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:07.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 --2- 172.21.15.185:0/1357883235 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbf900ba800 0x7fbf90145c90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:07.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 >> 172.21.15.185:0/1357883235 conn(0x7fbf9005ba80 msgr2=0x7fbf90142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:07.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 shutdown_connections 2020-11-16T22:15:07.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:07.947+0000 7fbf9492d700 1 -- 172.21.15.185:0/1357883235 wait complete. 2020-11-16T22:15:07.948 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:08.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.083+0000 7fdd9731c700 1 Processor -- start 2020-11-16T22:15:08.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.083+0000 7fdd9731c700 1 -- start start 2020-11-16T22:15:08.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 0x7fdd900b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd900682b0 0x7fdd90143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd90067510 con 0x7fdd90067070 2020-11-16T22:15:08.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900677b0 con 0x7fdd900682b0 2020-11-16T22:15:08.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900b17a0 con 0x7fdd900b7470 2020-11-16T22:15:08.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 0x7fdd900b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95b19700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fdd90067070 0x7fdd900b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50782/0 (socket says 172.21.15.185:50782) 2020-11-16T22:15:08.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95b19700 1 -- 172.21.15.185:0/3699510877 learned_addr learned my addr 172.21.15.185:0/3699510877 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd96b1b700 1 --2- 172.21.15.185:0/3699510877 >> v2:172.21.15.96:3300/0 conn(0x7fdd900682b0 0x7fdd90143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95318700 1 -- 172.21.15.185:0/3699510877 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341159017 0 0) 0x7fdd90067510 con 0x7fdd90067070 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95318700 1 -- 172.21.15.185:0/3699510877 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7c003240 con 0x7fdd90067070 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 -- 172.21.15.185:0/3699510877 >> v2:172.21.15.96:3300/0 conn(0x7fdd900682b0 msgr2=0x7fdd90143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 --2- 172.21.15.185:0/3699510877 >> v2:172.21.15.96:3300/0 conn(0x7fdd900682b0 0x7fdd90143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 -- 172.21.15.185:0/3699510877 >> v1:172.21.15.185:6789/0 conn(0x7fdd90067070 legacy=0x7fdd900b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 -- 172.21.15.185:0/3699510877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd900b1a40 con 0x7fdd900b7470 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9631a700 1 --2- 172.21.15.185:0/3699510877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 0x7fdd900b7850 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fdd8000b9a0 tx=0x7fdd80006df0).ready entity=mon.2 client_cookie=ebf0b4f7e032fbd7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95318700 1 -- 172.21.15.185:0/3699510877 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80009680 con 0x7fdd900b7470 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95318700 1 -- 172.21.15.185:0/3699510877 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800097e0 con 0x7fdd900b7470 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd95318700 1 -- 172.21.15.185:0/3699510877 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80009b60 con 0x7fdd900b7470 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- 172.21.15.185:0/3699510877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 msgr2=0x7fdd900b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 --2- 172.21.15.185:0/3699510877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 0x7fdd900b7850 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fdd8000b9a0 tx=0x7fdd80006df0).stop 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- 172.21.15.185:0/3699510877 shutdown_connections 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 --2- 172.21.15.185:0/3699510877 >> v2:172.21.15.96:3300/0 conn(0x7fdd900682b0 0x7fdd90143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 --2- 172.21.15.185:0/3699510877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd900b7470 0x7fdd900b7850 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- 172.21.15.185:0/3699510877 >> 172.21.15.185:0/3699510877 conn(0x7fdd900637b0 msgr2=0x7fdd900b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- 172.21.15.185:0/3699510877 shutdown_connections 2020-11-16T22:15:08.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.087+0000 7fdd9731c700 1 -- 172.21.15.185:0/3699510877 wait complete. 2020-11-16T22:15:08.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 Processor -- start 2020-11-16T22:15:08.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 -- start start 2020-11-16T22:15:08.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd90067070 0x7fdd900b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd900b7470 0x7fdd90151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd901547b0 con 0x7fdd900b7470 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90155610 con 0x7fdd900682b0 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9731c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900b6340 con 0x7fdd90067070 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd9631a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd90067070 0x7fdd900b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57350/0 (socket says 172.21.15.185:57350) 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 -- 172.21.15.185:0/2436905092 learned_addr learned my addr 172.21.15.185:0/2436905092 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:08.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd96b1b700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd900b7470 0x7fdd90151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 -- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd90067070 msgr2=0x7fdd900b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd90067070 0x7fdd900b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 -- 172.21.15.185:0/2436905092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd900b7470 msgr2=0x7fdd90151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd900b7470 0x7fdd90151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 -- 172.21.15.185:0/2436905092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd901549e0 con 0x7fdd900682b0 2020-11-16T22:15:08.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd95b19700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fdd8400c710 tx=0x7fdd840079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.091+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840072e0 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84007440 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd90154c80 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84005820 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd901518f0 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9005d200 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdd8401a060 con 0x7fdd900682b0 2020-11-16T22:15:08.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd8e7fc700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 0x7fdd68057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd8406d430 con 0x7fdd900682b0 2020-11-16T22:15:08.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd9631a700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 0x7fdd68057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.095+0000 7fdd9631a700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 0x7fdd68057520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fdd8000c010 tx=0x7fdd80015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.103+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdd84043690 con 0x7fdd900682b0 2020-11-16T22:15:08.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.303+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd9005c6d0 con 0x7fdd900682b0 2020-11-16T22:15:08.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd8e7fc700 1 -- 172.21.15.185:0/2436905092 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2484 (secure 0 0 0) 0x7fdd84018070 con 0x7fdd900682b0 2020-11-16T22:15:08.292 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:08.292 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":14},{"gid":4927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:07.936187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:08.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 msgr2=0x7fdd68057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 0x7fdd68057520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fdd8000c010 tx=0x7fdd80015000).stop 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 msgr2=0x7fdd90155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fdd8400c710 tx=0x7fdd840079e0).stop 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 shutdown_connections 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdd68055060 0x7fdd68057520 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd900b7470 0x7fdd90151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdd90067070 0x7fdd900b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 --2- 172.21.15.185:0/2436905092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdd900682b0 0x7fdd90155120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 >> 172.21.15.185:0/2436905092 conn(0x7fdd900637b0 msgr2=0x7fdd900abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 shutdown_connections 2020-11-16T22:15:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.307+0000 7fdd9731c700 1 -- 172.21.15.185:0/2436905092 wait complete. 2020-11-16T22:15:08.296 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 18 2020-11-16T22:15:08.308 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:15:08.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 Processor -- start 2020-11-16T22:15:08.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- start start 2020-11-16T22:15:08.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 0x7fb5f8145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f8061e60 con 0x7fb5f80b3090 2020-11-16T22:15:08.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f8060ef0 con 0x7fb5f80b3b80 2020-11-16T22:15:08.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8062090 con 0x7fb5f80b1840 2020-11-16T22:15:08.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fce59700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb5f80b3b80 0x7fb5f81421d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50798/0 (socket says 172.21.15.185:50798) 2020-11-16T22:15:08.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fce59700 1 -- 172.21.15.185:0/2385902330 learned_addr learned my addr 172.21.15.185:0/2385902330 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:08.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fde5b700 1 --2- 172.21.15.185:0/2385902330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 0x7fb5f8145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5e7fff700 1 -- 172.21.15.185:0/2385902330 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316514084 0 0) 0x7fb5f8060ef0 con 0x7fb5f80b3b80 2020-11-16T22:15:08.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5e7fff700 1 -- 172.21.15.185:0/2385902330 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e0003240 con 0x7fb5f80b3b80 2020-11-16T22:15:08.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fde5b700 1 -- 172.21.15.185:0/2385902330 >> v1:172.21.15.185:6789/0 conn(0x7fb5f80b3b80 legacy=0x7fb5f81421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fde5b700 1 -- 172.21.15.185:0/2385902330 >> v1:172.21.15.96:6789/0 conn(0x7fb5f80b3090 legacy=0x7fb5f81453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fde5b700 1 -- 172.21.15.185:0/2385902330 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80ad640 con 0x7fb5f80b1840 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fde5b700 1 --2- 172.21.15.185:0/2385902330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 0x7fb5f8145ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb5f0009cc0 tx=0x7fb5f00091d0).ready entity=mon.2 client_cookie=212613a4910f805a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5e7fff700 1 -- 172.21.15.185:0/2385902330 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000b9d0 con 0x7fb5f80b1840 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5e7fff700 1 -- 172.21.15.185:0/2385902330 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000bb30 con 0x7fb5f80b1840 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5e7fff700 1 -- 172.21.15.185:0/2385902330 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000beb0 con 0x7fb5f80b1840 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- 172.21.15.185:0/2385902330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 msgr2=0x7fb5f8145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/2385902330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 0x7fb5f8145ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb5f0009cc0 tx=0x7fb5f00091d0).stop 2020-11-16T22:15:08.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- 172.21.15.185:0/2385902330 shutdown_connections 2020-11-16T22:15:08.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/2385902330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b1840 0x7fb5f8145ac0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- 172.21.15.185:0/2385902330 >> 172.21.15.185:0/2385902330 conn(0x7fb5f8060130 msgr2=0x7fb5f8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:08.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- 172.21.15.185:0/2385902330 shutdown_connections 2020-11-16T22:15:08.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.447+0000 7fb5fe65c700 1 -- 172.21.15.185:0/2385902330 wait complete. 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 Processor -- start 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- start start 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 0x7fb5f813c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b3090 0x7fb5f813cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8145a40 con 0x7fb5f80b1840 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8154f50 con 0x7fb5f80b3b80 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8145c70 con 0x7fb5f80b3090 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fce59700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b3090 0x7fb5f813cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57366/0 (socket says 172.21.15.185:57366) 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 -- 172.21.15.185:0/1065781622 learned_addr learned my addr 172.21.15.185:0/1065781622 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:08.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fd65a700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 0x7fb5f813c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 -- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b3090 msgr2=0x7fb5f813cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b3090 0x7fb5f813cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 -- 172.21.15.185:0/1065781622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 msgr2=0x7fb5f813c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 0x7fb5f813c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 -- 172.21.15.185:0/1065781622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f81551f0 con 0x7fb5f80b3b80 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fd65a700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 0x7fb5f813c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:08.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fde5b700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb5f000a010 tx=0x7fb5f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000b9d0 con 0x7fb5f80b3b80 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f001d660 con 0x7fb5f80b3b80 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f80afb90 con 0x7fb5f80b3b80 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f001da00 con 0x7fb5f80b3b80 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f8145780 con 0x7fb5f80b3b80 2020-11-16T22:15:08.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.451+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f8154d80 con 0x7fb5f80b3b80 2020-11-16T22:15:08.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.455+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb5f002d070 con 0x7fb5f80b3b80 2020-11-16T22:15:08.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.455+0000 7fb5e57fa700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 0x7fb5d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.455+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb5f00704b0 con 0x7fb5f80b3b80 2020-11-16T22:15:08.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.455+0000 7fb5fd65a700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 0x7fb5d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.455+0000 7fb5fd65a700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 0x7fb5d0057520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb5f4000f20 tx=0x7fb5f4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.459+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb5f0046710 con 0x7fb5f80b3b80 2020-11-16T22:15:08.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5f8145e30 con 0x7fb5f80b3b80 2020-11-16T22:15:08.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5e57fa700 1 -- 172.21.15.185:0/1065781622 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fb5f0021090 con 0x7fb5f80b3b80 2020-11-16T22:15:08.598 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:08.598 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":32,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:07.653440+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:15:08.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 msgr2=0x7fb5d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 0x7fb5d0057520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fb5f4000f20 tx=0x7fb5f4009040).stop 2020-11-16T22:15:08.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 msgr2=0x7fb5f8110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb5f000a010 tx=0x7fb5f0016000).stop 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 shutdown_connections 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.611+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5d0055060 0x7fb5d0057520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5f80b3b80 0x7fb5f8110770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f80b1840 0x7fb5f813c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 --2- 172.21.15.185:0/1065781622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5f80b3090 0x7fb5f813cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 >> 172.21.15.185:0/1065781622 conn(0x7fb5f8060130 msgr2=0x7fb5f80b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:08.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 shutdown_connections 2020-11-16T22:15:08.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.615+0000 7fb5fe65c700 1 -- 172.21.15.185:0/1065781622 wait complete. 2020-11-16T22:15:08.805 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:15:08.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.963+0000 7f97a977d700 1 Processor -- start 2020-11-16T22:15:08.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.963+0000 7f97a977d700 1 -- start start 2020-11-16T22:15:08.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97a4067f70 0x7f97a4068350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a41446c0 con 0x7f97a40b60f0 2020-11-16T22:15:08.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a4144960 con 0x7f97a4143ff0 2020-11-16T22:15:08.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a4144c00 con 0x7f97a4067f70 2020-11-16T22:15:08.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a8f7c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f97a4143ff0 0x7f97a40b7590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50810/0 (socket says 172.21.15.185:50810) 2020-11-16T22:15:08.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a8f7c700 1 -- 172.21.15.185:0/3995659255 learned_addr learned my addr 172.21.15.185:0/3995659255 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:08.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a3fff700 1 --2- 172.21.15.185:0/3995659255 >> v2:172.21.15.96:3300/0 conn(0x7f97a4067f70 0x7f97a4068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808251160 0 0) 0x7f97a4144960 con 0x7f97a4143ff0 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9790003240 con 0x7f97a4143ff0 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2215380336 0 0) 0x7f9790003240 con 0x7f97a4143ff0 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a3fff700 1 -- 172.21.15.185:0/3995659255 >> v1:172.21.15.185:6789/0 conn(0x7f97a4143ff0 legacy=0x7f97a40b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a3fff700 1 -- 172.21.15.185:0/3995659255 >> v1:172.21.15.96:6789/0 conn(0x7f97a40b60f0 legacy=0x7f97a413dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a3fff700 1 -- 172.21.15.185:0/3995659255 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a413e460 con 0x7f97a4067f70 2020-11-16T22:15:08.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a3fff700 1 --2- 172.21.15.185:0/3995659255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4067f70 0x7f97a4068350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f979400b920 tx=0x7f9794006df0).ready entity=mon.0 client_cookie=883b4a5ab83224f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c001860 con 0x7f97a4067f70 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794011400 con 0x7f97a4067f70 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a2ffd700 1 -- 172.21.15.185:0/3995659255 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794011780 con 0x7f97a4067f70 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- 172.21.15.185:0/3995659255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4067f70 msgr2=0x7f97a4068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 --2- 172.21.15.185:0/3995659255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4067f70 0x7f97a4068350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f979400b920 tx=0x7f9794006df0).stop 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- 172.21.15.185:0/3995659255 shutdown_connections 2020-11-16T22:15:08.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 --2- 172.21.15.185:0/3995659255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4067f70 0x7f97a4068350 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- 172.21.15.185:0/3995659255 >> 172.21.15.185:0/3995659255 conn(0x7f97a4061eb0 msgr2=0x7f97a4067860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:08.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- 172.21.15.185:0/3995659255 shutdown_connections 2020-11-16T22:15:08.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.967+0000 7f97a977d700 1 -- 172.21.15.185:0/3995659255 wait complete. 2020-11-16T22:15:08.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 Processor -- start 2020-11-16T22:15:08.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- start start 2020-11-16T22:15:08.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 0x7f97a4149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4143ff0 0x7f97a4142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:08.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a413f130 con 0x7f97a4143ff0 2020-11-16T22:15:08.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4156e10 con 0x7f97a4067f70 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a413fc90 con 0x7f97a40b60f0 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 0x7f97a4149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50822/0 (socket says 172.21.15.185:50822) 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 0x7f97a4149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57376/0 (socket says 172.21.15.185:57376) 2020-11-16T22:15:08.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 -- 172.21.15.185:0/3607449051 learned_addr learned my addr 172.21.15.185:0/3607449051 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a8f7c700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4143ff0 0x7f97a4142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 -- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 msgr2=0x7f97a4149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 0x7f97a4149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 -- 172.21.15.185:0/3607449051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4143ff0 msgr2=0x7f97a4142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4143ff0 0x7f97a4142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 -- 172.21.15.185:0/3607449051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a413ff30 con 0x7f97a40b60f0 2020-11-16T22:15:08.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a37fe700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f979800c710 tx=0x7f9798007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:08.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800e490 con 0x7f97a40b60f0 2020-11-16T22:15:08.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979800e5f0 con 0x7f97a40b60f0 2020-11-16T22:15:08.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a4140230 con 0x7f97a40b60f0 2020-11-16T22:15:08.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800e990 con 0x7f97a40b60f0 2020-11-16T22:15:08.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.971+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a414a340 con 0x7f97a40b60f0 2020-11-16T22:15:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a405d200 con 0x7f97a40b60f0 2020-11-16T22:15:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9798019030 con 0x7f97a40b60f0 2020-11-16T22:15:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f9783fff700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 0x7f977c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f979806d020 con 0x7f97a40b60f0 2020-11-16T22:15:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f97a3fff700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 0x7f977c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.975+0000 7f97a3fff700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 0x7f977c057520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f979400b150 tx=0x7f9794015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:09.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:08.983+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9798043280 con 0x7f97a40b60f0 2020-11-16T22:15:09.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.191+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f97a40684a0 con 0x7f97a40b60f0 2020-11-16T22:15:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.199+0000 7f9783fff700 1 -- 172.21.15.185:0/3607449051 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f97980468a0 con 0x7f97a40b60f0 2020-11-16T22:15:09.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 msgr2=0x7f977c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 0x7f977c057520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f979400b150 tx=0x7f9794015000).stop 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 msgr2=0x7f97a4157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f979800c710 tx=0x7f9798007c10).stop 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 shutdown_connections 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f977c055060 0x7f977c057520 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97a4067f70 0x7f97a4149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97a4143ff0 0x7f97a4142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 --2- 172.21.15.185:0/3607449051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97a40b60f0 0x7f97a4157510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 >> 172.21.15.185:0/3607449051 conn(0x7f97a4061eb0 msgr2=0x7f97a4144fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:09.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 shutdown_connections 2020-11-16T22:15:09.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.203+0000 7f97a977d700 1 -- 172.21.15.185:0/3607449051 wait complete. 2020-11-16T22:15:09.193 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:15:09.206 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:09.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.343+0000 7f83b3fff700 1 Processor -- start 2020-11-16T22:15:09.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.343+0000 7f83b3fff700 1 -- start start 2020-11-16T22:15:09.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83b41443d0 0x7f83b4141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40b1fb0 con 0x7f83b40adc20 2020-11-16T22:15:09.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b4144840 con 0x7f83b41443d0 2020-11-16T22:15:09.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b405c0d0 con 0x7f83b40b3f20 2020-11-16T22:15:09.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83b41443d0 0x7f83b4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83b41443d0 0x7f83b4141270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57388/0 (socket says 172.21.15.185:57388) 2020-11-16T22:15:09.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 -- 172.21.15.185:0/4092477415 learned_addr learned my addr 172.21.15.185:0/4092477415 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:09.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 -- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 msgr2=0x7f83b40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:09.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 --2- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b40b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 -- 172.21.15.185:0/4092477415 >> v1:172.21.15.96:6789/0 conn(0x7f83b40adc20 legacy=0x7f83b40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 -- 172.21.15.185:0/4092477415 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40ad5d0 con 0x7f83b41443d0 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b37fe700 1 --2- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b4141270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f83ac009d50 tx=0x7f83ac009290).ready entity=mon.1 client_cookie=979f2571442c4e61 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b27fc700 1 -- 172.21.15.185:0/4092477415 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00bbc0 con 0x7f83b41443d0 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b27fc700 1 -- 172.21.15.185:0/4092477415 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac00bd20 con 0x7f83b41443d0 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b27fc700 1 -- 172.21.15.185:0/4092477415 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac011610 con 0x7f83b41443d0 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 msgr2=0x7f83b4141270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 --2- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b4141270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f83ac009d50 tx=0x7f83ac009290).stop 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- 172.21.15.185:0/4092477415 shutdown_connections 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 --2- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b4141270 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 --2- 172.21.15.185:0/4092477415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- 172.21.15.185:0/4092477415 >> 172.21.15.185:0/4092477415 conn(0x7f83b40b7840 msgr2=0x7f83b40b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- 172.21.15.185:0/4092477415 shutdown_connections 2020-11-16T22:15:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.347+0000 7f83b3fff700 1 -- 172.21.15.185:0/4092477415 wait complete. 2020-11-16T22:15:09.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.351+0000 7f83b3fff700 1 Processor -- start 2020-11-16T22:15:09.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.351+0000 7f83b3fff700 1 -- start start 2020-11-16T22:15:09.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83b40adc20 0x7f83b4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b4141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4147ee0 con 0x7f83b40adc20 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4148180 con 0x7f83b41443d0 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4148420 con 0x7f83b40b3f20 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83aa7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b4141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83aa7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b4141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50836/0 (socket says 172.21.15.185:50836) 2020-11-16T22:15:09.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57394/0 (socket says 172.21.15.185:57394) 2020-11-16T22:15:09.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83aa7fc700 1 -- 172.21.15.185:0/3012687369 learned_addr learned my addr 172.21.15.185:0/3012687369 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:09.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b2ffd700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83b40adc20 0x7f83b4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 -- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 msgr2=0x7f83b4141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b4141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 -- 172.21.15.185:0/3012687369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83b40adc20 msgr2=0x7f83b4153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83b40adc20 0x7f83b4153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 -- 172.21.15.185:0/3012687369 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b41486c0 con 0x7f83b41443d0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b37fe700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f83ac009d50 tx=0x7f83ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00bbc0 con 0x7f83b41443d0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac00be90 con 0x7f83b41443d0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac010cf0 con 0x7f83b41443d0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b40af490 con 0x7f83b41443d0 2020-11-16T22:15:09.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b414a0c0 con 0x7f83b41443d0 2020-11-16T22:15:09.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.355+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b405d200 con 0x7f83b41443d0 2020-11-16T22:15:09.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.359+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f83ac017070 con 0x7f83b41443d0 2020-11-16T22:15:09.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.359+0000 7f83ab7fe700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 0x7f8390057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:09.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.359+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f83ac06feb0 con 0x7f83b41443d0 2020-11-16T22:15:09.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.359+0000 7f83b2ffd700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 0x7f8390057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:09.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.359+0000 7f83b2ffd700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 0x7f8390057520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f83a4006fd0 tx=0x7f83a4006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:09.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.367+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f83ac046080 con 0x7f83b41443d0 2020-11-16T22:15:09.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83b405c6d0 con 0x7f83b41443d0 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83ab7fe700 1 -- 172.21.15.185:0/3012687369 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2484 (secure 0 0 0) 0x7f83ac015070 con 0x7f83b41443d0 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:09.557 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":14},{"gid":4927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:07.936187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:09.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 msgr2=0x7f8390057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 0x7f8390057520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f83a4006fd0 tx=0x7f83a4006df0).stop 2020-11-16T22:15:09.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 msgr2=0x7f83b413eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:09.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f83ac009d50 tx=0x7f83ac016000).stop 2020-11-16T22:15:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 shutdown_connections 2020-11-16T22:15:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8390055060 0x7f8390057520 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83b41443d0 0x7f83b413eca0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83b40adc20 0x7f83b4153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 --2- 172.21.15.185:0/3012687369 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83b40b3f20 0x7f83b4141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 >> 172.21.15.185:0/3012687369 conn(0x7f83b40b7840 msgr2=0x7f83b4142620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 shutdown_connections 2020-11-16T22:15:09.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:09.571+0000 7f83b3fff700 1 -- 172.21.15.185:0/3012687369 wait complete. 2020-11-16T22:15:09.562 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 18 2020-11-16T22:15:09.875 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:15:05.635355+0000', 'modified': '2020-11-16T22:15:07.936187+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4932': {'gid': 4932, 'name': 'b', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2155443116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2155443116}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2155443116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:15:09.876 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:15:09.876 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:15:10.019 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4932,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3890152069999999,"mdsmap_epoch":18,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":11.800348938000001} 2020-11-16T22:15:10.019 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4932, "mdsmap_epoch": 18, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 4.389015207, "state": "up:active", "uptime": 11.800348938, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:15:10.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:15:10.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:15:10.021 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:15:10.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:15:10.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:15:10.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:15:10.022 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:15:10.022 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:15:10.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.023 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:15:10.043 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:15:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:15:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:15:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.043 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:15:10.044 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.045 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:15:10.046 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:15:10.047 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:15:10.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.048 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:15:10.054 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:15:10.070+0000 7f9813ff6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:15:10.132 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:15:10.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.135 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:15:10.152 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:15:10.152 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:15:10.152 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:15:10.152 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:15:10.152 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:15:10.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.153 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:15:10.154 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:15:10.154 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:15:10.154 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:15:10.155 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:15:10.155 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:15:10.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.263 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:15:10.276 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:15:10.277 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:15:10.278 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:15:10.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:10.278 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:15:10.309 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:15:10.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 Processor -- start 2020-11-16T22:15:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 -- start start 2020-11-16T22:15:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 0x7f76881432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7688061e60 0x7f76881437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880abf90 con 0x7f76880669a0 2020-11-16T22:15:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76880ac230 con 0x7f7688061e60 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f7687fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76880ac4d0 con 0x7f76880acc10 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76867fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 0x7f76881432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76877fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7688061e60 0x7f76881437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76867fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 0x7f76881432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50846/0 (socket says 172.21.15.185:50846) 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76867fc700 1 -- 172.21.15.185:0/2909817337 learned_addr learned my addr 172.21.15.185:0/2909817337 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76877fe700 1 -- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 msgr2=0x7f76881432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:10.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76877fe700 1 --2- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 0x7f76881432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.451+0000 7f76877fe700 1 -- 172.21.15.185:0/2909817337 >> v1:172.21.15.96:6789/0 conn(0x7f76880669a0 legacy=0x7f7688066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f76877fe700 1 -- 172.21.15.185:0/2909817337 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880ac770 con 0x7f7688061e60 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f76877fe700 1 --2- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f76881437d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f7680009c90 tx=0x7f76800091d0).ready entity=mon.1 client_cookie=3876becd95f70d1e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7685ffb700 1 -- 172.21.15.185:0/2909817337 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768000ba30 con 0x7f7688061e60 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7685ffb700 1 -- 172.21.15.185:0/2909817337 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768000bb90 con 0x7f7688061e60 2020-11-16T22:15:10.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7685ffb700 1 -- 172.21.15.185:0/2909817337 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680011400 con 0x7f7688061e60 2020-11-16T22:15:10.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 msgr2=0x7f76881437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f76881437d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f7680009c90 tx=0x7f76800091d0).stop 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- 172.21.15.185:0/2909817337 shutdown_connections 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f76881437d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- 172.21.15.185:0/2909817337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76880acc10 0x7f76881432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- 172.21.15.185:0/2909817337 >> 172.21.15.185:0/2909817337 conn(0x7f7688060130 msgr2=0x7f7688060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- 172.21.15.185:0/2909817337 shutdown_connections 2020-11-16T22:15:10.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- 172.21.15.185:0/2909817337 wait complete. 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 Processor -- start 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- start start 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76880669a0 0x7f76881178c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 0x7f7688117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688141d10 con 0x7f76880669a0 2020-11-16T22:15:10.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688153b40 con 0x7f7688061e60 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7687fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7688153de0 con 0x7f7688113fb0 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f76877fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 0x7f7688117db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7686ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7686ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57406/0 (socket says 172.21.15.185:57406) 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f7686ffd700 1 -- 172.21.15.185:0/3026283995 learned_addr learned my addr 172.21.15.185:0/3026283995 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f76877fe700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 0x7f7688117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50852/0 (socket says 172.21.15.185:50852) 2020-11-16T22:15:10.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.455+0000 7f76867fc700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76880669a0 0x7f76881178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 -- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 msgr2=0x7f7688117db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 0x7f7688117db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 -- 172.21.15.185:0/3026283995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76880669a0 msgr2=0x7f76881178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76880669a0 0x7f76881178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 -- 172.21.15.185:0/3026283995 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688154080 con 0x7f7688061e60 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7686ffd700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f767800c730 tx=0x7f7678007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:10.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800e490 con 0x7f7688061e60 2020-11-16T22:15:10.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767800e610 con 0x7f7688061e60 2020-11-16T22:15:10.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800e9d0 con 0x7f7688061e60 2020-11-16T22:15:10.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7688154320 con 0x7f7688061e60 2020-11-16T22:15:10.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688153750 con 0x7f7688061e60 2020-11-16T22:15:10.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7678010030 con 0x7f7688061e60 2020-11-16T22:15:10.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76881182a0 con 0x7f7688061e60 2020-11-16T22:15:10.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 0x7f765c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f767806dc00 con 0x7f7688061e60 2020-11-16T22:15:10.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.459+0000 7f76867fc700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 0x7f765c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.463+0000 7f76867fc700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 0x7f765c057520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f7688114510 tx=0x7f7670008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:10.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.467+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7678073020 con 0x7f7688061e60 2020-11-16T22:15:10.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.623+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f768805d200 con 0x7f7688061e60 2020-11-16T22:15:10.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.623+0000 7f767effd700 1 -- 172.21.15.185:0/3026283995 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f7678043e60 con 0x7f7688061e60 2020-11-16T22:15:10.611 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:10.611 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":32,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:07.653440+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:15:10.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 msgr2=0x7f765c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 0x7f765c057520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f7688114510 tx=0x7f7670008040).stop 2020-11-16T22:15:10.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 msgr2=0x7f7688113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f767800c730 tx=0x7f7678007c30).stop 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 shutdown_connections 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f765c055060 0x7f765c057520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7688061e60 0x7f7688113ac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7688113fb0 0x7f7688117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 --2- 172.21.15.185:0/3026283995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76880669a0 0x7f76881178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 >> 172.21.15.185:0/3026283995 conn(0x7f7688060130 msgr2=0x7f7688066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 shutdown_connections 2020-11-16T22:15:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.627+0000 7f7687fff700 1 -- 172.21.15.185:0/3026283995 wait complete. 2020-11-16T22:15:10.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:15:03.444879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:15:04.633004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:15:10.626 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:15:10.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.767+0000 7f813359e700 1 Processor -- start 2020-11-16T22:15:10.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.767+0000 7f813359e700 1 -- start start 2020-11-16T22:15:10.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f813405bc50 0x7f8134144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:10.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340b2690 con 0x7f8134144d60 2020-11-16T22:15:10.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81340ad350 con 0x7f813405bc50 2020-11-16T22:15:10.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81340b2930 con 0x7f81340b38a0 2020-11-16T22:15:10.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f8131d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f813405bc50 0x7f8134144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:10.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50856/0 (socket says 172.21.15.185:50856) 2020-11-16T22:15:10.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 -- 172.21.15.185:0/3958176304 learned_addr learned my addr 172.21.15.185:0/3958176304 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:10.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 -- 172.21.15.185:0/3958176304 >> v2:172.21.15.185:3300/0 conn(0x7f813405bc50 msgr2=0x7f8134144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:10.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 --2- 172.21.15.185:0/3958176304 >> v2:172.21.15.185:3300/0 conn(0x7f813405bc50 0x7f8134144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:10.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 -- 172.21.15.185:0/3958176304 >> v1:172.21.15.185:6789/0 conn(0x7f8134144d60 legacy=0x7f813413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 -- 172.21.15.185:0/3958176304 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81341443d0 con 0x7f81340b38a0 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813259c700 1 --2- 172.21.15.185:0/3958176304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8124009d50 tx=0x7f8124009290).ready entity=mon.2 client_cookie=a3ba21a493079c2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813159a700 1 -- 172.21.15.185:0/3958176304 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400bbc0 con 0x7f81340b38a0 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813159a700 1 -- 172.21.15.185:0/3958176304 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812400bd20 con 0x7f81340b38a0 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813159a700 1 -- 172.21.15.185:0/3958176304 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011610 con 0x7f81340b38a0 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- 172.21.15.185:0/3958176304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 msgr2=0x7f81340b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:10.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 --2- 172.21.15.185:0/3958176304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8124009d50 tx=0x7f8124009290).stop 2020-11-16T22:15:10.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- 172.21.15.185:0/3958176304 shutdown_connections 2020-11-16T22:15:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 --2- 172.21.15.185:0/3958176304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81340b38a0 0x7f81340b2060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 --2- 172.21.15.185:0/3958176304 >> v2:172.21.15.185:3300/0 conn(0x7f813405bc50 0x7f8134144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- 172.21.15.185:0/3958176304 >> 172.21.15.185:0/3958176304 conn(0x7f8134063df0 msgr2=0x7f81340641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- 172.21.15.185:0/3958176304 shutdown_connections 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.771+0000 7f813359e700 1 -- 172.21.15.185:0/3958176304 wait complete. 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.775+0000 7f813359e700 1 Processor -- start 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.775+0000 7f813359e700 1 -- start start 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.775+0000 7f813359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f813405bc50 0x7f81340ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.775+0000 7f813359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.775+0000 7f813359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81340af650 con 0x7f813405bc50 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813413e4e0 con 0x7f81340b38a0 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813413dfa0 con 0x7f8134144d60 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8132d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8132d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50866/0 (socket says 172.21.15.185:50866) 2020-11-16T22:15:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57420/0 (socket says 172.21.15.185:57420) 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8132d9d700 1 -- 172.21.15.185:0/2527603808 learned_addr learned my addr 172.21.15.185:0/2527603808 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813259c700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f813405bc50 0x7f81340ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 -- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 msgr2=0x7f8134151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 -- 172.21.15.185:0/2527603808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f813405bc50 msgr2=0x7f81340ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f813405bc50 0x7f81340ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 -- 172.21.15.185:0/2527603808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813413e240 con 0x7f81340b38a0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8132d9d700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f8131d9b700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f81340b36d0 tx=0x7f811c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00c820 con 0x7f81340b38a0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00c980 con 0x7f81340b38a0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00cd00 con 0x7f81340b38a0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81340adb30 con 0x7f81340b38a0 2020-11-16T22:15:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8134151ee0 con 0x7f81340b38a0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813405d200 con 0x7f81340b38a0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.779+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f811c01a030 con 0x7f81340b38a0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.783+0000 7f812a7fc700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 0x7f8110057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.783+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f811c06d830 con 0x7f81340b38a0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.783+0000 7f813259c700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 0x7f8110057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.783+0000 7f813259c700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 0x7f8110057520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f8134067c80 tx=0x7f8124016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.791+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f811c043b40 con 0x7f81340b38a0 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.943+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8134060b20 con 0x7f81340b38a0 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.943+0000 7f812a7fc700 1 -- 172.21.15.185:0/2527603808 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v32) v1 ==== 110+0+10 (secure 0 0 0) 0x7f811c047160 con 0x7f81340b38a0 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.943+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 msgr2=0x7f8110057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 0x7f8110057520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f8134067c80 tx=0x7f8124016000).stop 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 msgr2=0x7f81340afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f81340b36d0 tx=0x7f811c006df0).stop 2020-11-16T22:15:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 shutdown_connections 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8110055060 0x7f8110057520 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8134144d60 0x7f8134151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f813405bc50 0x7f81340ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 --2- 172.21.15.185:0/2527603808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81340b38a0 0x7f81340afab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 >> 172.21.15.185:0/2527603808 conn(0x7f8134063df0 msgr2=0x7f81341402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 shutdown_connections 2020-11-16T22:15:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:10.947+0000 7f813359e700 1 -- 172.21.15.185:0/2527603808 wait complete. 2020-11-16T22:15:11.178 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:15:11.178 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:15:11.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.323+0000 7f4d166bf700 1 Processor -- start 2020-11-16T22:15:11.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.323+0000 7f4d166bf700 1 -- start start 2020-11-16T22:15:11.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d101421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100b2f30 con 0x7f4d100b1a20 2020-11-16T22:15:11.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10060fe0 con 0x7f4d1005ba80 2020-11-16T22:15:11.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10142dc0 con 0x7f4d10145d70 2020-11-16T22:15:11.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d15ebe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4d1005ba80 0x7f4d101426b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50868/0 (socket says 172.21.15.185:50868) 2020-11-16T22:15:11.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d15ebe700 1 -- 172.21.15.185:0/1219269001 learned_addr learned my addr 172.21.15.185:0/1219269001 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:11.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d14ebc700 1 --2- 172.21.15.185:0/1219269001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d101421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d07fff700 1 -- 172.21.15.185:0/1219269001 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320230474 0 0) 0x7f4d10060fe0 con 0x7f4d1005ba80 2020-11-16T22:15:11.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d07fff700 1 -- 172.21.15.185:0/1219269001 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cfc003240 con 0x7f4d1005ba80 2020-11-16T22:15:11.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d14ebc700 1 -- 172.21.15.185:0/1219269001 >> v1:172.21.15.185:6789/0 conn(0x7f4d1005ba80 legacy=0x7f4d101426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d14ebc700 1 -- 172.21.15.185:0/1219269001 >> v1:172.21.15.96:6789/0 conn(0x7f4d100b1a20 legacy=0x7f4d101414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d14ebc700 1 -- 172.21.15.185:0/1219269001 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10061280 con 0x7f4d10145d70 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d14ebc700 1 --2- 172.21.15.185:0/1219269001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d101421c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4d0c00b9d0 tx=0x7f4d0c006df0).ready entity=mon.2 client_cookie=ff5a85ad2298d54 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d07fff700 1 -- 172.21.15.185:0/1219269001 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0097b0 con 0x7f4d10145d70 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d07fff700 1 -- 172.21.15.185:0/1219269001 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c009910 con 0x7f4d10145d70 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d07fff700 1 -- 172.21.15.185:0/1219269001 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c009c90 con 0x7f4d10145d70 2020-11-16T22:15:11.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- 172.21.15.185:0/1219269001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 msgr2=0x7f4d101421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 --2- 172.21.15.185:0/1219269001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d101421c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4d0c00b9d0 tx=0x7f4d0c006df0).stop 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- 172.21.15.185:0/1219269001 shutdown_connections 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 --2- 172.21.15.185:0/1219269001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d101421c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- 172.21.15.185:0/1219269001 >> 172.21.15.185:0/1219269001 conn(0x7f4d10063df0 msgr2=0x7f4d100641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- 172.21.15.185:0/1219269001 shutdown_connections 2020-11-16T22:15:11.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.327+0000 7f4d166bf700 1 -- 172.21.15.185:0/1219269001 wait complete. 2020-11-16T22:15:11.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 Processor -- start 2020-11-16T22:15:11.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 -- start start 2020-11-16T22:15:11.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d100b1a20 0x7f4d101555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100b0b80 con 0x7f4d100b1a20 2020-11-16T22:15:11.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10145670 con 0x7f4d1005ba80 2020-11-16T22:15:11.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d166bf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10154e90 con 0x7f4d10145d70 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d156bd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d156bd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57432/0 (socket says 172.21.15.185:57432) 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50880/0 (socket says 172.21.15.185:50880) 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d156bd700 1 -- 172.21.15.185:0/4007900429 learned_addr learned my addr 172.21.15.185:0/4007900429 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d14ebc700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d100b1a20 0x7f4d101555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 -- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 msgr2=0x7f4d10151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 -- 172.21.15.185:0/4007900429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d100b1a20 msgr2=0x7f4d101555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d100b1a20 0x7f4d101555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 -- 172.21.15.185:0/4007900429 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10155130 con 0x7f4d10145d70 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d156bd700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:11.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.331+0000 7f4d15ebe700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f4d1005bf30 tx=0x7f4d08007ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800ce00 con 0x7f4d10145d70 2020-11-16T22:15:11.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08005b80 con 0x7f4d10145d70 2020-11-16T22:15:11.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10151c30 con 0x7f4d10145d70 2020-11-16T22:15:11.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08007110 con 0x7f4d10145d70 2020-11-16T22:15:11.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d101444c0 con 0x7f4d10145d70 2020-11-16T22:15:11.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100b7ba0 con 0x7f4d10145d70 2020-11-16T22:15:11.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4d08010030 con 0x7f4d10145d70 2020-11-16T22:15:11.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 0x7f4ce8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d0806dc20 con 0x7f4d10145d70 2020-11-16T22:15:11.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d156bd700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 0x7f4ce8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.335+0000 7f4d156bd700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 0x7f4ce8057520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f4d00001760 tx=0x7f4d00009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.347+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4d100b7ba0 con 0x7f4d10145d70 2020-11-16T22:15:11.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d100b7ba0 con 0x7f4d10145d70 2020-11-16T22:15:11.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d057fa700 1 -- 172.21.15.185:0/4007900429 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d100b7ba0 con 0x7f4d10145d70 2020-11-16T22:15:11.481 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:15:11.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 msgr2=0x7f4ce8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 0x7f4ce8057520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f4d00001760 tx=0x7f4d00009040).stop 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 msgr2=0x7f4d100b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f4d1005bf30 tx=0x7f4d08007ca0).stop 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 shutdown_connections 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ce8055060 0x7f4ce8057520 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d1005ba80 0x7f4d10151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d10145d70 0x7f4d100b4fa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 --2- 172.21.15.185:0/4007900429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d100b1a20 0x7f4d101555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 >> 172.21.15.185:0/4007900429 conn(0x7f4d10063df0 msgr2=0x7f4d101420e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 shutdown_connections 2020-11-16T22:15:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.495+0000 7f4d166bf700 1 -- 172.21.15.185:0/4007900429 wait complete. 2020-11-16T22:15:11.495 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:15:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 Processor -- start 2020-11-16T22:15:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 -- start start 2020-11-16T22:15:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e600ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2e600b13d0 0x7f2e600b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6014c660 con 0x7f2e600b3920 2020-11-16T22:15:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e600ac120 con 0x7f2e600b13d0 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e657d3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e600ac3c0 con 0x7f2e600b3080 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e600ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2e600b13d0 0x7f2e600b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2e600b13d0 0x7f2e600b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57442/0 (socket says 172.21.15.185:57442) 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 -- 172.21.15.185:0/2960910983 learned_addr learned my addr 172.21.15.185:0/2960910983 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 -- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 msgr2=0x7f2e600ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 --2- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e600ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 -- 172.21.15.185:0/2960910983 >> v1:172.21.15.96:6789/0 conn(0x7f2e600b3920 legacy=0x7f2e60147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 -- 172.21.15.185:0/2960910983 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600acd50 con 0x7f2e600b13d0 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5f7fe700 1 --2- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e600b1bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2e48006fd0 tx=0x7f2e48006eb0).ready entity=mon.1 client_cookie=bbe058f99c64742c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5effd700 1 -- 172.21.15.185:0/2960910983 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48009810 con 0x7f2e600b13d0 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5effd700 1 -- 172.21.15.185:0/2960910983 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48009970 con 0x7f2e600b13d0 2020-11-16T22:15:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.635+0000 7f2e5effd700 1 -- 172.21.15.185:0/2960910983 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48009cf0 con 0x7f2e600b13d0 2020-11-16T22:15:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 msgr2=0x7f2e600b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e600b1bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f2e48006fd0 tx=0x7f2e48006eb0).stop 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- 172.21.15.185:0/2960910983 shutdown_connections 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e600ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- 172.21.15.185:0/2960910983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e600b1bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- 172.21.15.185:0/2960910983 >> 172.21.15.185:0/2960910983 conn(0x7f2e60060130 msgr2=0x7f2e60060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- 172.21.15.185:0/2960910983 shutdown_connections 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- 172.21.15.185:0/2960910983 wait complete. 2020-11-16T22:15:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 Processor -- start 2020-11-16T22:15:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- start start 2020-11-16T22:15:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e60146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 0x7f2e600ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6010f640 con 0x7f2e600b3920 2020-11-16T22:15:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6014b360 con 0x7f2e600b13d0 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e657d3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6014b590 con 0x7f2e600b3080 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e5ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e5ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57446/0 (socket says 172.21.15.185:57446) 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.639+0000 7f2e5ffff700 1 -- 172.21.15.185:0/1695045150 learned_addr learned my addr 172.21.15.185:0/1695045150 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5f7fe700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e60146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e64fd2700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 0x7f2e600ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 -- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 msgr2=0x7f2e60146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e60146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 -- 172.21.15.185:0/1695045150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 msgr2=0x7f2e600ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 0x7f2e600ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 -- 172.21.15.185:0/1695045150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6014b830 con 0x7f2e600b13d0 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e64fd2700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 0x7f2e600ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5ffff700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2e5800ebd0 tx=0x7f2e58006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800c9b0 con 0x7f2e600b13d0 2020-11-16T22:15:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5800cb10 con 0x7f2e600b13d0 2020-11-16T22:15:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6014bad0 con 0x7f2e600b13d0 2020-11-16T22:15:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800ce90 con 0x7f2e600b13d0 2020-11-16T22:15:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e601137d0 con 0x7f2e600b13d0 2020-11-16T22:15:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6005d200 con 0x7f2e600b13d0 2020-11-16T22:15:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2e58012600 con 0x7f2e600b13d0 2020-11-16T22:15:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 0x7f2e30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e5806e230 con 0x7f2e600b13d0 2020-11-16T22:15:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5f7fe700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 0x7f2e30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.643+0000 7f2e5f7fe700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 0x7f2e30057520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f2e4800b1b0 tx=0x7f2e48015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.651+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2e58044490 con 0x7f2e600b13d0 2020-11-16T22:15:11.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.807+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e600b20e0 con 0x7f2e600b13d0 2020-11-16T22:15:11.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.807+0000 7f2e57fff700 1 -- 172.21.15.185:0/1695045150 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2e58047ab0 con 0x7f2e600b13d0 2020-11-16T22:15:11.794 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:15:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 msgr2=0x7f2e30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 0x7f2e30057520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f2e4800b1b0 tx=0x7f2e48015000).stop 2020-11-16T22:15:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 msgr2=0x7f2e60158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2e5800ebd0 tx=0x7f2e58006df0).stop 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 shutdown_connections 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2e30055060 0x7f2e30057520 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2e600b13d0 0x7f2e60158300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e600b3920 0x7f2e600ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 --2- 172.21.15.185:0/1695045150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2e600b3080 0x7f2e60146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 >> 172.21.15.185:0/1695045150 conn(0x7f2e60060130 msgr2=0x7f2e600ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 shutdown_connections 2020-11-16T22:15:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.811+0000 7f2e657d3700 1 -- 172.21.15.185:0/1695045150 wait complete. 2020-11-16T22:15:11.810 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:15:11.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.951+0000 7feb1eac7700 1 Processor -- start 2020-11-16T22:15:11.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.951+0000 7feb1eac7700 1 -- start start 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7feb180b3da0 0x7feb1813f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb18060900 con 0x7feb180ae5e0 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb180606a0 con 0x7feb18060e60 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1805bf50 con 0x7feb180b3da0 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1c863700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7feb18060e60 0x7feb18061240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50892/0 (socket says 172.21.15.185:50892) 2020-11-16T22:15:11.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1c863700 1 -- 172.21.15.185:0/1695920472 learned_addr learned my addr 172.21.15.185:0/1695920472 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:11.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1d064700 1 --2- 172.21.15.185:0/1695920472 >> v2:172.21.15.185:3300/0 conn(0x7feb180b3da0 0x7feb1813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb177fe700 1 -- 172.21.15.185:0/1695920472 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897042462 0 0) 0x7feb180606a0 con 0x7feb18060e60 2020-11-16T22:15:11.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb177fe700 1 -- 172.21.15.185:0/1695920472 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf8003240 con 0x7feb18060e60 2020-11-16T22:15:11.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1d064700 1 -- 172.21.15.185:0/1695920472 >> v1:172.21.15.185:6789/0 conn(0x7feb18060e60 legacy=0x7feb18061240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1d064700 1 -- 172.21.15.185:0/1695920472 >> v1:172.21.15.96:6789/0 conn(0x7feb180ae5e0 legacy=0x7feb181457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1d064700 1 -- 172.21.15.185:0/1695920472 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1805bcb0 con 0x7feb180b3da0 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1d064700 1 --2- 172.21.15.185:0/1695920472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb180b3da0 0x7feb1813f340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7feb0800b9a0 tx=0x7feb08006df0).ready entity=mon.1 client_cookie=eb56e1ea7cbc10d2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb177fe700 1 -- 172.21.15.185:0/1695920472 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080097b0 con 0x7feb180b3da0 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb177fe700 1 -- 172.21.15.185:0/1695920472 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb08009910 con 0x7feb180b3da0 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb177fe700 1 -- 172.21.15.185:0/1695920472 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08009c90 con 0x7feb180b3da0 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- 172.21.15.185:0/1695920472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb180b3da0 msgr2=0x7feb1813f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 --2- 172.21.15.185:0/1695920472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb180b3da0 0x7feb1813f340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7feb0800b9a0 tx=0x7feb08006df0).stop 2020-11-16T22:15:11.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- 172.21.15.185:0/1695920472 shutdown_connections 2020-11-16T22:15:11.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 --2- 172.21.15.185:0/1695920472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb180b3da0 0x7feb1813f340 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- 172.21.15.185:0/1695920472 >> 172.21.15.185:0/1695920472 conn(0x7feb18063f60 msgr2=0x7feb18064340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:11.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- 172.21.15.185:0/1695920472 shutdown_connections 2020-11-16T22:15:11.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.955+0000 7feb1eac7700 1 -- 172.21.15.185:0/1695920472 wait complete. 2020-11-16T22:15:11.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 Processor -- start 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- start start 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb180ae5e0 0x7feb18151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 0x7feb1813e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb18111640 con 0x7feb180ae5e0 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb18111820 con 0x7feb18060e60 2020-11-16T22:15:11.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb18111ac0 con 0x7feb180b3da0 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1c863700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1c863700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57460/0 (socket says 172.21.15.185:57460) 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1c863700 1 -- 172.21.15.185:0/802032688 learned_addr learned my addr 172.21.15.185:0/802032688 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 0x7feb1813e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb17fff700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb180ae5e0 0x7feb18151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 -- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 msgr2=0x7feb180afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 -- 172.21.15.185:0/802032688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb180ae5e0 msgr2=0x7feb18151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:11.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb180ae5e0 0x7feb18151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:11.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 -- 172.21.15.185:0/802032688 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18111d60 con 0x7feb180b3da0 2020-11-16T22:15:11.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1c863700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:11.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1d064700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 0x7feb1813e6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7feb08012000 tx=0x7feb08015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08006aa0 con 0x7feb180b3da0 2020-11-16T22:15:11.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0801d5d0 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb18148db0 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0801d970 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.959+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb180b03f0 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb18111ef0 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feb0801a040 con 0x7feb180b3da0 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb14ff9700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 0x7feaf4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:11.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb0806f0b0 con 0x7feb180b3da0 2020-11-16T22:15:11.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb1c863700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 0x7feaf4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:11.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.963+0000 7feb1c863700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 0x7feaf4057520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7feb10001e60 tx=0x7feb10009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:11.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:11.971+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7feb080453c0 con 0x7feb180b3da0 2020-11-16T22:15:12.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.123+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb1805d4e0 con 0x7feb180b3da0 2020-11-16T22:15:12.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.123+0000 7feb14ff9700 1 -- 172.21.15.185:0/802032688 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+10 (secure 0 0 0) 0x7feb08023070 con 0x7feb180b3da0 2020-11-16T22:15:12.111 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:15:12.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 msgr2=0x7feaf4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 0x7feaf4057520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7feb10001e60 tx=0x7feb10009040).stop 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 msgr2=0x7feb1813e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 0x7feb1813e6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7feb08012000 tx=0x7feb08015000).stop 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 shutdown_connections 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feaf4055060 0x7feaf4057520 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feb18060e60 0x7feb180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feb180b3da0 0x7feb1813e6d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 --2- 172.21.15.185:0/802032688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb180ae5e0 0x7feb18151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 >> 172.21.15.185:0/802032688 conn(0x7feb18063f60 msgr2=0x7feb18145720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 shutdown_connections 2020-11-16T22:15:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.127+0000 7feb1eac7700 1 -- 172.21.15.185:0/802032688 wait complete. 2020-11-16T22:15:12.125 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:15:12.126 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:12.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 Processor -- start 2020-11-16T22:15:12.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 -- start start 2020-11-16T22:15:12.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 0x7f8b54062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:12.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b54067930 con 0x7f8b54067de0 2020-11-16T22:15:12.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540632e0 con 0x7f8b541446c0 2020-11-16T22:15:12.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b5ba0f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54066ae0 con 0x7f8b54062900 2020-11-16T22:15:12.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b58faa700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f8b541446c0 0x7f8b54067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50906/0 (socket says 172.21.15.185:50906) 2020-11-16T22:15:12.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b597ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 0x7f8b54062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:12.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.259+0000 7f8b58faa700 1 -- 172.21.15.185:0/2519345342 learned_addr learned my addr 172.21.15.185:0/2519345342 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:12.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b597ab700 1 -- 172.21.15.185:0/2519345342 >> v1:172.21.15.185:6789/0 conn(0x7f8b541446c0 legacy=0x7f8b54067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b597ab700 1 -- 172.21.15.185:0/2519345342 >> v1:172.21.15.96:6789/0 conn(0x7f8b54067de0 legacy=0x7f8b541408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b43fff700 1 -- 172.21.15.185:0/2519345342 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748631314 0 0) 0x7f8b540632e0 con 0x7f8b541446c0 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b597ab700 1 -- 172.21.15.185:0/2519345342 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54066d80 con 0x7f8b54062900 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b597ab700 1 --2- 172.21.15.185:0/2519345342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 0x7f8b54062ce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8b50009cc0 tx=0x7f8b500091d0).ready entity=mon.2 client_cookie=a41cb11b9eeef9fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b43fff700 1 -- 172.21.15.185:0/2519345342 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000bbc0 con 0x7f8b54062900 2020-11-16T22:15:12.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b43fff700 1 -- 172.21.15.185:0/2519345342 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5000bd20 con 0x7f8b54062900 2020-11-16T22:15:12.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b43fff700 1 -- 172.21.15.185:0/2519345342 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50011610 con 0x7f8b54062900 2020-11-16T22:15:12.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/2519345342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 msgr2=0x7f8b54062ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/2519345342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 0x7f8b54062ce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8b50009cc0 tx=0x7f8b500091d0).stop 2020-11-16T22:15:12.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/2519345342 shutdown_connections 2020-11-16T22:15:12.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/2519345342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b54062900 0x7f8b54062ce0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/2519345342 >> 172.21.15.185:0/2519345342 conn(0x7f8b540b4410 msgr2=0x7f8b54060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:12.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/2519345342 shutdown_connections 2020-11-16T22:15:12.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.263+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/2519345342 wait complete. 2020-11-16T22:15:12.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 Processor -- start 2020-11-16T22:15:12.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- start start 2020-11-16T22:15:12.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b54062900 0x7f8b541538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:12.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b54067de0 0x7f8b54153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5413ad50 con 0x7f8b54067de0 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b541542c0 con 0x7f8b54062900 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54091290 con 0x7f8b541446c0 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:12.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b597ab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b54062900 0x7f8b541538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:12.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50918/0 (socket says 172.21.15.185:50918) 2020-11-16T22:15:12.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 -- 172.21.15.185:0/4035169419 learned_addr learned my addr 172.21.15.185:0/4035169419 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:12.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b58faa700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b54067de0 0x7f8b54153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:12.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 -- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b54062900 msgr2=0x7f8b541538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b54062900 0x7f8b541538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 -- 172.21.15.185:0/4035169419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b54067de0 msgr2=0x7f8b54153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b54067de0 0x7f8b54153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 -- 172.21.15.185:0/4035169419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5413afb0 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b59fac700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f8b4c00c710 tx=0x7f8b4c007b20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00e490 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c00e5f0 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5413b340 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00e970 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.267+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b54090c90 con 0x7f8b541446c0 2020-11-16T22:15:12.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b54153380 con 0x7f8b541446c0 2020-11-16T22:15:12.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8b4c019030 con 0x7f8b541446c0 2020-11-16T22:15:12.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b417fa700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 0x7f8b2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:12.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b597ab700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 0x7f8b2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:12.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b4c06d070 con 0x7f8b541446c0 2020-11-16T22:15:12.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.271+0000 7f8b597ab700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 0x7f8b2c057520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8b50009210 tx=0x7f8b50016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:12.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.279+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8b4c043380 con 0x7f8b541446c0 2020-11-16T22:15:12.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b5405d4e0 con 0x7f8b541446c0 2020-11-16T22:15:12.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b417fa700 1 -- 172.21.15.185:0/4035169419 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2484 (secure 0 0 0) 0x7f8b4c0469a0 con 0x7f8b541446c0 2020-11-16T22:15:12.465 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:12.465 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":14},{"gid":4927,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:07.936187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:12.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 msgr2=0x7f8b2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 0x7f8b2c057520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8b50009210 tx=0x7f8b50016000).stop 2020-11-16T22:15:12.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 msgr2=0x7f8b5408f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:12.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f8b4c00c710 tx=0x7f8b4c007b20).stop 2020-11-16T22:15:12.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 shutdown_connections 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b2c055060 0x7f8b2c057520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b54062900 0x7f8b541538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b541446c0 0x7f8b5408f010 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 --2- 172.21.15.185:0/4035169419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b54067de0 0x7f8b54153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 >> 172.21.15.185:0/4035169419 conn(0x7f8b540b4410 msgr2=0x7f8b54060fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 shutdown_connections 2020-11-16T22:15:12.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:12.479+0000 7f8b5ba0f700 1 -- 172.21.15.185:0/4035169419 wait complete. 2020-11-16T22:15:12.469 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 18 2020-11-16T22:15:12.788 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:15:05.635355+0000', 'modified': '2020-11-16T22:15:07.936187+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4932': {'gid': 4932, 'name': 'b', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2155443116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2155443116}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2155443116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:15:12.788 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:15:12.789 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:15:12.875 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:12.881 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:12.883 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:12.931 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4932,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.301123392,"mdsmap_epoch":18,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":14.712452514000001} 2020-11-16T22:15:12.931 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4932, "mdsmap_epoch": 18, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 7.301123392, "state": "up:active", "uptime": 14.712452514, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:15:12.932 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:15:12.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:12.932 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:15:12.944 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:15:12.944 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:15:13.085 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:15:13.085 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:15:13.112 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:15:13.112 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:15:13.119 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:15:13.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:13.120 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:15:13.149 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:15:13.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.299+0000 7f3bdd2ea700 1 Processor -- start 2020-11-16T22:15:13.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.299+0000 7f3bdd2ea700 1 -- start start 2020-11-16T22:15:13.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3bd80afc60 0x7f3bd80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd80ad490 con 0x7f3bd8144860 2020-11-16T22:15:13.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd80ad730 con 0x7f3bd80b7440 2020-11-16T22:15:13.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd80ad9d0 con 0x7f3bd80afc60 2020-11-16T22:15:13.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd67fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3bd80afc60 0x7f3bd80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd6ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3bd80b7440 0x7f3bd813dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50922/0 (socket says 172.21.15.185:50922) 2020-11-16T22:15:13.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd6ffd700 1 -- 172.21.15.185:0/3700354550 learned_addr learned my addr 172.21.15.185:0/3700354550 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:13.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd67fc700 1 -- 172.21.15.185:0/3700354550 >> v1:172.21.15.185:6789/0 conn(0x7f3bd80b7440 legacy=0x7f3bd813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd67fc700 1 -- 172.21.15.185:0/3700354550 >> v1:172.21.15.96:6789/0 conn(0x7f3bd8144860 legacy=0x7f3bd80ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd5ffb700 1 -- 172.21.15.185:0/3700354550 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685806571 0 0) 0x7f3bd80ad730 con 0x7f3bd80b7440 2020-11-16T22:15:13.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd67fc700 1 -- 172.21.15.185:0/3700354550 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd813e5b0 con 0x7f3bd80afc60 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd67fc700 1 --2- 172.21.15.185:0/3700354550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd80afc60 0x7f3bd80b7820 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f3bc000b9a0 tx=0x7f3bc0006df0).ready entity=mon.1 client_cookie=f08d2cb7c6b4de12 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd5ffb700 1 -- 172.21.15.185:0/3700354550 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00097b0 con 0x7f3bd80afc60 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd5ffb700 1 -- 172.21.15.185:0/3700354550 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0009910 con 0x7f3bd80afc60 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bd5ffb700 1 -- 172.21.15.185:0/3700354550 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0009c90 con 0x7f3bd80afc60 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/3700354550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd80afc60 msgr2=0x7f3bd80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/3700354550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd80afc60 0x7f3bd80b7820 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f3bc000b9a0 tx=0x7f3bc0006df0).stop 2020-11-16T22:15:13.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/3700354550 shutdown_connections 2020-11-16T22:15:13.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/3700354550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd80afc60 0x7f3bd80b7820 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/3700354550 >> 172.21.15.185:0/3700354550 conn(0x7f3bd8067860 msgr2=0x7f3bd8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:13.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/3700354550 shutdown_connections 2020-11-16T22:15:13.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.303+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/3700354550 wait complete. 2020-11-16T22:15:13.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 Processor -- start 2020-11-16T22:15:13.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- start start 2020-11-16T22:15:13.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 0x7f3bd8147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd8144860 0x7f3bd80b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80b1a20 con 0x7f3bd80b7440 2020-11-16T22:15:13.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80b2bc0 con 0x7f3bd8144860 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80b1c50 con 0x7f3bd80afc60 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50932/0 (socket says 172.21.15.185:50932) 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 -- 172.21.15.185:0/1978569726 learned_addr learned my addr 172.21.15.185:0/1978569726 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd77fe700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd8144860 0x7f3bd80b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd67fc700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 0x7f3bd8147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 -- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd8144860 msgr2=0x7f3bd80b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd8144860 0x7f3bd80b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 -- 172.21.15.185:0/1978569726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 msgr2=0x7f3bd8147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 0x7f3bd8147170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 -- 172.21.15.185:0/1978569726 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80b1ef0 con 0x7f3bd80afc60 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd67fc700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 0x7f3bd8147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:13.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bd6ffd700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3bc800c760 tx=0x7f3bc8007a60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80072e0 con 0x7f3bd80afc60 2020-11-16T22:15:13.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8007440 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd80b2190 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd813fec0 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.307+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8005840 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd805d4e0 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3bc801a060 con 0x7f3bd80afc60 2020-11-16T22:15:13.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bceffd700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 0x7f3bb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bd67fc700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 0x7f3bb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bc806d530 con 0x7f3bd80afc60 2020-11-16T22:15:13.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.311+0000 7f3bd67fc700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 0x7f3bb4057520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f3bc0006e30 tx=0x7f3bc0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.315+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3bc8043790 con 0x7f3bd80afc60 2020-11-16T22:15:13.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.519+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f3bd80605a0 con 0x7f3bd80afc60 2020-11-16T22:15:13.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.691+0000 7f3bceffd700 1 -- 172.21.15.185:0/1978569726 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3bc8018070 con 0x7f3bd80afc60 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 msgr2=0x7f3bb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 0x7f3bb4057520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f3bc0006e30 tx=0x7f3bc0015000).stop 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 msgr2=0x7f3bd81568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3bc800c760 tx=0x7f3bc8007a60).stop 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 shutdown_connections 2020-11-16T22:15:13.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bb4055060 0x7f3bb4057520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd8144860 0x7f3bd80b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd80afc60 0x7f3bd81568f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 --2- 172.21.15.185:0/1978569726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd80b7440 0x7f3bd8147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 >> 172.21.15.185:0/1978569726 conn(0x7f3bd8067860 msgr2=0x7f3bd80add90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 shutdown_connections 2020-11-16T22:15:13.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.695+0000 7f3bdd2ea700 1 -- 172.21.15.185:0/1978569726 wait complete. 2020-11-16T22:15:13.699 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:13.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.827+0000 7f2fe940f700 1 Processor -- start 2020-11-16T22:15:13.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.827+0000 7f2fe940f700 1 -- start start 2020-11-16T22:15:13.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 0x7f2fe4143380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40b3020 con 0x7f2fe40b7610 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe40b3280 con 0x7f2fe4142f30 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40b3520 con 0x7f2fe4067910 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50940/0 (socket says 172.21.15.185:50940) 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3045057849 learned_addr learned my addr 172.21.15.185:0/3045057849 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:13.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe8c0e700 1 --2- 172.21.15.185:0/3045057849 >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 0x7f2fe4143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3045057849 >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 msgr2=0x7f2fe4143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 --2- 172.21.15.185:0/3045057849 >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 0x7f2fe4143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3045057849 >> v1:172.21.15.96:6789/0 conn(0x7f2fe40b7610 legacy=0x7f2fe41441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3045057849 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4067f40 con 0x7f2fe4067910 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe8c0e700 1 --2- 172.21.15.185:0/3045057849 >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 0x7f2fe4143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fdb7fe700 1 --2- 172.21.15.185:0/3045057849 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f2fcc006fd0 tx=0x7f2fcc006eb0).ready entity=mon.2 client_cookie=73d79b4e50cdcf43 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe37fe700 1 -- 172.21.15.185:0/3045057849 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc009890 con 0x7f2fe4067910 2020-11-16T22:15:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe37fe700 1 -- 172.21.15.185:0/3045057849 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0099f0 con 0x7f2fe4067910 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe37fe700 1 -- 172.21.15.185:0/3045057849 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc009d70 con 0x7f2fe4067910 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- 172.21.15.185:0/3045057849 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 msgr2=0x7f2fe4144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3045057849 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f2fcc006fd0 tx=0x7f2fcc006eb0).stop 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- 172.21.15.185:0/3045057849 shutdown_connections 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3045057849 >> v2:172.21.15.185:3300/0 conn(0x7f2fe4142f30 0x7f2fe4143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3045057849 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe4067910 0x7f2fe4144900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- 172.21.15.185:0/3045057849 >> 172.21.15.185:0/3045057849 conn(0x7f2fe40637b0 msgr2=0x7f2fe40b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- 172.21.15.185:0/3045057849 shutdown_connections 2020-11-16T22:15:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- 172.21.15.185:0/3045057849 wait complete. 2020-11-16T22:15:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 Processor -- start 2020-11-16T22:15:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 -- start start 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 0x7f2fe4149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.831+0000 7f2fe940f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2fe4142f30 0x7f2fe40ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4141780 con 0x7f2fe4067910 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40abef0 con 0x7f2fe4142f30 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40ac190 con 0x7f2fe40b7610 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50948/0 (socket says 172.21.15.185:50948) 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3832187352 learned_addr learned my addr 172.21.15.185:0/3832187352 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe8c0e700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2fe4142f30 0x7f2fe40ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe3fff700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 0x7f2fe4149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2fe4142f30 msgr2=0x7f2fe40ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2fe4142f30 0x7f2fe40ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3832187352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 msgr2=0x7f2fe4149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 0x7f2fe4149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 -- 172.21.15.185:0/3832187352 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40ac430 con 0x7f2fe40b7610 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe3fff700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 0x7f2fe4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fdb7fe700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f2fe40b48c0 tx=0x7f2fcc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc011d50 con 0x7f2fe40b7610 2020-11-16T22:15:13.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc01e460 con 0x7f2fe40b7610 2020-11-16T22:15:13.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc01e820 con 0x7f2fe40b7610 2020-11-16T22:15:13.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe414a090 con 0x7f2fe40b7610 2020-11-16T22:15:13.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe40b5e10 con 0x7f2fe40b7610 2020-11-16T22:15:13.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe40ade70 con 0x7f2fe40b7610 2020-11-16T22:15:13.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2fcc025030 con 0x7f2fe40b7610 2020-11-16T22:15:13.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.835+0000 7f2fe0ff9700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 0x7f2fc0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:13.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.839+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2fcc06f570 con 0x7f2fe40b7610 2020-11-16T22:15:13.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.839+0000 7f2fe3fff700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 0x7f2fc0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:13.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.839+0000 7f2fe3fff700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 0x7f2fc0057520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2fd40020a0 tx=0x7f2fd4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:13.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:13.843+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2fcc0457d0 con 0x7f2fe40b7610 2020-11-16T22:15:14.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.043+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fe405d200 con 0x7f2fe40b7610 2020-11-16T22:15:14.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe0ff9700 1 -- 172.21.15.185:0/3832187352 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2500 (secure 0 0 0) 0x7f2fcc023020 con 0x7f2fe40b7610 2020-11-16T22:15:14.032 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:14.032 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:13.696065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4932,"mds_1":4927},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:creating","state_seq":1,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:14.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 msgr2=0x7f2fc0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 0x7f2fc0057520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2fd40020a0 tx=0x7f2fd4009040).stop 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 msgr2=0x7f2fe40ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f2fe40b48c0 tx=0x7f2fcc015000).stop 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 shutdown_connections 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2fc0055060 0x7f2fc0057520 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:14.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2fe4142f30 0x7f2fe40ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:14.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fe4067910 0x7f2fe4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:14.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 --2- 172.21.15.185:0/3832187352 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2fe40b7610 0x7f2fe40ae180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:14.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 >> 172.21.15.185:0/3832187352 conn(0x7f2fe40637b0 msgr2=0x7f2fe413ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:14.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 shutdown_connections 2020-11-16T22:15:14.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:14.047+0000 7f2fe940f700 1 -- 172.21.15.185:0/3832187352 wait complete. 2020-11-16T22:15:14.036 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 20 2020-11-16T22:15:14.351 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:15:05.635355+0000', 'modified': '2020-11-16T22:15:13.696065+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 4932, 'mds_1': 4927}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4927': {'gid': 4927, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.96:6833/404895834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 404895834}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 404895834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4932': {'gid': 4932, 'name': 'b', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2155443116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2155443116}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2155443116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:15:14.351 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4927:up:creating 2020-11-16T22:15:15.055 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:15:15.070+0000 7f9813ff6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:15:15.353 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:15.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 Processor -- start 2020-11-16T22:15:15.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- start start 2020-11-16T22:15:15.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd08c063020 0x7fd08c0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c144ba0 0x7fd08c147db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c1484c0 con 0x7fd08c0b0300 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c144ff0 con 0x7fd08c063020 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c1482a0 con 0x7fd08c144ba0 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd08c063020 0x7fd08c0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08a59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd08c0b0300 0x7fd08c13dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50956/0 (socket says 172.21.15.185:50956) 2020-11-16T22:15:15.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08a59c700 1 -- 172.21.15.185:0/554356392 learned_addr learned my addr 172.21.15.185:0/554356392 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:15.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08ad9d700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c144ba0 0x7fd08c147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08959a700 1 -- 172.21.15.185:0/554356392 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026711124 0 0) 0x7fd08c1484c0 con 0x7fd08c0b0300 2020-11-16T22:15:15.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08959a700 1 -- 172.21.15.185:0/554356392 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003240 con 0x7fd08c0b0300 2020-11-16T22:15:15.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 -- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c144ba0 msgr2=0x7fd08c147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c144ba0 0x7fd08c147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 -- 172.21.15.185:0/554356392 >> v1:172.21.15.185:6789/0 conn(0x7fd08c0b0300 legacy=0x7fd08c13dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 -- 172.21.15.185:0/554356392 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c1441c0 con 0x7fd08c063020 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd089d9b700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c063020 0x7fd08c0b06e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fd07c009cc0 tx=0x7fd07c0091d0).ready entity=mon.1 client_cookie=2f33e5f533ad8b89 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08959a700 1 -- 172.21.15.185:0/554356392 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00ba70 con 0x7fd08c063020 2020-11-16T22:15:15.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08959a700 1 -- 172.21.15.185:0/554356392 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c00bbd0 con 0x7fd08c063020 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd08959a700 1 -- 172.21.15.185:0/554356392 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011400 con 0x7fd08c063020 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c063020 msgr2=0x7fd08c0b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c063020 0x7fd08c0b06e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fd07c009cc0 tx=0x7fd07c0091d0).stop 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- 172.21.15.185:0/554356392 shutdown_connections 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c144ba0 0x7fd08c147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 --2- 172.21.15.185:0/554356392 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c063020 0x7fd08c0b06e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- 172.21.15.185:0/554356392 >> 172.21.15.185:0/554356392 conn(0x7fd08c0b4410 msgr2=0x7fd08c060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:15.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- 172.21.15.185:0/554356392 shutdown_connections 2020-11-16T22:15:15.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.499+0000 7fd090e67700 1 -- 172.21.15.185:0/554356392 wait complete. 2020-11-16T22:15:15.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 Processor -- start 2020-11-16T22:15:15.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- start start 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 0x7fd08c142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 0x7fd08c08ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c08f9f0 con 0x7fd08c063020 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c1530a0 con 0x7fd08c144ba0 2020-11-16T22:15:15.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0918d0 con 0x7fd08c0b0300 2020-11-16T22:15:15.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57528/0 (socket says 172.21.15.185:57528) 2020-11-16T22:15:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd089d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 0x7fd08c08ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 -- 172.21.15.185:0/1309023637 learned_addr learned my addr 172.21.15.185:0/1309023637 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08a59c700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 0x7fd08c142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 -- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 msgr2=0x7fd08c08ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 0x7fd08c08ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 -- 172.21.15.185:0/1309023637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 msgr2=0x7fd08c142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 0x7fd08c142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 -- 172.21.15.185:0/1309023637 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c091b70 con 0x7fd08c144ba0 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08a59c700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 0x7fd08c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd089d9b700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 0x7fd08c08ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:15.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd08ad9d700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd08400eba0 tx=0x7fd084006de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400c820 con 0x7fd08c144ba0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08400c980 con 0x7fd08c144ba0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c090930 con 0x7fd08c144ba0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400cd20 con 0x7fd08c144ba0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.503+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c0601c0 con 0x7fd08c144ba0 2020-11-16T22:15:15.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c153210 con 0x7fd08c144ba0 2020-11-16T22:15:15.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd084012600 con 0x7fd08c144ba0 2020-11-16T22:15:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd0827fc700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 0x7fd068057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd08a59c700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 0x7fd068057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd08406e010 con 0x7fd08c144ba0 2020-11-16T22:15:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.507+0000 7fd08a59c700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 0x7fd068057520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fd0740055a0 tx=0x7fd074009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:15.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.515+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd084044210 con 0x7fd08c144ba0 2020-11-16T22:15:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.715+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd08c05d4e0 con 0x7fd08c144ba0 2020-11-16T22:15:15.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.715+0000 7fd0827fc700 1 -- 172.21.15.185:0/1309023637 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2513 (secure 0 0 0) 0x7fd084047830 con 0x7fd08c144ba0 2020-11-16T22:15:15.703 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:15.703 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:15.710031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:15.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 msgr2=0x7fd068057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 0x7fd068057520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fd0740055a0 tx=0x7fd074009040).stop 2020-11-16T22:15:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 msgr2=0x7fd08c090440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd08400eba0 tx=0x7fd084006de0).stop 2020-11-16T22:15:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 shutdown_connections 2020-11-16T22:15:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd068055060 0x7fd068057520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd08c144ba0 0x7fd08c090440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd08c063020 0x7fd08c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 --2- 172.21.15.185:0/1309023637 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd08c0b0300 0x7fd08c08ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 >> 172.21.15.185:0/1309023637 conn(0x7fd08c0b4410 msgr2=0x7fd08c060ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 shutdown_connections 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:15.719+0000 7fd090e67700 1 -- 172.21.15.185:0/1309023637 wait complete. 2020-11-16T22:15:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 23 2020-11-16T22:15:16.213 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:15:05.635355+0000', 'modified': '2020-11-16T22:15:15.710031+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4932, 'mds_1': 4927, 'mds_2': 4682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4682': {'gid': 4682, 'name': 'a', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.185:6835/959337378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 959337378}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 959337378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4927': {'gid': 4927, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6833/404895834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 404895834}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 404895834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_4932': {'gid': 4932, 'name': 'b', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/2155443116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2155443116}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2155443116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:15:16.213 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:15:16.214 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:15:16.355 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":4682,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":1.654027412,"mdsmap_epoch":23,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":18.145641243} 2020-11-16T22:15:16.356 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4682, "mdsmap_epoch": 23, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 1.654027412, "state": "up:active", "uptime": 18.145641243, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:15:16.356 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:15:16.493 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":4927,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.8016795910000001,"mdsmap_epoch":23,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":18.278409301} 2020-11-16T22:15:16.494 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4927, "mdsmap_epoch": 23, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 2.801679591, "state": "up:active", "uptime": 18.278409301, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:15:16.495 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:15:16.622 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":4932,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.992500252999999,"mdsmap_epoch":23,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":18.403846122000001} 2020-11-16T22:15:16.622 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 4932, "mdsmap_epoch": 23, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 10.992500253, "state": "up:active", "uptime": 18.403846122, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:15:16.623 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:16.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 Processor -- start 2020-11-16T22:15:16.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 -- start start 2020-11-16T22:15:16.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f01f4062900 0x7f01f4062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4067930 con 0x7f01f41446c0 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40632e0 con 0x7f01f4062900 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fbb4d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4066ae0 con 0x7f01f4067de0 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50980/0 (socket says 172.21.15.185:50980) 2020-11-16T22:15:16.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 -- 172.21.15.185:0/1665442439 learned_addr learned my addr 172.21.15.185:0/1665442439 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:16.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01f98e9700 1 --2- 172.21.15.185:0/1665442439 >> v2:172.21.15.96:3300/0 conn(0x7f01f4062900 0x7f01f4062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 -- 172.21.15.185:0/1665442439 >> v2:172.21.15.96:3300/0 conn(0x7f01f4062900 msgr2=0x7f01f4062ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:16.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 --2- 172.21.15.185:0/1665442439 >> v2:172.21.15.96:3300/0 conn(0x7f01f4062900 0x7f01f4062ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 -- 172.21.15.185:0/1665442439 >> v1:172.21.15.96:6789/0 conn(0x7f01f41446c0 legacy=0x7f01f4067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 -- 172.21.15.185:0/1665442439 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4066d80 con 0x7f01f4067de0 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.767+0000 7f01fa0ea700 1 --2- 172.21.15.185:0/1665442439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f01ec009d50 tx=0x7f01ec009290).ready entity=mon.2 client_cookie=82df17d0fdaf2733 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01f88e7700 1 -- 172.21.15.185:0/1665442439 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00bbc0 con 0x7f01f4067de0 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01f88e7700 1 -- 172.21.15.185:0/1665442439 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00bd20 con 0x7f01f4067de0 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01f88e7700 1 -- 172.21.15.185:0/1665442439 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec005cb0 con 0x7f01f4067de0 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- 172.21.15.185:0/1665442439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 msgr2=0x7f01f40b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/1665442439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f01ec009d50 tx=0x7f01ec009290).stop 2020-11-16T22:15:16.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- 172.21.15.185:0/1665442439 shutdown_connections 2020-11-16T22:15:16.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/1665442439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f40b1840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/1665442439 >> v2:172.21.15.96:3300/0 conn(0x7f01f4062900 0x7f01f4062ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- 172.21.15.185:0/1665442439 >> 172.21.15.185:0/1665442439 conn(0x7f01f40b4410 msgr2=0x7f01f4060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:16.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- 172.21.15.185:0/1665442439 shutdown_connections 2020-11-16T22:15:16.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- 172.21.15.185:0/1665442439 wait complete. 2020-11-16T22:15:16.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 Processor -- start 2020-11-16T22:15:16.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.771+0000 7f01fbb4d700 1 -- start start 2020-11-16T22:15:16.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f4140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f41446c0 0x7f01f41411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4142150 con 0x7f01f41446c0 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4152180 con 0x7f01f4062900 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4140590 con 0x7f01f4067de0 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f90e8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f4140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57538/0 (socket says 172.21.15.185:57538) 2020-11-16T22:15:16.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 -- 172.21.15.185:0/3943368039 learned_addr learned my addr 172.21.15.185:0/3943368039 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:16.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fa0ea700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f41446c0 0x7f01f41411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 -- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 msgr2=0x7f01f4140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f4140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 -- 172.21.15.185:0/3943368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f41446c0 msgr2=0x7f01f41411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f41446c0 0x7f01f41411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 -- 172.21.15.185:0/3943368039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4140830 con 0x7f01f4062900 2020-11-16T22:15:16.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f90e8700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f4140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01f98e9700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f01f000ebd0 tx=0x7f01f0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000c8e0 con 0x7f01f4062900 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000ca40 con 0x7f01f4062900 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4127290 con 0x7f01f4062900 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000cdc0 con 0x7f01f4062900 2020-11-16T22:15:16.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.775+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41278f0 con 0x7f01f4062900 2020-11-16T22:15:16.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4152380 con 0x7f01f4062900 2020-11-16T22:15:16.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f01f0012600 con 0x7f01f4062900 2020-11-16T22:15:16.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01e1ffb700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 0x7f01cc057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:16.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01f006e050 con 0x7f01f4062900 2020-11-16T22:15:16.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01f90e8700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 0x7f01cc057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:16.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.779+0000 7f01f90e8700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 0x7f01cc057550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f01f413f500 tx=0x7f01e4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:16.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.787+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f01f00442b0 con 0x7f01f4062900 2020-11-16T22:15:16.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.987+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01f41409c0 con 0x7f01f4062900 2020-11-16T22:15:16.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.987+0000 7f01e1ffb700 1 -- 172.21.15.185:0/3943368039 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2513 (secure 0 0 0) 0x7f01f00478d0 con 0x7f01f4062900 2020-11-16T22:15:16.973 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:16.973 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:15.710031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.987+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 msgr2=0x7f01cc057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.987+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 0x7f01cc057550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f01f413f500 tx=0x7f01e4009040).stop 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.987+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 msgr2=0x7f01f4126da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f01f000ebd0 tx=0x7f01f0006df0).stop 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 shutdown_connections 2020-11-16T22:15:16.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f01cc055090 0x7f01cc057550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f01f4062900 0x7f01f4126da0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f41446c0 0x7f01f41411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 --2- 172.21.15.185:0/3943368039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f01f4067de0 0x7f01f4140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 >> 172.21.15.185:0/3943368039 conn(0x7f01f40b4410 msgr2=0x7f01f40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 shutdown_connections 2020-11-16T22:15:16.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:16.991+0000 7f01fbb4d700 1 -- 172.21.15.185:0/3943368039 wait complete. 2020-11-16T22:15:16.978 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 23 2020-11-16T22:15:16.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:16.994 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:15:17.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:17.062 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:15:18.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:18.231 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:15:18.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:18.278 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:15:18.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:18.311 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:15:18.347 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:18.485 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:15:18.485 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:18.486 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:20.487 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:20.625 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:15:20.626 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:20.627 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:22.628 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:22.769 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:15:22.769 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:22.770 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:24.771 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:24.906 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:15:24.907 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:24.907 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:26.909 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:27.044 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:15:27.045 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:27.045 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:29.047 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:15:29.183 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2020-11-16T22:15:29.184 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:29.185 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:15:29.185 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:15:29.313 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:15:29.314 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:15:29.314 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:15:31.316 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:15:31.454 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:15:31.455 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:15:31.456 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:15:33.457 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:15:33.591 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:15:33.592 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:15:33.593 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:15:35.594 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:15:35.731 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:15:35.731 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:15:35.732 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:15:35.733 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:15:35.870 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"15","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:15:35.871 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "15" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:15:35.872 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:15:35.872 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:35.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 Processor -- start 2020-11-16T22:15:35.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- start start 2020-11-16T22:15:35.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 0x7fcdac14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:35.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0b82e0 con 0x7fcdac0b4720 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0b7650 con 0x7fcdac144b90 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b78f0 con 0x7fcdac14cd00 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda9d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fcdac144b90 0x7fcdac0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50986/0 (socket says 172.21.15.185:50986) 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda9d9b700 1 -- 172.21.15.185:0/1852873828 learned_addr learned my addr 172.21.15.185:0/1852873828 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdaad9d700 1 --2- 172.21.15.185:0/1852873828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 0x7fcdac14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175278123 0 0) 0x7fcdac0b7650 con 0x7fcdac144b90 2020-11-16T22:15:35.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd90003240 con 0x7fcdac144b90 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063199753 0 0) 0x7fcdac0b82e0 con 0x7fcdac0b4720 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac0b7650 con 0x7fcdac0b4720 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1852873828 >> v1:172.21.15.185:6789/0 conn(0x7fcdac144b90 legacy=0x7fcdac0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1852873828 >> v1:172.21.15.96:6789/0 conn(0x7fcdac0b4720 legacy=0x7fcdac0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1852873828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0b1ee0 con 0x7fcdac14cd00 2020-11-16T22:15:35.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdaad9d700 1 --2- 172.21.15.185:0/1852873828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 0x7fcdac14f0f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fcda4009c40 tx=0x7fcda40091d0).ready entity=mon.2 client_cookie=861d878360e389fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00014e0 con 0x7fcdac14cd00 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda400b730 con 0x7fcdac14cd00 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcda959a700 1 -- 172.21.15.185:0/1852873828 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400bab0 con 0x7fcdac14cd00 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- 172.21.15.185:0/1852873828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 msgr2=0x7fcdac14f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1852873828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 0x7fcdac14f0f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fcda4009c40 tx=0x7fcda40091d0).stop 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- 172.21.15.185:0/1852873828 shutdown_connections 2020-11-16T22:15:36.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1852873828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac14cd00 0x7fcdac14f0f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- 172.21.15.185:0/1852873828 >> 172.21.15.185:0/1852873828 conn(0x7fcdac067860 msgr2=0x7fcdac067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- 172.21.15.185:0/1852873828 shutdown_connections 2020-11-16T22:15:36.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.011+0000 7fcdab59e700 1 -- 172.21.15.185:0/1852873828 wait complete. 2020-11-16T22:15:36.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 Processor -- start 2020-11-16T22:15:36.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- start start 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac0b4720 0x7fcdac153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 0x7fcdac128d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b11a0 con 0x7fcdac0b4720 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0ad1b0 con 0x7fcdac14cd00 2020-11-16T22:15:36.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0ad450 con 0x7fcdac144b90 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcda9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 0x7fcdac128d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57556/0 (socket says 172.21.15.185:57556) 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcda9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 0x7fcdac128d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:50998/0 (socket says 172.21.15.185:50998) 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1081144907 learned_addr learned my addr 172.21.15.185:0/1081144907 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaa59c700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac0b4720 0x7fcdac153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 msgr2=0x7fcdac128d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 0x7fcdac128d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1081144907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac0b4720 msgr2=0x7fcdac153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac0b4720 0x7fcdac153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 -- 172.21.15.185:0/1081144907 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac1567d0 con 0x7fcdac14cd00 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdaad9d700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcdac0ba830 tx=0x7fcdac0b3a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4021070 con 0x7fcdac14cd00 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4011e00 con 0x7fcdac14cd00 2020-11-16T22:15:36.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac156a70 con 0x7fcdac14cd00 2020-11-16T22:15:36.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4005ca0 con 0x7fcdac14cd00 2020-11-16T22:15:36.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.015+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac147490 con 0x7fcdac14cd00 2020-11-16T22:15:36.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac156c00 con 0x7fcdac14cd00 2020-11-16T22:15:36.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcda400f070 con 0x7fcdac14cd00 2020-11-16T22:15:36.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcd9a7fc700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 0x7fcd80057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcda406f930 con 0x7fcdac14cd00 2020-11-16T22:15:36.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcdaa59c700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 0x7fcd80057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.019+0000 7fcdaa59c700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 0x7fcd80057520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fcd9c007d70 tx=0x7fcd9c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.023+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcda4045b90 con 0x7fcdac14cd00 2020-11-16T22:15:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.219+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdac05d200 con 0x7fcdac14cd00 2020-11-16T22:15:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcd9a7fc700 1 -- 172.21.15.185:0/1081144907 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2517 (secure 0 0 0) 0x7fcda40491b0 con 0x7fcdac14cd00 2020-11-16T22:15:36.208 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:36.208 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:34.137417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 msgr2=0x7fcd80057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 0x7fcd80057520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fcd9c007d70 tx=0x7fcd9c009040).stop 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 msgr2=0x7fcdac156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcdac0ba830 tx=0x7fcdac0b3a20).stop 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 shutdown_connections 2020-11-16T22:15:36.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd80055060 0x7fcd80057520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdac14cd00 0x7fcdac156f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdac0b4720 0x7fcdac153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 --2- 172.21.15.185:0/1081144907 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdac144b90 0x7fcdac128d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 >> 172.21.15.185:0/1081144907 conn(0x7fcdac067860 msgr2=0x7fcdac12d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 shutdown_connections 2020-11-16T22:15:36.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.223+0000 7fcdab59e700 1 -- 172.21.15.185:0/1081144907 wait complete. 2020-11-16T22:15:36.212 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 25 2020-11-16T22:15:36.223 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:15:36.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.363+0000 7f673f59e700 1 Processor -- start 2020-11-16T22:15:36.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.363+0000 7f673f59e700 1 -- start start 2020-11-16T22:15:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f67400b2060 0x7f67401413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673f59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67400b2690 con 0x7f674005bc50 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67400b2930 con 0x7f67400b38a0 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67400b24b0 con 0x7f67400b2060 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f67400b38a0 0x7f6740144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:50998/0 (socket says 172.21.15.185:50998) 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673e59c700 1 -- 172.21.15.185:0/3680903412 learned_addr learned my addr 172.21.15.185:0/3680903412 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3680903412 >> v2:172.21.15.96:3300/0 conn(0x7f67400b2060 0x7f67401413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407300538 0 0) 0x7f67400b2930 con 0x7f67400b38a0 2020-11-16T22:15:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6724003240 con 0x7f67400b38a0 2020-11-16T22:15:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673ed9d700 1 -- 172.21.15.185:0/3680903412 >> v1:172.21.15.185:6789/0 conn(0x7f67400b38a0 legacy=0x7f6740144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673ed9d700 1 -- 172.21.15.185:0/3680903412 >> v1:172.21.15.96:6789/0 conn(0x7f674005bc50 legacy=0x7f6740140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743828705 0 0) 0x7f67400b2690 con 0x7f674005bc50 2020-11-16T22:15:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673ed9d700 1 -- 172.21.15.185:0/3680903412 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67401443d0 con 0x7f67400b2060 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3680903412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67400b2060 0x7f67401413d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6738009c90 tx=0x7f67380091d0).ready entity=mon.0 client_cookie=411426719f807aa9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800ba70 con 0x7f67400b2060 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800bbd0 con 0x7f67400b2060 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.367+0000 7f673d59a700 1 -- 172.21.15.185:0/3680903412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738011400 con 0x7f67400b2060 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- 172.21.15.185:0/3680903412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67400b2060 msgr2=0x7f67401413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 --2- 172.21.15.185:0/3680903412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67400b2060 0x7f67401413d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6738009c90 tx=0x7f67380091d0).stop 2020-11-16T22:15:36.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- 172.21.15.185:0/3680903412 shutdown_connections 2020-11-16T22:15:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 --2- 172.21.15.185:0/3680903412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67400b2060 0x7f67401413d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- 172.21.15.185:0/3680903412 >> 172.21.15.185:0/3680903412 conn(0x7f6740063df0 msgr2=0x7f67400641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- 172.21.15.185:0/3680903412 shutdown_connections 2020-11-16T22:15:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- 172.21.15.185:0/3680903412 wait complete. 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 Processor -- start 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- start start 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674005bc50 0x7f6740143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 0x7f67401548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 0x7f6740154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740138300 con 0x7f674005bc50 2020-11-16T22:15:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740155430 con 0x7f67400b2060 2020-11-16T22:15:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67401556d0 con 0x7f67400b38a0 2020-11-16T22:15:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 0x7f6740154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 0x7f67401548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 0x7f6740154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51012/0 (socket says 172.21.15.185:51012) 2020-11-16T22:15:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.371+0000 7f673ed9d700 1 -- 172.21.15.185:0/700597588 learned_addr learned my addr 172.21.15.185:0/700597588 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673e59c700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674005bc50 0x7f6740143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 -- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 msgr2=0x7f6740154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 0x7f6740154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 -- 172.21.15.185:0/700597588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674005bc50 msgr2=0x7f6740143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674005bc50 0x7f6740143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 -- 172.21.15.185:0/700597588 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400ad590 con 0x7f67400b2060 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673dd9b700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 0x7f67401548a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f673400c760 tx=0x7f6734007910).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 -- 172.21.15.185:0/700597588 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400e490 con 0x7f67400b2060 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 -- 172.21.15.185:0/700597588 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400e5f0 con 0x7f67400b2060 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67400ada30 con 0x7f67400b2060 2020-11-16T22:15:36.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 -- 172.21.15.185:0/700597588 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400e9b0 con 0x7f67400b2060 2020-11-16T22:15:36.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6740138f90 con 0x7f67400b2060 2020-11-16T22:15:36.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 -- 172.21.15.185:0/700597588 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6734019030 con 0x7f67400b2060 2020-11-16T22:15:36.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 0x7f6714057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f672e7fc700 1 -- 172.21.15.185:0/700597588 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f673406d2d0 con 0x7f67400b2060 2020-11-16T22:15:36.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673e59c700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 0x7f6714057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.375+0000 7f673e59c700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 0x7f6714057520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f67300055a0 tx=0x7f6730009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 Processor -- start 2020-11-16T22:15:36.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 -- start start 2020-11-16T22:15:36.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1d7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1af0bb0 0x1d7ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c084b0 con 0x1b5e7b0 2020-11-16T22:15:36.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cbc930 con 0x1af0bb0 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b7c6f0 con 0x1d7cda0 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51020/0 (socket says 172.21.15.185:51020) 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1af0bb0 0x1d7ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 -- 172.21.15.185:0/2292790604 learned_addr learned my addr 172.21.15.185:0/2292790604 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 -- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1af0bb0 msgr2=0x1d7ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1af0bb0 0x1d7ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673e59c700 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1d7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 -- 172.21.15.185:0/2292790604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 msgr2=0x1d7c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1d7c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 -- 172.21.15.185:0/2292790604 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b775a0 con 0x1d7cda0 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f673ed9d700 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f67380114b0 tx=0x7f6738016000).ready entity=mon.2 client_cookie=dd2d9c514630576c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f671b7fe700 1 -- 172.21.15.185:0/2292790604 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380117c0 con 0x1d7cda0 2020-11-16T22:15:36.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f671b7fe700 1 -- 172.21.15.185:0/2292790604 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738011920 con 0x1d7cda0 2020-11-16T22:15:36.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.379+0000 7f671b7fe700 1 -- 172.21.15.185:0/2292790604 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738011ca0 con 0x1d7cda0 2020-11-16T22:15:36.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 msgr2=0x1d7b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f67380114b0 tx=0x7f6738016000).stop 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/2292790604 shutdown_connections 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d7cda0 0x1d7b380 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1d7c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- 172.21.15.185:0/2292790604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1af0bb0 0x1d7ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/2292790604 >> 172.21.15.185:0/2292790604 conn(0x1d2ce40 msgr2=0x1d79dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/2292790604 shutdown_connections 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/2292790604 wait complete. 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- start start 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 Processor -- start 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 0x1dc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1dc1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 0x1daaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ca96e0 con 0x1b5e7b0 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ca9820 con 0x1d7cda0 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1caaa90 con 0x1af0bb0 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 0x1dc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 10 client.0 ll_register_callbacks cb 0x1c9fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 0x1dc0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51024/0 (socket says 172.21.15.185:51024) 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673e59c700 1 -- 172.21.15.185:0/3280351947 learned_addr learned my addr 172.21.15.185:0/3280351947 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 0x1daaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673dd9b700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1dc1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 -- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 msgr2=0x1dc0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 0x1dc0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 -- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 msgr2=0x1dc1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1dc1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 -- 172.21.15.185:0/3280351947 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c9a280 con 0x1d7cda0 2020-11-16T22:15:36.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 0x1daaca0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6738009b10 tx=0x7f673801f040).ready entity=mon.1 client_cookie=33389ef447b5c804 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f6718ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673801da30 con 0x1d7cda0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f673801db90 con 0x1d7cda0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d5d6c0 con 0x1d7cda0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6738026430 con 0x1d7cda0 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.383+0000 7f67537e5740 10 client.5167 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:36.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1acffd0 con 0x1d7cda0 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d5d6c0 con 0x1d7cda0 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f6738030070 con 0x1d7cda0 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 10 client.5167 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 10 client.5167 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 20 client.5167 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1dc3b90 0x1db0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 4 client.5167 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:36.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1b9ac50 con 0x1dc3b90 2020-11-16T22:15:36.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f673e59c700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1dc3b90 0x1db0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 10 client.5167 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:36.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f673e59c700 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1dc3b90 0x1db0950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c981ba2021c18c6f server_cookie=aef05c6690c5e9c3 in_seq=0 out_seq=0 2020-11-16T22:15:36.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 1 -- 172.21.15.185:0/3280351947 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1b9ac50 con 0x1dc3b90 2020-11-16T22:15:36.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 10 client.5167 handle_command_reply: tid=0 2020-11-16T22:15:36.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 client.5167 shutdown 2020-11-16T22:15:36.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 msgr2=0x1daaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 0x1daaca0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6738009b10 tx=0x7f673801f040).stop 2020-11-16T22:15:36.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 shutdown_connections 2020-11-16T22:15:36.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1dc3b90 0x1db0950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d7cda0 0x1daaca0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1af0bb0 0x1dc0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f6718ff9700 0 client.5167 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 --2- 172.21.15.185:0/3280351947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b5e7b0 0x1dc1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 >> 172.21.15.185:0/3280351947 conn(0x1d2ce40 msgr2=0x1daef10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 shutdown_connections 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 1 -- 172.21.15.185:0/3280351947 wait complete. 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.387+0000 7f67537e5740 20 client.5167 trim_cache size 0 max 16384 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 Processor -- start 2020-11-16T22:15:36.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 -- start start 2020-11-16T22:15:36.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1d7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1d78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c81e40 con 0x1af0bb0 2020-11-16T22:15:36.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d56800 con 0x1b5e7b0 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d5d6c0 con 0x1d2ce40 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57588/0 (socket says 172.21.15.185:57588) 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 -- 172.21.15.185:0/3776942045 learned_addr learned my addr 172.21.15.185:0/3776942045 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673e59c700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1d7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673dd9b700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1d78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 -- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 msgr2=0x1d7c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1d7c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 -- 172.21.15.185:0/3776942045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 msgr2=0x1d78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1d78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 -- 172.21.15.185:0/3776942045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d79180 con 0x1b5e7b0 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673dd9b700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1d78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:36.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f673ed9d700 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f6738011580 tx=0x7f673802b040).ready entity=mon.1 client_cookie=2e29ebe2cbd40b45 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f671bfff700 1 -- 172.21.15.185:0/3776942045 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380092d0 con 0x1b5e7b0 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f671bfff700 1 -- 172.21.15.185:0/3776942045 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673801d650 con 0x1b5e7b0 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.403+0000 7f671bfff700 1 -- 172.21.15.185:0/3776942045 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738032cd0 con 0x1b5e7b0 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 msgr2=0x1daed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f6738011580 tx=0x7f673802b040).stop 2020-11-16T22:15:36.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/3776942045 shutdown_connections 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1daed20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1d7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- 172.21.15.185:0/3776942045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1d78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/3776942045 >> 172.21.15.185:0/3776942045 conn(0x7f6708002320 msgr2=0x1d716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/3776942045 shutdown_connections 2020-11-16T22:15:36.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/3776942045 wait complete. 2020-11-16T22:15:36.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- start start 2020-11-16T22:15:36.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 Processor -- start 2020-11-16T22:15:36.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1daf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1e16170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daf810 con 0x1af0bb0 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d793b0 con 0x1b5e7b0 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dc4080 con 0x1d2ce40 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 10 client.0 ll_register_callbacks cb 0x1d769a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1e16170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51042/0 (socket says 172.21.15.185:51042) 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 -- 172.21.15.185:0/1250301947 learned_addr learned my addr 172.21.15.185:0/1250301947 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673e59c700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1daf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 -- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 msgr2=0x1e16170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1e16170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 -- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 msgr2=0x1daf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1daf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 -- 172.21.15.185:0/1250301947 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6708008160 con 0x1d2ce40 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673e59c700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1daf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f6718ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f673ed9d700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f673800bba0 tx=0x7f673802a040).ready entity=mon.2 client_cookie=c12dd549cb679a63 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738026430 con 0x1d2ce40 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6738026590 con 0x1d2ce40 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6738011920 con 0x1d2ce40 2020-11-16T22:15:36.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6738026960 con 0x1d2ce40 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 10 client.4883 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1b0e720 con 0x1d2ce40 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.407+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738011920 con 0x1d2ce40 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f6738030030 con 0x1d2ce40 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 10 client.4883 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 10 client.4883 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:36.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 20 client.4883 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1d7cda0 0x1dbf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 4 client.4883 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1ba1f50 con 0x1d7cda0 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f673e59c700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1d7cda0 0x1dbf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f6718ff9700 10 client.4883 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.411+0000 7f673e59c700 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1d7cda0 0x1dbf150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fada1a191e6a6b52 server_cookie=8c26b1724fff3de7 in_seq=0 out_seq=0 2020-11-16T22:15:36.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f6718ff9700 1 -- 172.21.15.185:0/1250301947 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1ba1f50 con 0x1d7cda0 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f6718ff9700 10 client.4883 handle_command_reply: tid=0 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 client.4883 shutdown 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 msgr2=0x1dba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f673800bba0 tx=0x7f673802a040).stop 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 shutdown_connections 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d2ce40 0x1dba780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1af0bb0 0x1daf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:36.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1d7cda0 0x1dbf150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 --2- 172.21.15.185:0/1250301947 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b5e7b0 0x1e16170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f6718ff9700 0 client.4883 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:37.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 >> 172.21.15.185:0/1250301947 conn(0x7f6708002320 msgr2=0x1d716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 shutdown_connections 2020-11-16T22:15:37.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 1 -- 172.21.15.185:0/1250301947 wait complete. 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.435+0000 7f67537e5740 20 client.4883 trim_cache size 0 max 16384 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 msgr2=0x7f6714057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 0x7f6714057520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f67300055a0 tx=0x7f6730009040).stop 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 msgr2=0x7f67401548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 0x7f67401548a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f673400c760 tx=0x7f6734007910).stop 2020-11-16T22:15:37.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 shutdown_connections 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6714055060 0x7f6714057520 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67400b38a0 0x7f6740154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674005bc50 0x7f6740143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 --2- 172.21.15.185:0/700597588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67400b2060 0x7f67401548a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 >> 172.21.15.185:0/700597588 conn(0x7f6740063df0 msgr2=0x7f6740060520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 shutdown_connections 2020-11-16T22:15:37.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:36.439+0000 7f673f59e700 1 -- 172.21.15.185:0/700597588 wait complete. 2020-11-16T22:15:37.009 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:37.009 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:15:37.009 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:15:37.009 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:37.010 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:37.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 Processor -- start 2020-11-16T22:15:37.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 -- start start 2020-11-16T22:15:37.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 0x7f2c540b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c54141100 0x7f2c54141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c54066ad0 con 0x7f2c540b7470 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c54140d10 con 0x7f2c54141100 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c58a10700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54144d90 con 0x7f2c540b2920 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c4bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 0x7f2c540b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c54141100 0x7f2c54141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c54141100 0x7f2c54141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57606/0 (socket says 172.21.15.185:57606) 2020-11-16T22:15:37.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 -- 172.21.15.185:0/3473283990 learned_addr learned my addr 172.21.15.185:0/3473283990 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c4bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 0x7f2c540b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51048/0 (socket says 172.21.15.185:51048) 2020-11-16T22:15:37.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 -- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 msgr2=0x7f2c540b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 --2- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 0x7f2c540b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 -- 172.21.15.185:0/3473283990 >> v1:172.21.15.96:6789/0 conn(0x7f2c540b7470 legacy=0x7f2c5413e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 -- 172.21.15.185:0/3473283990 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54144ff0 con 0x7f2c54141100 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c53fff700 1 --2- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c54141100 0x7f2c54141590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2c4c009d50 tx=0x7f2c4c009290).ready entity=mon.1 client_cookie=49748278da72f3ec server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c52ffd700 1 -- 172.21.15.185:0/3473283990 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00bc50 con 0x7f2c54141100 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c52ffd700 1 -- 172.21.15.185:0/3473283990 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c00bdb0 con 0x7f2c54141100 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.159+0000 7f2c52ffd700 1 -- 172.21.15.185:0/3473283990 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c011610 con 0x7f2c54141100 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c54141100 msgr2=0x7f2c54141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c54141100 0x7f2c54141590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2c4c009d50 tx=0x7f2c4c009290).stop 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- 172.21.15.185:0/3473283990 shutdown_connections 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c54141100 0x7f2c54141590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- 172.21.15.185:0/3473283990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c540b2920 0x7f2c540b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- 172.21.15.185:0/3473283990 >> 172.21.15.185:0/3473283990 conn(0x7f2c540637b0 msgr2=0x7f2c540b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- 172.21.15.185:0/3473283990 shutdown_connections 2020-11-16T22:15:37.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- 172.21.15.185:0/3473283990 wait complete. 2020-11-16T22:15:37.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 Processor -- start 2020-11-16T22:15:37.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- start start 2020-11-16T22:15:37.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c540b7470 0x7f2c540ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 0x7f2c540acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c541424a0 con 0x7f2c540b7470 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540b65f0 con 0x7f2c540b2920 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c58a10700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54142840 con 0x7f2c54141100 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 0x7f2c540acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57608/0 (socket says 172.21.15.185:57608) 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c537fe700 1 -- 172.21.15.185:0/1305240197 learned_addr learned my addr 172.21.15.185:0/1305240197 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c4bfff700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c540b7470 0x7f2c540ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 -- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 msgr2=0x7f2c54143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 -- 172.21.15.185:0/1305240197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c540b7470 msgr2=0x7f2c540ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c540b7470 0x7f2c540ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 -- 172.21.15.185:0/1305240197 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5413ed30 con 0x7f2c54141100 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c537fe700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.163+0000 7f2c53fff700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 0x7f2c540acc10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2c4c013000 tx=0x7f2c4c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00bc50 con 0x7f2c54141100 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c011650 con 0x7f2c54141100 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5413efd0 con 0x7f2c54141100 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c010cf0 con 0x7f2c54141100 2020-11-16T22:15:37.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c54143af0 con 0x7f2c54141100 2020-11-16T22:15:37.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5413f160 con 0x7f2c54141100 2020-11-16T22:15:37.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2c4c017070 con 0x7f2c54141100 2020-11-16T22:15:37.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 0x7f2c30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c4c06fe40 con 0x7f2c54141100 2020-11-16T22:15:37.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c537fe700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 0x7f2c30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.167+0000 7f2c537fe700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 0x7f2c30057520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2c440017a0 tx=0x7f2c44009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.175+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2c4c046120 con 0x7f2c54141100 2020-11-16T22:15:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.379+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c5405d200 con 0x7f2c54141100 2020-11-16T22:15:37.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.379+0000 7f2c4b7fe700 1 -- 172.21.15.185:0/1305240197 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2517 (secure 0 0 0) 0x7f2c4c015070 con 0x7f2c54141100 2020-11-16T22:15:37.366 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:37.366 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:34.137417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 msgr2=0x7f2c30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 0x7f2c30057520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2c440017a0 tx=0x7f2c44009040).stop 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 msgr2=0x7f2c540acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 0x7f2c540acc10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2c4c013000 tx=0x7f2c4c016000).stop 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 shutdown_connections 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c30055060 0x7f2c30057520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c540b2920 0x7f2c54143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c54141100 0x7f2c540acc10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 --2- 172.21.15.185:0/1305240197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c540b7470 0x7f2c540ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 >> 172.21.15.185:0/1305240197 conn(0x7f2c540637b0 msgr2=0x7f2c54067c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 shutdown_connections 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.383+0000 7f2c58a10700 1 -- 172.21.15.185:0/1305240197 wait complete. 2020-11-16T22:15:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 25 2020-11-16T22:15:37.380 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:15:37.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 Processor -- start 2020-11-16T22:15:37.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 -- start start 2020-11-16T22:15:37.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbb2c0b39e0 0x7fbb2c140be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0b26b0 con 0x7fbb2c0b2d40 2020-11-16T22:15:37.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c1410d0 con 0x7fbb2c0b3120 2020-11-16T22:15:37.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb33133700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0b2410 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb31930700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbb2c0b39e0 0x7fbb2c140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb32932700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbb2c0b3120 0x7fbb2c13d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51056/0 (socket says 172.21.15.185:51056) 2020-11-16T22:15:37.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.519+0000 7fbb32932700 1 -- 172.21.15.185:0/3543116880 learned_addr learned my addr 172.21.15.185:0/3543116880 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:37.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb3112f700 1 -- 172.21.15.185:0/3543116880 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132351957 0 0) 0x7fbb24001450 con 0x7fbb2c0b3120 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb3112f700 1 -- 172.21.15.185:0/3543116880 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb14003240 con 0x7fbb2c0b3120 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb31930700 1 -- 172.21.15.185:0/3543116880 >> v1:172.21.15.185:6789/0 conn(0x7fbb2c0b3120 legacy=0x7fbb2c13d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb31930700 1 -- 172.21.15.185:0/3543116880 >> v1:172.21.15.96:6789/0 conn(0x7fbb2c0b2d40 legacy=0x7fbb2c144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb31930700 1 -- 172.21.15.185:0/3543116880 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c1442c0 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb31930700 1 --2- 172.21.15.185:0/3543116880 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c140be0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fbb1c00b9a0 tx=0x7fbb1c006df0).ready entity=mon.1 client_cookie=21b53e7b7d5de813 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb3112f700 1 -- 172.21.15.185:0/3543116880 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0097b0 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb3112f700 1 -- 172.21.15.185:0/3543116880 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c009910 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb3112f700 1 -- 172.21.15.185:0/3543116880 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c009c90 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 -- 172.21.15.185:0/3543116880 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b39e0 msgr2=0x7fbb2c140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 --2- 172.21.15.185:0/3543116880 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c140be0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fbb1c00b9a0 tx=0x7fbb1c006df0).stop 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 -- 172.21.15.185:0/3543116880 shutdown_connections 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 --2- 172.21.15.185:0/3543116880 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c140be0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 -- 172.21.15.185:0/3543116880 >> 172.21.15.185:0/3543116880 conn(0x7fbb2c060130 msgr2=0x7fbb2c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 -- 172.21.15.185:0/3543116880 shutdown_connections 2020-11-16T22:15:37.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.523+0000 7fbb33133700 1 -- 172.21.15.185:0/3543116880 wait complete. 2020-11-16T22:15:37.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 Processor -- start 2020-11-16T22:15:37.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- start start 2020-11-16T22:15:37.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b2d40 0x7fbb2c1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c13f960 con 0x7fbb2c0b39e0 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c13fb90 con 0x7fbb2c0b2d40 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0ac8c0 con 0x7fbb2c0b3120 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51070/0 (socket says 172.21.15.185:51070) 2020-11-16T22:15:37.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 -- 172.21.15.185:0/377635514 learned_addr learned my addr 172.21.15.185:0/377635514 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb32131700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b2d40 0x7fbb2c1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb32932700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 -- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b2d40 msgr2=0x7fbb2c1439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b2d40 0x7fbb2c1439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 -- 172.21.15.185:0/377635514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 msgr2=0x7fbb2c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 -- 172.21.15.185:0/377635514 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0acb60 con 0x7fbb2c0b3120 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb32932700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb31930700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fbb1c006e30 tx=0x7fbb1c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb1a7fc700 1 -- 172.21.15.185:0/377635514 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c011dd0 con 0x7fbb2c0b3120 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb1a7fc700 1 -- 172.21.15.185:0/377635514 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c01e460 con 0x7fbb2c0b3120 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c1409f0 con 0x7fbb2c0b3120 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb1a7fc700 1 -- 172.21.15.185:0/377635514 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c01e7e0 con 0x7fbb2c0b3120 2020-11-16T22:15:37.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.527+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c140300 con 0x7fbb2c0b3120 2020-11-16T22:15:37.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb1a7fc700 1 -- 172.21.15.185:0/377635514 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb1c025030 con 0x7fbb2c0b3120 2020-11-16T22:15:37.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb1a7fc700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 0x7fbb04057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb32131700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 0x7fbb04057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb1a7fc700 1 -- 172.21.15.185:0/377635514 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb1c06f770 con 0x7fbb2c0b3120 2020-11-16T22:15:37.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb32131700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 0x7fbb04057520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbb28003300 tx=0x7fbb28007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb42e26740 1 Processor -- start 2020-11-16T22:15:37.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.531+0000 7fbb42e26740 1 -- start start 2020-11-16T22:15:37.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20127b0 0x22308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x222f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x20bc4b0 con 0x2230da0 2020-11-16T22:15:37.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2170930 con 0x1fa4bb0 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20306f0 con 0x20127b0 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb32131700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20127b0 0x22308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57632/0 (socket says 172.21.15.185:57632) 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 -- 172.21.15.185:0/2919788658 learned_addr learned my addr 172.21.15.185:0/2919788658 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb32932700 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x222f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 -- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20127b0 msgr2=0x22308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20127b0 0x22308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 -- 172.21.15.185:0/2919788658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 msgr2=0x222f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x222f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 -- 172.21.15.185:0/2919788658 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x202b5a0 con 0x1fa4bb0 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb32932700 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x222f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb31930700 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fbb1c049120 tx=0x7fbb1c011000).ready entity=mon.1 client_cookie=92fb8693b294c372 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb0b7fe700 1 -- 172.21.15.185:0/2919788658 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c06dea0 con 0x1fa4bb0 2020-11-16T22:15:37.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb0b7fe700 1 -- 172.21.15.185:0/2919788658 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0477f0 con 0x1fa4bb0 2020-11-16T22:15:37.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb0b7fe700 1 -- 172.21.15.185:0/2919788658 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c048ce0 con 0x1fa4bb0 2020-11-16T22:15:37.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 msgr2=0x222ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fbb1c049120 tx=0x7fbb1c011000).stop 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- 172.21.15.185:0/2919788658 shutdown_connections 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x222f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20127b0 0x22308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2919788658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x222ee90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- 172.21.15.185:0/2919788658 >> 172.21.15.185:0/2919788658 conn(0x21e0e40 msgr2=0x222ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- 172.21.15.185:0/2919788658 shutdown_connections 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- 172.21.15.185:0/2919788658 wait complete. 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 -- start start 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.535+0000 7fbb42e26740 1 Processor -- start 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa4bb0 0x2274bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x225eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x215d6e0 con 0x2230da0 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x215d820 con 0x20127b0 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x215ea90 con 0x1fa4bb0 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57638/0 (socket says 172.21.15.185:57638) 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 -- 172.21.15.185:0/2487393895 learned_addr learned my addr 172.21.15.185:0/2487393895 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb32131700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa4bb0 0x2274bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 10 client.0 ll_register_callbacks cb 0x2153ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb32932700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x225eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 -- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa4bb0 msgr2=0x2274bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa4bb0 0x2274bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 -- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 msgr2=0x225eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x225eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 -- 172.21.15.185:0/2487393895 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x214e280 con 0x20127b0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb32932700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x225eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb31930700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fbb1c000dd0 tx=0x7fbb1c020070).ready entity=mon.1 client_cookie=90d6a06932227976 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c06dea0 con 0x20127b0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb1c048750 con 0x20127b0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22116c0 con 0x20127b0 2020-11-16T22:15:37.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1c048910 con 0x20127b0 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 10 client.5194 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1f83fd0 con 0x20127b0 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22116c0 con 0x20127b0 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fbb1c038070 con 0x20127b0 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 10 client.5194 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 10 client.5194 resolve_mds: resolved ID 'c' to GID 4927 2020-11-16T22:15:37.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 20 client.5194 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:37.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x22663a0 0x2264950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 4 client.5194 mds_command: new command op to 4927 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:37.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 --> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x204ec50 con 0x22663a0 2020-11-16T22:15:37.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb32131700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x22663a0 0x2264950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb08ff9700 10 client.5194 ms_handle_connect on v2:172.21.15.96:6832/404895834 2020-11-16T22:15:37.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.539+0000 7fbb32131700 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x22663a0 0x2264950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=294970190397dfe0 server_cookie=1d325a5434c7b987 in_seq=0 out_seq=0 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb08ff9700 1 -- 172.21.15.185:0/2487393895 <== mds.1 v2:172.21.15.96:6832/404895834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x204ec50 con 0x22663a0 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb08ff9700 10 client.5194 handle_command_reply: tid=0 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 client.5194 shutdown 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 msgr2=0x22750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fbb1c000dd0 tx=0x7fbb1c020070).stop 2020-11-16T22:15:37.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 shutdown_connections 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x22663a0 0x2264950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2230da0 0x225eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa4bb0 0x2274bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 --2- 172.21.15.185:0/2487393895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x22750a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb08ff9700 0 client.5194 ms_handle_reset on v2:172.21.15.96:6832/404895834 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 >> 172.21.15.185:0/2487393895 conn(0x21e0e40 msgr2=0x222ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 shutdown_connections 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 1 -- 172.21.15.185:0/2487393895 wait complete. 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.543+0000 7fbb42e26740 20 client.5194 trim_cache size 0 max 16384 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 Processor -- start 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 -- start start 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1fa4bb0 0x2273bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x222d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2135e40 con 0x1fa4bb0 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x222b3f0 con 0x20127b0 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x227cd30 con 0x21e0e40 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51088/0 (socket says 172.21.15.185:51088) 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 -- 172.21.15.185:0/789733885 learned_addr learned my addr 172.21.15.185:0/789733885 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32932700 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x222d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb31930700 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1fa4bb0 0x2273bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 -- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 msgr2=0x222d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x222d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 -- 172.21.15.185:0/789733885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1fa4bb0 msgr2=0x2273bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1fa4bb0 0x2273bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 -- 172.21.15.185:0/789733885 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80081d0 con 0x21e0e40 2020-11-16T22:15:37.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb32131700 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fbb28017600 tx=0x7fbb2801a040).ready entity=mon.2 client_cookie=cc7bfed20f4080f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb0bfff700 1 -- 172.21.15.185:0/789733885 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2801c040 con 0x21e0e40 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb0bfff700 1 -- 172.21.15.185:0/789733885 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb28021970 con 0x21e0e40 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.559+0000 7fbb0bfff700 1 -- 172.21.15.185:0/789733885 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28021cf0 con 0x21e0e40 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 msgr2=0x2274f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fbb28017600 tx=0x7fbb2801a040).stop 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- 172.21.15.185:0/789733885 shutdown_connections 2020-11-16T22:15:37.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20127b0 0x222d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2274f90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 --2- 172.21.15.185:0/789733885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1fa4bb0 0x2273bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- 172.21.15.185:0/789733885 >> 172.21.15.185:0/789733885 conn(0x7fbaf8002320 msgr2=0x225e970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- 172.21.15.185:0/789733885 shutdown_connections 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- 172.21.15.185:0/789733885 wait complete. 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 -- start start 2020-11-16T22:15:37.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.563+0000 7fbb42e26740 1 Processor -- start 2020-11-16T22:15:37.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x22629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20127b0 0x227ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2279740 con 0x20127b0 2020-11-16T22:15:37.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22c6d60 con 0x1fa4bb0 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x22c7000 con 0x21e0e40 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 10 client.0 ll_register_callbacks cb 0x2263b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51098/0 (socket says 172.21.15.185:51098) 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 -- 172.21.15.185:0/3915533618 learned_addr learned my addr 172.21.15.185:0/3915533618 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32131700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x22629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb31930700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20127b0 0x227ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 -- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 msgr2=0x22629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x22629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 -- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20127b0 msgr2=0x227ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20127b0 0x227ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 -- 172.21.15.185:0/3915533618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1fc2720 con 0x21e0e40 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32131700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x22629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:37.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32932700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fbb240012a0 tx=0x7fbb24009040).ready entity=mon.2 client_cookie=57a1f3eb34a36510 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240167c0 con 0x21e0e40 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb24016920 con 0x21e0e40 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb1c0477f0 con 0x21e0e40 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb24016cf0 con 0x21e0e40 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 10 client.4904 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:37.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1fcde00 con 0x21e0e40 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0477f0 con 0x21e0e40 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fbb240159f0 con 0x21e0e40 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 10 client.4904 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 10 client.4904 resolve_mds: resolved ID 'c' to GID 4927 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 20 client.4904 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x2230da0 0x227a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:37.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 4 client.4904 mds_command: new command op to 4927 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 --> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x2055f50 con 0x2230da0 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32131700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x2230da0 0x227a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb08ff9700 10 client.4904 ms_handle_connect on v2:172.21.15.96:6832/404895834 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.567+0000 7fbb32131700 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x2230da0 0x227a2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=ca86e4c2921107f4 server_cookie=7ff7b945c16c2bbe in_seq=0 out_seq=0 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb08ff9700 1 -- 172.21.15.185:0/3915533618 <== mds.1 v2:172.21.15.96:6832/404895834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x2055f50 con 0x2230da0 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb08ff9700 10 client.4904 handle_command_reply: tid=0 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 client.4904 shutdown 2020-11-16T22:15:37.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 msgr2=0x2279b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fbb240012a0 tx=0x7fbb24009040).stop 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 shutdown_connections 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:6832/404895834,v1:172.21.15.96:6833/404895834] conn(0x2230da0 0x227a2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1fa4bb0 0x22629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21e0e40 0x2279b70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:37.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 --2- 172.21.15.185:0/3915533618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20127b0 0x227ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 >> 172.21.15.185:0/3915533618 conn(0x7fbaf8002320 msgr2=0x225e950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb08ff9700 0 client.4904 ms_handle_reset on v2:172.21.15.96:6832/404895834 2020-11-16T22:15:38.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 shutdown_connections 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 1 -- 172.21.15.185:0/3915533618 wait complete. 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb42e26740 20 client.4904 trim_cache size 0 max 16384 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 msgr2=0x7fbb04057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 0x7fbb04057520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fbb28003300 tx=0x7fbb28007b90).stop 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 msgr2=0x7fbb2c133960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fbb1c006e30 tx=0x7fbb1c015000).stop 2020-11-16T22:15:38.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 shutdown_connections 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb04055060 0x7fbb04057520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb2c0b2d40 0x7fbb2c1439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb2c0b39e0 0x7fbb2c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 --2- 172.21.15.185:0/377635514 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb2c0b3120 0x7fbb2c133960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 >> 172.21.15.185:0/377635514 conn(0x7fbb2c060130 msgr2=0x7fbb2c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 shutdown_connections 2020-11-16T22:15:38.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:37.587+0000 7fbb33133700 1 -- 172.21.15.185:0/377635514 wait complete. 2020-11-16T22:15:38.161 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:38.161 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:15:38.161 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:15:38.161 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:38.161 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:38.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 Processor -- start 2020-11-16T22:15:38.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 -- start start 2020-11-16T22:15:38.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 0x7f79f813de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f80b7e10 con 0x7f79f81418d0 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80b7670 con 0x7f79f80b4d40 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fda4f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80b78d0 con 0x7f79f80675b0 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fca4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fca4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51104/0 (socket says 172.21.15.185:51104) 2020-11-16T22:15:38.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fca4d700 1 -- 172.21.15.185:0/3572624454 learned_addr learned my addr 172.21.15.185:0/3572624454 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.311+0000 7f79fd24e700 1 --2- 172.21.15.185:0/3572624454 >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 0x7f79f813de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fca4d700 1 -- 172.21.15.185:0/3572624454 >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 msgr2=0x7f79f813de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fca4d700 1 --2- 172.21.15.185:0/3572624454 >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 0x7f79f813de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fca4d700 1 -- 172.21.15.185:0/3572624454 >> v1:172.21.15.96:6789/0 conn(0x7f79f81418d0 legacy=0x7f79f80b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fca4d700 1 -- 172.21.15.185:0/3572624454 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80b7b70 con 0x7f79f80675b0 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fd24e700 1 --2- 172.21.15.185:0/3572624454 >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 0x7f79f813de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fca4d700 1 --2- 172.21.15.185:0/3572624454 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f79e8006fd0 tx=0x7f79e8006eb0).ready entity=mon.2 client_cookie=761e8b3bc61679c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79f77fe700 1 -- 172.21.15.185:0/3572624454 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80098d0 con 0x7f79f80675b0 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79f77fe700 1 -- 172.21.15.185:0/3572624454 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8009a30 con 0x7f79f80675b0 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79f77fe700 1 -- 172.21.15.185:0/3572624454 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8009db0 con 0x7f79f80675b0 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- 172.21.15.185:0/3572624454 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 msgr2=0x7f79f8068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- 172.21.15.185:0/3572624454 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f79e8006fd0 tx=0x7f79e8006eb0).stop 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- 172.21.15.185:0/3572624454 shutdown_connections 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- 172.21.15.185:0/3572624454 >> v2:172.21.15.96:3300/0 conn(0x7f79f80b4d40 0x7f79f813de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- 172.21.15.185:0/3572624454 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80675b0 0x7f79f8068460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- 172.21.15.185:0/3572624454 >> 172.21.15.185:0/3572624454 conn(0x7f79f805ba80 msgr2=0x7f79f805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- 172.21.15.185:0/3572624454 shutdown_connections 2020-11-16T22:15:38.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- 172.21.15.185:0/3572624454 wait complete. 2020-11-16T22:15:38.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 Processor -- start 2020-11-16T22:15:38.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- start start 2020-11-16T22:15:38.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79f80675b0 0x7f79f81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79f81418d0 0x7f79f80b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f81579d0 con 0x7f79f81418d0 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8138480 con 0x7f79f80675b0 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79fda4f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8138720 con 0x7f79f80b4d40 2020-11-16T22:15:38.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.315+0000 7f79f7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51110/0 (socket says 172.21.15.185:51110) 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 -- 172.21.15.185:0/670970170 learned_addr learned my addr 172.21.15.185:0/670970170 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fca4d700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79f80675b0 0x7f79f81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fd24e700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79f81418d0 0x7f79f80b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 -- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79f80675b0 msgr2=0x7f79f81574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79f80675b0 0x7f79f81574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 -- 172.21.15.185:0/670970170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79f81418d0 msgr2=0x7f79f80b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79f81418d0 0x7f79f80b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 -- 172.21.15.185:0/670970170 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f81389c0 con 0x7f79f80b4d40 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f7fff700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f79ec00c710 tx=0x7f79ec007bf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007020 con 0x7f79f80b4d40 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec007180 con 0x7f79f80b4d40 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f8138c60 con 0x7f79f80b4d40 2020-11-16T22:15:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0064f0 con 0x7f79f80b4d40 2020-11-16T22:15:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f8144b40 con 0x7f79f80b4d40 2020-11-16T22:15:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f8138df0 con 0x7f79f80b4d40 2020-11-16T22:15:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f79ec010070 con 0x7f79f80b4d40 2020-11-16T22:15:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 0x7f79d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79ec06d060 con 0x7f79f80b4d40 2020-11-16T22:15:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.319+0000 7f79fca4d700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 0x7f79d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.323+0000 7f79fca4d700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 0x7f79d0057520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f79e8006ef0 tx=0x7f79e8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.327+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f79ec0432c0 con 0x7f79f80b4d40 2020-11-16T22:15:38.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.527+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f805d200 con 0x7f79f80b4d40 2020-11-16T22:15:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79f4ff9700 1 -- 172.21.15.185:0/670970170 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2517 (secure 0 0 0) 0x7f79ec0468e0 con 0x7f79f80b4d40 2020-11-16T22:15:38.515 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:38.515 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:34.137417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 msgr2=0x7f79d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 0x7f79d0057520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f79e8006ef0 tx=0x7f79e8015000).stop 2020-11-16T22:15:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 msgr2=0x7f79f813bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f79ec00c710 tx=0x7f79ec007bf0).stop 2020-11-16T22:15:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 shutdown_connections 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79d0055060 0x7f79d0057520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79f80675b0 0x7f79f81574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79f81418d0 0x7f79f80b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 --2- 172.21.15.185:0/670970170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f79f80b4d40 0x7f79f813bad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.531+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 >> 172.21.15.185:0/670970170 conn(0x7f79f805ba80 msgr2=0x7f79f80b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.535+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 shutdown_connections 2020-11-16T22:15:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.535+0000 7f79fda4f700 1 -- 172.21.15.185:0/670970170 wait complete. 2020-11-16T22:15:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 25 2020-11-16T22:15:38.535 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:15:38.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.671+0000 7fec42747700 1 Processor -- start 2020-11-16T22:15:38.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.671+0000 7fec42747700 1 -- start start 2020-11-16T22:15:38.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 0x7fec3c0ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec3c1475d0 0x7fec3c147a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0b7290 con 0x7fec3c0b5d70 2020-11-16T22:15:38.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0b4bf0 con 0x7fec3c1475d0 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b74c0 con 0x7fec3c0b4470 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec3c1475d0 0x7fec3c147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec3c1475d0 0x7fec3c147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57676/0 (socket says 172.21.15.185:57676) 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 -- 172.21.15.185:0/1929954189 learned_addr learned my addr 172.21.15.185:0/1929954189 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41745700 1 --2- 172.21.15.185:0/1929954189 >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 0x7fec3c0ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 -- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 msgr2=0x7fec3c0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 --2- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c0b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 -- 172.21.15.185:0/1929954189 >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 msgr2=0x7fec3c0ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 --2- 172.21.15.185:0/1929954189 >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 0x7fec3c0ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 -- 172.21.15.185:0/1929954189 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0b7760 con 0x7fec3c1475d0 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41745700 1 --2- 172.21.15.185:0/1929954189 >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 0x7fec3c0ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec41f46700 1 --2- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c1475d0 0x7fec3c147a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fec3c0b3090 tx=0x7fec34009290).ready entity=mon.1 client_cookie=dc9695f85248f617 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec33fff700 1 -- 172.21.15.185:0/1929954189 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3400bbc0 con 0x7fec3c1475d0 2020-11-16T22:15:38.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec33fff700 1 -- 172.21.15.185:0/1929954189 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3400bd20 con 0x7fec3c1475d0 2020-11-16T22:15:38.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec33fff700 1 -- 172.21.15.185:0/1929954189 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34011610 con 0x7fec3c1475d0 2020-11-16T22:15:38.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c1475d0 msgr2=0x7fec3c147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c1475d0 0x7fec3c147a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fec3c0b3090 tx=0x7fec34009290).stop 2020-11-16T22:15:38.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- 172.21.15.185:0/1929954189 shutdown_connections 2020-11-16T22:15:38.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c1475d0 0x7fec3c147a50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- 172.21.15.185:0/1929954189 >> v2:172.21.15.96:3300/0 conn(0x7fec3c0b5d70 0x7fec3c0ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 --2- 172.21.15.185:0/1929954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c0b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- 172.21.15.185:0/1929954189 >> 172.21.15.185:0/1929954189 conn(0x7fec3c05ba80 msgr2=0x7fec3c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- 172.21.15.185:0/1929954189 shutdown_connections 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.675+0000 7fec42747700 1 -- 172.21.15.185:0/1929954189 wait complete. 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 Processor -- start 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- start start 2020-11-16T22:15:38.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec3c1475d0 0x7fec3c13d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c113f80 con 0x7fec3c1475d0 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0ad7b0 con 0x7fec3c0b5d70 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0ae250 con 0x7fec3c0b4470 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57678/0 (socket says 172.21.15.185:57678) 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec40f44700 1 -- 172.21.15.185:0/1147149441 learned_addr learned my addr 172.21.15.185:0/1147149441 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41f46700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec3c1475d0 0x7fec3c13d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 -- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 msgr2=0x7fec3c143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 -- 172.21.15.185:0/1147149441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec3c1475d0 msgr2=0x7fec3c13d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec3c1475d0 0x7fec3c13d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 -- 172.21.15.185:0/1147149441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0adfb0 con 0x7fec3c0b4470 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec40f44700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec41745700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c113a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fec3800d910 tx=0x7fec38008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec317fa700 1 -- 172.21.15.185:0/1147149441 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380125f0 con 0x7fec3c0b4470 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec317fa700 1 -- 172.21.15.185:0/1147149441 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38012750 con 0x7fec3c0b4470 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec317fa700 1 -- 172.21.15.185:0/1147149441 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38012b10 con 0x7fec3c0b4470 2020-11-16T22:15:38.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c0ada10 con 0x7fec3c0b4470 2020-11-16T22:15:38.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.679+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c0ae4a0 con 0x7fec3c0b4470 2020-11-16T22:15:38.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.683+0000 7fec317fa700 1 -- 172.21.15.185:0/1147149441 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fec38013480 con 0x7fec3c0b4470 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.683+0000 7fec317fa700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 0x7fec14057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.683+0000 7fec317fa700 1 -- 172.21.15.185:0/1147149441 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3806df50 con 0x7fec3c0b4470 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.683+0000 7fec40f44700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 0x7fec14057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.683+0000 7fec40f44700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 0x7fec14057520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fec3c145e70 tx=0x7fec2c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 Processor -- start 2020-11-16T22:15:38.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- start start 2020-11-16T22:15:38.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 0x1ea48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c18bb0 0x1ea2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d304b0 con 0x1c867b0 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de4930 con 0x1c18bb0 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ca46f0 con 0x1ea4da0 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c18bb0 0x1ea2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51132/0 (socket says 172.21.15.185:51132) 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 -- 172.21.15.185:0/3455955205 learned_addr learned my addr 172.21.15.185:0/3455955205 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41745700 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 0x1ea48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 -- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c18bb0 msgr2=0x1ea2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c18bb0 0x1ea2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 -- 172.21.15.185:0/3455955205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 msgr2=0x1ea48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 0x1ea48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 -- 172.21.15.185:0/3455955205 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1c9f5a0 con 0x1ea4da0 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41745700 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 0x1ea48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec41f46700 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fec34011000 tx=0x7fec34016000).ready entity=mon.2 client_cookie=7f6b8fefdfc13652 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec1e7fc700 1 -- 172.21.15.185:0/3455955205 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34011610 con 0x1ea4da0 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec1e7fc700 1 -- 172.21.15.185:0/3455955205 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34021070 con 0x1ea4da0 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec1e7fc700 1 -- 172.21.15.185:0/3455955205 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34006ca0 con 0x1ea4da0 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 msgr2=0x1ea3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fec34011000 tx=0x7fec34016000).stop 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- 172.21.15.185:0/3455955205 shutdown_connections 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ea4da0 0x1ea3380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c867b0 0x1ea48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 --2- 172.21.15.185:0/3455955205 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c18bb0 0x1ea2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- 172.21.15.185:0/3455955205 >> 172.21.15.185:0/3455955205 conn(0x1e54e40 msgr2=0x1ea1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- 172.21.15.185:0/3455955205 shutdown_connections 2020-11-16T22:15:38.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- 172.21.15.185:0/3455955205 wait complete. 2020-11-16T22:15:38.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 -- start start 2020-11-16T22:15:38.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.687+0000 7fec5243a740 1 Processor -- start 2020-11-16T22:15:38.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1eec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ea4da0 0x1ea5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd16e0 con 0x1ea4da0 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd1820 con 0x1c867b0 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd2a90 con 0x1c18bb0 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 10 client.0 ll_register_callbacks cb 0x1dc7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1eec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51136/0 (socket says 172.21.15.185:51136) 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 -- 172.21.15.185:0/1706927604 learned_addr learned my addr 172.21.15.185:0/1706927604 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41f46700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ea4da0 0x1ea5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 -- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 msgr2=0x1eec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1eec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 -- 172.21.15.185:0/1706927604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ea4da0 msgr2=0x1ea5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ea4da0 0x1ea5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 -- 172.21.15.185:0/1706927604 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1dc2280 con 0x1c18bb0 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec40f44700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1eec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec41745700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fec38011eb0 tx=0x7fec38018000).ready entity=mon.2 client_cookie=2fce42ac5dbf880 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380446d0 con 0x1c18bb0 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec38043e10 con 0x1c18bb0 2020-11-16T22:15:38.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e856c0 con 0x1c18bb0 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec38046f80 con 0x1c18bb0 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 10 client.4922 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1bf7fd0 con 0x1c18bb0 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e856c0 con 0x1c18bb0 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fec3806c720 con 0x1c18bb0 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 10 client.4922 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:38.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 10 client.4922 resolve_mds: resolved ID 'a' to GID 4682 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 20 client.4922 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ee86e0 0x1ee8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 4 client.4922 mds_command: new command op to 4682 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 --> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1cc2c50 con 0x1ee86e0 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec40f44700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ee86e0 0x1ee8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec0b7fe700 10 client.4922 ms_handle_connect on v2:172.21.15.185:6834/959337378 2020-11-16T22:15:38.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.691+0000 7fec40f44700 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ee86e0 0x1ee8ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=8ea918ecc4a497bd server_cookie=b5064aba8c11d049 in_seq=0 out_seq=0 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec0b7fe700 1 -- 172.21.15.185:0/1706927604 <== mds.2 v2:172.21.15.185:6834/959337378 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1cc2c50 con 0x1ee86e0 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec0b7fe700 10 client.4922 handle_command_reply: tid=0 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 client.4922 shutdown 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 msgr2=0x1ea31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fec38011eb0 tx=0x7fec38018000).stop 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 shutdown_connections 2020-11-16T22:15:38.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ee86e0 0x1ee8ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ea4da0 0x1ea5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1ea31e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 --2- 172.21.15.185:0/1706927604 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1eec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec0b7fe700 0 client.4922 ms_handle_reset on v2:172.21.15.185:6834/959337378 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 >> 172.21.15.185:0/1706927604 conn(0x1e54e40 msgr2=0x1ed4760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 shutdown_connections 2020-11-16T22:15:38.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 1 -- 172.21.15.185:0/1706927604 wait complete. 2020-11-16T22:15:38.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.695+0000 7fec5243a740 20 client.4922 trim_cache size 0 max 16384 2020-11-16T22:15:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 Processor -- start 2020-11-16T22:15:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 -- start start 2020-11-16T22:15:38.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e54e40 0x1eda360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c18bb0 0x1e9f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da9e40 con 0x1c18bb0 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e86200 con 0x1c867b0 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ea1100 con 0x1e54e40 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e54e40 0x1eda360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57704/0 (socket says 172.21.15.185:57704) 2020-11-16T22:15:38.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 -- 172.21.15.185:0/3407867250 learned_addr learned my addr 172.21.15.185:0/3407867250 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec40f44700 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c18bb0 0x1e9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 -- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e54e40 msgr2=0x1eda360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e54e40 0x1eda360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 -- 172.21.15.185:0/3407867250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c18bb0 msgr2=0x1e9f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c18bb0 0x1e9f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.711+0000 7fec41f46700 1 -- 172.21.15.185:0/3407867250 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10008160 con 0x1c867b0 2020-11-16T22:15:38.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec41f46700 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fec3c05c5f0 tx=0x7fec3401f070).ready entity=mon.1 client_cookie=9520bee2dd28802 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec1effd700 1 -- 172.21.15.185:0/3407867250 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3402c040 con 0x1c867b0 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec1effd700 1 -- 172.21.15.185:0/3407867250 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34011610 con 0x1c867b0 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec1effd700 1 -- 172.21.15.185:0/3407867250 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34006c10 con 0x1c867b0 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 msgr2=0x1ea5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fec3c05c5f0 tx=0x7fec3401f070).stop 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- 172.21.15.185:0/3407867250 shutdown_connections 2020-11-16T22:15:38.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1ea5400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e54e40 0x1eda360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 --2- 172.21.15.185:0/3407867250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c18bb0 0x1e9f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- 172.21.15.185:0/3407867250 >> 172.21.15.185:0/3407867250 conn(0x7fec10002320 msgr2=0x1ea3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- 172.21.15.185:0/3407867250 shutdown_connections 2020-11-16T22:15:38.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- 172.21.15.185:0/3407867250 wait complete. 2020-11-16T22:15:38.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 -- start start 2020-11-16T22:15:38.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.715+0000 7fec5243a740 1 Processor -- start 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1f344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 0x1f349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f34ed0 con 0x1e54e40 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ea5780 con 0x1c867b0 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ea24c0 con 0x1c18bb0 2020-11-16T22:15:38.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 10 client.0 ll_register_callbacks cb 0x1e9e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec40f44700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1f344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51150/0 (socket says 172.21.15.185:51150) 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 -- 172.21.15.185:0/214903029 learned_addr learned my addr 172.21.15.185:0/214903029 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41f46700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 0x1f349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 -- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 msgr2=0x1f344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1f344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 -- 172.21.15.185:0/214903029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 msgr2=0x1f349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 0x1f349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 -- 172.21.15.185:0/214903029 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c36720 con 0x1c18bb0 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41f46700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 0x1f349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec41745700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fec38047d40 tx=0x7fec38026000).ready entity=mon.2 client_cookie=feec32b98e3faf64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:38.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38046ac0 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3806cbd0 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec34021070 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec38043e10 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 10 client.4928 fetch_fsmap learned FSMap version 25 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1c41e00 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34021070 con 0x1c18bb0 2020-11-16T22:15:38.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 25) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fec38046f80 con 0x1c18bb0 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 10 client.4928 fetch_fsmap finished waiting for FSMap version 25 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 10 client.4928 resolve_mds: resolved ID 'a' to GID 4682 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 20 client.4928 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ea4da0 0x1f3a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 4 client.4928 mds_command: new command op to 4682 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 --> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1cc9f50 con 0x1ea4da0 2020-11-16T22:15:38.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec40f44700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ea4da0 0x1f3a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:38.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec0b7fe700 10 client.4928 ms_handle_connect on v2:172.21.15.185:6834/959337378 2020-11-16T22:15:38.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.719+0000 7fec40f44700 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ea4da0 0x1f3a9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=4e8e3d417e77909c server_cookie=538354f9d873da87 in_seq=0 out_seq=0 2020-11-16T22:15:38.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec0b7fe700 1 -- 172.21.15.185:0/214903029 <== mds.2 v2:172.21.15.185:6834/959337378 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1cc9f50 con 0x1ea4da0 2020-11-16T22:15:38.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec0b7fe700 10 client.4928 handle_command_reply: tid=0 2020-11-16T22:15:38.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 client.4928 shutdown 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 msgr2=0x1eda510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fec38047d40 tx=0x7fec38026000).stop 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 shutdown_connections 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:6834/959337378,v1:172.21.15.185:6835/959337378] conn(0x1ea4da0 0x1f3a9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e54e40 0x1f349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec0b7fe700 0 client.4928 ms_handle_reset on v2:172.21.15.185:6834/959337378 2020-11-16T22:15:38.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1c18bb0 0x1eda510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 --2- 172.21.15.185:0/214903029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c867b0 0x1f344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 >> 172.21.15.185:0/214903029 conn(0x7fec10002320 msgr2=0x1e9f700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 shutdown_connections 2020-11-16T22:15:38.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 1 -- 172.21.15.185:0/214903029 wait complete. 2020-11-16T22:15:38.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.739+0000 7fec5243a740 20 client.4928 trim_cache size 0 max 16384 2020-11-16T22:15:38.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 msgr2=0x7fec14057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 0x7fec14057520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fec3c145e70 tx=0x7fec2c006df0).stop 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 msgr2=0x7fec3c113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c113a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fec3800d910 tx=0x7fec38008040).stop 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 shutdown_connections 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec14055060 0x7fec14057520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec3c1475d0 0x7fec3c13d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec3c0b4470 0x7fec3c113a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 --2- 172.21.15.185:0/1147149441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec3c0b5d70 0x7fec3c143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 >> 172.21.15.185:0/1147149441 conn(0x7fec3c05ba80 msgr2=0x7fec3c144b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 shutdown_connections 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:38.743+0000 7fec42747700 1 -- 172.21.15.185:0/1147149441 wait complete. 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:15:38.731 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:15:38.732 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:38.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:38.747 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:15:38.804 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/ 2020-11-16T22:15:38.805 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4 2020-11-16T22:15:38.805 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5 2020-11-16T22:15:38.805 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:15:38.805 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:15:38.805 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:15:38.810 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:38.811 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:38.811 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:38.811 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:38.812 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:15:38.812 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:38.921 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627778 2020-11-16T22:15:39.112 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:39.113 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:39.113 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:39.113 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:39.114 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:15:39.114 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:39.225 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627779 2020-11-16T22:15:39.416 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:39.417 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:39.417 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:39.417 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:39.418 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:15:39.418 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:39.534 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627780 2020-11-16T22:15:39.723 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:39.724 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:39.724 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:39.725 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:39.725 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:15:39.725 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:39.883 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627781 2020-11-16T22:15:40.071 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:40.072 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:40.072 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:40.072 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:40.073 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:15:40.073 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:40.184 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627782 2020-11-16T22:15:40.372 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:15:40.372 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:15:40.373 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:15:40.373 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:15:40.373 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:15:40.374 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:15:40.496 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627783 2020-11-16T22:15:40.684 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:40.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 Processor -- start 2020-11-16T22:15:40.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 -- start start 2020-11-16T22:15:40.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d7009c900 0x7f1d7009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1d70144840 0x7f1d700b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7009d410 con 0x7f1d7009b270 2020-11-16T22:15:40.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d70144cf0 con 0x7f1d70144840 2020-11-16T22:15:40.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.823+0000 7f1d78787700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d700b6b20 con 0x7f1d7009c900 2020-11-16T22:15:40.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d7009c900 0x7f1d7009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d7009c900 0x7f1d7009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57714/0 (socket says 172.21.15.185:57714) 2020-11-16T22:15:40.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 -- 172.21.15.185:0/2935089747 learned_addr learned my addr 172.21.15.185:0/2935089747 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:40.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76d24700 1 --2- 172.21.15.185:0/2935089747 >> v2:172.21.15.96:3300/0 conn(0x7f1d70144840 0x7f1d700b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 -- 172.21.15.185:0/2935089747 >> v2:172.21.15.96:3300/0 conn(0x7f1d70144840 msgr2=0x7f1d700b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:40.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 --2- 172.21.15.185:0/2935089747 >> v2:172.21.15.96:3300/0 conn(0x7f1d70144840 0x7f1d700b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:40.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 -- 172.21.15.185:0/2935089747 >> v1:172.21.15.96:6789/0 conn(0x7f1d7009b270 legacy=0x7f1d701476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:40.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 -- 172.21.15.185:0/2935089747 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7009b010 con 0x7f1d7009c900 2020-11-16T22:15:40.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d76523700 1 --2- 172.21.15.185:0/2935089747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d7009cce0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f1d700bb700 tx=0x7f1d60006eb0).ready entity=mon.1 client_cookie=ec8fda9b7e76883d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d75521700 1 -- 172.21.15.185:0/2935089747 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60009810 con 0x7f1d7009c900 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d75521700 1 -- 172.21.15.185:0/2935089747 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60009970 con 0x7f1d7009c900 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d75521700 1 -- 172.21.15.185:0/2935089747 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60009cf0 con 0x7f1d7009c900 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 -- 172.21.15.185:0/2935089747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 msgr2=0x7f1d7009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 --2- 172.21.15.185:0/2935089747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d7009cce0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f1d700bb700 tx=0x7f1d60006eb0).stop 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 -- 172.21.15.185:0/2935089747 shutdown_connections 2020-11-16T22:15:40.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 --2- 172.21.15.185:0/2935089747 >> v2:172.21.15.96:3300/0 conn(0x7f1d70144840 0x7f1d700b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:40.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 --2- 172.21.15.185:0/2935089747 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d7009cce0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:40.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 -- 172.21.15.185:0/2935089747 >> 172.21.15.185:0/2935089747 conn(0x7f1d70068330 msgr2=0x7f1d700ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:40.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 -- 172.21.15.185:0/2935089747 shutdown_connections 2020-11-16T22:15:40.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.827+0000 7f1d78787700 1 -- 172.21.15.185:0/2935089747 wait complete. 2020-11-16T22:15:40.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 Processor -- start 2020-11-16T22:15:40.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 -- start start 2020-11-16T22:15:40.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d70156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70144840 0x7f1d70157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7013ca50 con 0x7f1d70144840 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7013eea0 con 0x7f1d7009c900 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7013db30 con 0x7f1d7009b270 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d76523700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d70156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d76d24700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70144840 0x7f1d70157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d76523700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51164/0 (socket says 172.21.15.185:51164) 2020-11-16T22:15:40.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d76523700 1 -- 172.21.15.185:0/3573490762 learned_addr learned my addr 172.21.15.185:0/3573490762 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:40.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 -- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 msgr2=0x7f1d70146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:40.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:40.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 -- 172.21.15.185:0/3573490762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70144840 msgr2=0x7f1d70157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:40.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70144840 0x7f1d70157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 -- 172.21.15.185:0/3573490762 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7013ddd0 con 0x7f1d7009c900 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d76523700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d75d22700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d70156f80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1d6c00c710 tx=0x7f1d6c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c007020 con 0x7f1d7009c900 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c007180 con 0x7f1d7009c900 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0064f0 con 0x7f1d7009c900 2020-11-16T22:15:40.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.831+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7013f0d0 con 0x7f1d7009c900 2020-11-16T22:15:40.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d700b4f20 con 0x7f1d7009c900 2020-11-16T22:15:40.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d70156df0 con 0x7f1d7009c900 2020-11-16T22:15:40.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1d6c010030 con 0x7f1d7009c900 2020-11-16T22:15:40.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d5e7fc700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 0x7f1d48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:40.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d6c06d2e0 con 0x7f1d7009c900 2020-11-16T22:15:40.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d76523700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 0x7f1d48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:40.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.835+0000 7f1d76523700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 0x7f1d48057520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1d6000b1b0 tx=0x7f1d60015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:40.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:40.843+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1d6c043540 con 0x7f1d7009c900 2020-11-16T22:15:41.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.043+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d7005d4e0 con 0x7f1d7009c900 2020-11-16T22:15:41.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.043+0000 7f1d5e7fc700 1 -- 172.21.15.185:0/3573490762 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7f1d6c001e60 con 0x7f1d7009c900 2020-11-16T22:15:41.031 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:41.031 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:41.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 msgr2=0x7f1d48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 0x7f1d48057520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1d6000b1b0 tx=0x7f1d60015000).stop 2020-11-16T22:15:41.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 msgr2=0x7f1d70156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d70156f80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f1d6c00c710 tx=0x7f1d6c007b90).stop 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 shutdown_connections 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d48055060 0x7f1d48057520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70144840 0x7f1d70157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d7009b270 0x7f1d70146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 --2- 172.21.15.185:0/3573490762 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d7009c900 0x7f1d70156f80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 >> 172.21.15.185:0/3573490762 conn(0x7f1d70068330 msgr2=0x7f1d70141860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 shutdown_connections 2020-11-16T22:15:41.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.047+0000 7f1d78787700 1 -- 172.21.15.185:0/3573490762 wait complete. 2020-11-16T22:15:41.035 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:15:41.046 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b tag path /d1/d2/d3 tag123 2020-11-16T22:15:41.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 Processor -- start 2020-11-16T22:15:41.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 -- start start 2020-11-16T22:15:41.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54d40b6120 0x7f54d4144de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d40b51e0 con 0x7f54d40b75a0 2020-11-16T22:15:41.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d41452d0 con 0x7f54d40b6120 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54dbea3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40b4ca0 con 0x7f54d40b6890 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54d40b6120 0x7f54d4144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51176/0 (socket says 172.21.15.185:51176) 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d9c3f700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f54d40b75a0 0x7f54d40b5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51166/0 (socket says 172.21.15.185:51166) 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54da440700 1 -- 172.21.15.185:0/2062049323 learned_addr learned my addr 172.21.15.185:0/2062049323 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54d40b6120 0x7f54d4144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57730/0 (socket says 172.21.15.185:57730) 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 -- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 msgr2=0x7f54d412d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 -- 172.21.15.185:0/2062049323 >> v1:172.21.15.185:6789/0 conn(0x7f54d40b75a0 legacy=0x7f54d40b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d8c3d700 1 -- 172.21.15.185:0/2062049323 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329309922 0 0) 0x7f54d40b51e0 con 0x7f54d40b75a0 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 -- 172.21.15.185:0/2062049323 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d40b4f40 con 0x7f54d40b6120 2020-11-16T22:15:41.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54da440700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d943e700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d4144de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f54d000b9a0 tx=0x7f54d0006df0).ready entity=mon.1 client_cookie=94433d7a43e7d889 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.191+0000 7f54d8c3d700 1 -- 172.21.15.185:0/2062049323 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d00097b0 con 0x7f54d40b6120 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54d8c3d700 1 -- 172.21.15.185:0/2062049323 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0009910 con 0x7f54d40b6120 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54d8c3d700 1 -- 172.21.15.185:0/2062049323 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0009c90 con 0x7f54d40b6120 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 msgr2=0x7f54d4144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d4144de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f54d000b9a0 tx=0x7f54d0006df0).stop 2020-11-16T22:15:41.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- 172.21.15.185:0/2062049323 shutdown_connections 2020-11-16T22:15:41.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d412d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- 172.21.15.185:0/2062049323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d4144de0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- 172.21.15.185:0/2062049323 >> 172.21.15.185:0/2062049323 conn(0x7f54d4067860 msgr2=0x7f54d4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- 172.21.15.185:0/2062049323 shutdown_connections 2020-11-16T22:15:41.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- 172.21.15.185:0/2062049323 wait complete. 2020-11-16T22:15:41.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 Processor -- start 2020-11-16T22:15:41.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- start start 2020-11-16T22:15:41.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d41481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d4129330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54d40b75a0 0x7f54d4143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4157060 con 0x7f54d40b75a0 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4157300 con 0x7f54d40b6120 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.195+0000 7f54dbea3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d41575a0 con 0x7f54d40b6890 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d943e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d4129330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d41481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d943e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d4129330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51180/0 (socket says 172.21.15.185:51180) 2020-11-16T22:15:41.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d943e700 1 -- 172.21.15.185:0/3765366284 learned_addr learned my addr 172.21.15.185:0/3765366284 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54da440700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54d40b75a0 0x7f54d4143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 msgr2=0x7f54d4129330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d4129330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3765366284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54d40b75a0 msgr2=0x7f54d4143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54d40b75a0 0x7f54d4143670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3765366284 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d4157840 con 0x7f54d40b6120 2020-11-16T22:15:41.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d41481a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f54c800c730 tx=0x7f54c8007960).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 -- 172.21.15.185:0/3765366284 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c80072e0 con 0x7f54d40b6120 2020-11-16T22:15:41.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 -- 172.21.15.185:0/3765366284 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8005530 con 0x7f54d40b6120 2020-11-16T22:15:41.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d4129820 con 0x7f54d40b6120 2020-11-16T22:15:41.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 -- 172.21.15.185:0/3765366284 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c80058d0 con 0x7f54d40b6120 2020-11-16T22:15:41.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d40b16c0 con 0x7f54d40b6120 2020-11-16T22:15:41.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 -- 172.21.15.185:0/3765366284 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f54c801a060 con 0x7f54d40b6120 2020-11-16T22:15:41.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 0x7f54ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54c5ffb700 1 -- 172.21.15.185:0/3765366284 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54c806d250 con 0x7f54d40b6120 2020-11-16T22:15:41.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d943e700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 0x7f54ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.199+0000 7f54d943e700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 0x7f54ac057520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f54d40aefb0 tx=0x7f54d0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.203+0000 7f54ea934740 1 Processor -- start 2020-11-16T22:15:41.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.203+0000 7f54ea934740 1 -- start start 2020-11-16T22:15:41.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2039c20 0x22f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2151520 con 0x2039c20 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22059a0 con 0x22ff860 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20c5760 con 0x20a7820 2020-11-16T22:15:41.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51188/0 (socket says 172.21.15.185:51188) 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57746/0 (socket says 172.21.15.185:57746) 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3201965933 learned_addr learned my addr 172.21.15.185:0/3201965933 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d943e700 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2039c20 0x22f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 msgr2=0x22fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3201965933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2039c20 msgr2=0x22f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2039c20 0x22f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 -- 172.21.15.185:0/3201965933 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20c0610 con 0x20a7820 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54da440700 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f54c8043670 tx=0x7f54c8010000).ready entity=mon.2 client_cookie=caadd4898a052ca2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54b2ffd700 1 -- 172.21.15.185:0/3201965933 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c804e070 con 0x20a7820 2020-11-16T22:15:41.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54b2ffd700 1 -- 172.21.15.185:0/3201965933 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c806d0d0 con 0x20a7820 2020-11-16T22:15:41.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54b2ffd700 1 -- 172.21.15.185:0/3201965933 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8046660 con 0x20a7820 2020-11-16T22:15:41.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 msgr2=0x22ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f54c8043670 tx=0x7f54c8010000).stop 2020-11-16T22:15:41.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- 172.21.15.185:0/3201965933 shutdown_connections 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22ff860 0x22fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22ff370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 --2- 172.21.15.185:0/3201965933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2039c20 0x22f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- 172.21.15.185:0/3201965933 >> 172.21.15.185:0/3201965933 conn(0x2275eb0 msgr2=0x22fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- 172.21.15.185:0/3201965933 shutdown_connections 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- 172.21.15.185:0/3201965933 wait complete. 2020-11-16T22:15:41.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 -- start start 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.207+0000 7f54ea934740 1 Processor -- start 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x230d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x22f8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f2750 con 0x20a7820 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f2890 con 0x2039c20 2020-11-16T22:15:41.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x21f3b00 con 0x22e34c0 2020-11-16T22:15:41.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 10 client.0 ll_register_callbacks cb 0x21e8b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:41.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x230d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51192/0 (socket says 172.21.15.185:51192) 2020-11-16T22:15:41.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 -- 172.21.15.185:0/2946991855 learned_addr learned my addr 172.21.15.185:0/2946991855 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x230d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57752/0 (socket says 172.21.15.185:57752) 2020-11-16T22:15:41.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54d943e700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x22f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 -- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 msgr2=0x230d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x230d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 -- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 msgr2=0x22f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x22f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 -- 172.21.15.185:0/2946991855 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21e32f0 con 0x22e34c0 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54da440700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f54cc014f70 tx=0x7f54cc008040).ready entity=mon.2 client_cookie=6dac844ffb94e7de server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:41.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc012d90 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54cc016430 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20e3cc0 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54cc016800 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 10 client.4943 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x2019040 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20e3cc0 con 0x22e34c0 2020-11-16T22:15:41.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f54cc01f3f0 con 0x22e34c0 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 10 client.4943 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 10 client.4943 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 20 client.4943 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2309660 0x2309a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 4 client.4943 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x20eafc0 con 0x2309660 2020-11-16T22:15:41.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2309660 0x2309a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54a7fff700 10 client.4943 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.211+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2309660 0x2309a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9c87496179aa86c9 server_cookie=2ec275ef869058fe in_seq=0 out_seq=0 2020-11-16T22:15:41.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54a7fff700 1 -- 172.21.15.185:0/2946991855 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20eafc0 con 0x2309660 2020-11-16T22:15:41.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54a7fff700 10 client.4943 handle_command_reply: tid=0 2020-11-16T22:15:41.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 client.4943 shutdown 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 msgr2=0x22e50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f54cc014f70 tx=0x7f54cc008040).stop 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 shutdown_connections 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2309660 0x2309a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x230d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22e34c0 0x22e50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 --2- 172.21.15.185:0/2946991855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x22f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 >> 172.21.15.185:0/2946991855 conn(0x2275eb0 msgr2=0x22fbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54a7fff700 0 client.4943 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 shutdown_connections 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 1 -- 172.21.15.185:0/2946991855 wait complete. 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.215+0000 7f54ea934740 20 client.4943 trim_cache size 0 max 16384 2020-11-16T22:15:41.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 Processor -- start 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 -- start start 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 0x22e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x22f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21caeb0 con 0x2275eb0 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22c51c0 con 0x2039c20 2020-11-16T22:15:41.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x229f4b0 con 0x20a7820 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54d943e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x22f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54d943e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x22f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57756/0 (socket says 172.21.15.185:57756) 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.231+0000 7f54d943e700 1 -- 172.21.15.185:0/1796579830 learned_addr learned my addr 172.21.15.185:0/1796579830 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 0x22e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 -- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 msgr2=0x22f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x22f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 -- 172.21.15.185:0/1796579830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 msgr2=0x22e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 0x22e2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 -- 172.21.15.185:0/1796579830 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00081d0 con 0x20a7820 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 0x22e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22fb640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f54cc01fe60 tx=0x7f54cc022000).ready entity=mon.2 client_cookie=112d8777437ba26 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54b37fe700 1 -- 172.21.15.185:0/1796579830 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc009800 con 0x20a7820 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54b37fe700 1 -- 172.21.15.185:0/1796579830 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc002030 con 0x20a7820 2020-11-16T22:15:41.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54b37fe700 1 -- 172.21.15.185:0/1796579830 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc01fb50 con 0x20a7820 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 msgr2=0x22fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22fb640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f54cc01fe60 tx=0x7f54cc022000).stop 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- 172.21.15.185:0/1796579830 shutdown_connections 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20a7820 0x22fb640 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2275eb0 0x22e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- 172.21.15.185:0/1796579830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x22f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- 172.21.15.185:0/1796579830 >> 172.21.15.185:0/1796579830 conn(0x7f54a0002320 msgr2=0x22bfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- 172.21.15.185:0/1796579830 shutdown_connections 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- 172.21.15.185:0/1796579830 wait complete. 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- start start 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 Processor -- start 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x23564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2275eb0 0x23569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2303cd0 con 0x20a7820 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22ff340 con 0x2039c20 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2303a30 con 0x2275eb0 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54ea934740 10 client.0 ll_register_callbacks cb 0x22e4980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54da440700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2275eb0 0x23569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54d9c3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57764/0 (socket says 172.21.15.185:57764) 2020-11-16T22:15:41.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.235+0000 7f54d9c3f700 1 -- 172.21.15.185:0/2210688508 learned_addr learned my addr 172.21.15.185:0/2210688508 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d943e700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x23564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 -- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2275eb0 msgr2=0x23569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2275eb0 0x23569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 -- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 msgr2=0x23564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x23564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 -- 172.21.15.185:0/2210688508 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2057790 con 0x2039c20 2020-11-16T22:15:41.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d9c3f700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f54c8043670 tx=0x7f54c805c280).ready entity=mon.1 client_cookie=a4d552d3cf0ee1e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c805b070 con 0x2039c20 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22bae80 con 0x2039c20 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54c8043240 con 0x2039c20 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c80462e0 con 0x2039c20 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 10 client.5248 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:41.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x2062e70 con 0x2039c20 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22bae80 con 0x2039c20 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f54c8005a30 con 0x2039c20 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 10 client.5248 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 10 client.5248 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 20 client.5248 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x22e34c0 0x2360b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 4 client.5248 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",t,a,g, ,p,a,t,h,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",t,a,g,",:, ,",t,a,g,1,2,3,",}] 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " t a g p a t h " , " p a t h " : " / d 1 / d 2 / d 3 " , " t a g " : " t a g 1 2 3 " }) v1 -- 0x20c8240 con 0x22e34c0 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d943e700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x22e34c0 0x2360b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54a7fff700 10 client.5248 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.239+0000 7f54d943e700 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x22e34c0 0x2360b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4ec7858e5f05de19 server_cookie=8e99031a84d3d2f9 in_seq=0 out_seq=0 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54a7fff700 1 -- 172.21.15.185:0/2210688508 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+80 (crc 0 0 0) 0x20c8240 con 0x22e34c0 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54a7fff700 10 client.5248 handle_command_reply: tid=0 2020-11-16T22:15:41.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 client.5248 shutdown 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 msgr2=0x2303060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f54c8043670 tx=0x7f54c805c280).stop 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 shutdown_connections 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x22e34c0 0x2360b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2039c20 0x2303060 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2275eb0 0x23569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54a7fff700 0 client.5248 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 --2- 172.21.15.185:0/2210688508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20a7820 0x23564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 >> 172.21.15.185:0/2210688508 conn(0x7f54a0002320 msgr2=0x22f5100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 shutdown_connections 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 1 -- 172.21.15.185:0/2210688508 wait complete. 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54ea934740 20 client.5248 trim_cache size 0 max 16384 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 msgr2=0x7f54ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 0x7f54ac057520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f54d40aefb0 tx=0x7f54d0015000).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 msgr2=0x7f54d41481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d41481a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f54c800c730 tx=0x7f54c8007960).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 shutdown_connections 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54ac055060 0x7f54ac057520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54d40b6120 0x7f54d41481a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54d40b75a0 0x7f54d4143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 --2- 172.21.15.185:0/3765366284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54d40b6890 0x7f54d4129330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 >> 172.21.15.185:0/3765366284 conn(0x7f54d4067860 msgr2=0x7f54d40baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 shutdown_connections 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.243+0000 7f54dbea3700 1 -- 172.21.15.185:0/3765366284 wait complete. 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0, 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stdout: "scrub_tag": "tag123", 2020-11-16T22:15:41.428 INFO:teuthology.orchestra.run.smithi185.stdout: "mode": "asynchronous" 2020-11-16T22:15:41.429 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:41.429 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:41.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.575+0000 7f7bcb59e700 1 Processor -- start 2020-11-16T22:15:41.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.575+0000 7f7bcb59e700 1 -- start start 2020-11-16T22:15:41.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7bcc0b7470 0x7f7bcc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7bcc0682b0 0x7f7bcc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc067510 con 0x7f7bcc0682b0 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0677b0 con 0x7f7bcc0b7470 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b17a0 con 0x7f7bcc067070 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bca59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7bcc0b7470 0x7f7bcc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51216/0 (socket says 172.21.15.185:51216) 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 -- 172.21.15.185:0/2836836775 learned_addr learned my addr 172.21.15.185:0/2836836775 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcad9d700 1 --2- 172.21.15.185:0/2836836775 >> v2:172.21.15.96:3300/0 conn(0x7f7bcc0682b0 0x7f7bcc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 -- 172.21.15.185:0/2836836775 >> v2:172.21.15.185:3300/0 conn(0x7f7bcc0b7470 msgr2=0x7f7bcc0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 --2- 172.21.15.185:0/2836836775 >> v2:172.21.15.185:3300/0 conn(0x7f7bcc0b7470 0x7f7bcc0b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 -- 172.21.15.185:0/2836836775 >> v2:172.21.15.96:3300/0 conn(0x7f7bcc0682b0 msgr2=0x7f7bcc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 --2- 172.21.15.185:0/2836836775 >> v2:172.21.15.96:3300/0 conn(0x7f7bcc0682b0 0x7f7bcc0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 -- 172.21.15.185:0/2836836775 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0b1a40 con 0x7f7bcc067070 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc1d9b700 1 --2- 172.21.15.185:0/2836836775 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f7bcc0b7d50 tx=0x7f7bb4006eb0).ready entity=mon.2 client_cookie=6d0841d43e143b5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc9d9b700 1 -- 172.21.15.185:0/2836836775 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400f810 con 0x7f7bcc067070 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc9d9b700 1 -- 172.21.15.185:0/2836836775 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb400f970 con 0x7f7bcc067070 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bc9d9b700 1 -- 172.21.15.185:0/2836836775 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400fcf0 con 0x7f7bcc067070 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- 172.21.15.185:0/2836836775 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 msgr2=0x7f7bcc143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/2836836775 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f7bcc0b7d50 tx=0x7f7bb4006eb0).stop 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- 172.21.15.185:0/2836836775 shutdown_connections 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/2836836775 >> v2:172.21.15.96:3300/0 conn(0x7f7bcc0682b0 0x7f7bcc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/2836836775 >> v2:172.21.15.185:3300/0 conn(0x7f7bcc0b7470 0x7f7bcc0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/2836836775 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc067070 0x7f7bcc143a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- 172.21.15.185:0/2836836775 >> 172.21.15.185:0/2836836775 conn(0x7f7bcc0637b0 msgr2=0x7f7bcc0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- 172.21.15.185:0/2836836775 shutdown_connections 2020-11-16T22:15:41.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.579+0000 7f7bcb59e700 1 -- 172.21.15.185:0/2836836775 wait complete. 2020-11-16T22:15:41.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 Processor -- start 2020-11-16T22:15:41.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 -- start start 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 0x7f7bcc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 0x7f7bcc0b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bcc0b7470 0x7f7bcc0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0acbd0 con 0x7f7bcc0b7470 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0ad0b0 con 0x7f7bcc067070 2020-11-16T22:15:41.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcb59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc14f9a0 con 0x7f7bcc0682b0 2020-11-16T22:15:41.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bc1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 0x7f7bcc0b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bca59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 0x7f7bcc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bc1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 0x7f7bcc0b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51220/0 (socket says 172.21.15.185:51220) 2020-11-16T22:15:41.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bc1d9b700 1 -- 172.21.15.185:0/1884166374 learned_addr learned my addr 172.21.15.185:0/1884166374 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.583+0000 7f7bcad9d700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bcc0b7470 0x7f7bcc0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 -- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 msgr2=0x7f7bcc0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 0x7f7bcc0b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 -- 172.21.15.185:0/1884166374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bcc0b7470 msgr2=0x7f7bcc0adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bcc0b7470 0x7f7bcc0adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 -- 172.21.15.185:0/1884166374 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc14fc00 con 0x7f7bcc067070 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bca59c700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 0x7f7bcc0b0530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7bbc00eba0 tx=0x7f7bbc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00c820 con 0x7f7bcc067070 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00c980 con 0x7f7bcc067070 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc0ace00 con 0x7f7bcc067070 2020-11-16T22:15:41.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00cd00 con 0x7f7bcc067070 2020-11-16T22:15:41.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc0ac760 con 0x7f7bcc067070 2020-11-16T22:15:41.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc05d200 con 0x7f7bcc067070 2020-11-16T22:15:41.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7bbc012600 con 0x7f7bcc067070 2020-11-16T22:15:41.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 0x7f7ba8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7bbc06e010 con 0x7f7bcc067070 2020-11-16T22:15:41.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.587+0000 7f7bc1d9b700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 0x7f7ba8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.591+0000 7f7bc1d9b700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 0x7f7ba8057520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f7bcc0ac6f0 tx=0x7f7bb4017000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.591+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7bbc044270 con 0x7f7bcc067070 2020-11-16T22:15:41.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.795+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bcc05c6d0 con 0x7f7bcc067070 2020-11-16T22:15:41.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bc2ffd700 1 -- 172.21.15.185:0/1884166374 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7f7bbc00ce60 con 0x7f7bcc067070 2020-11-16T22:15:41.783 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:41.783 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:41.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 msgr2=0x7f7ba8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 0x7f7ba8057520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f7bcc0ac6f0 tx=0x7f7bb4017000).stop 2020-11-16T22:15:41.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 msgr2=0x7f7bcc0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 0x7f7bcc0b0530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7bbc00eba0 tx=0x7f7bbc006df0).stop 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 shutdown_connections 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ba8055060 0x7f7ba8057520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7bcc067070 0x7f7bcc0b0530 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bcc0b7470 0x7f7bcc0adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 --2- 172.21.15.185:0/1884166374 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7bcc0682b0 0x7f7bcc0b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 >> 172.21.15.185:0/1884166374 conn(0x7f7bcc0637b0 msgr2=0x7f7bcc1404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 shutdown_connections 2020-11-16T22:15:41.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.799+0000 7f7bcb59e700 1 -- 172.21.15.185:0/1884166374 wait complete. 2020-11-16T22:15:41.787 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:15:41.798 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:15:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 Processor -- start 2020-11-16T22:15:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 -- start start 2020-11-16T22:15:41.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 0x7fb0ac1414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb0ac145d70 0x7fb0ac1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ac0b2f30 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ac060fe0 con 0x7fb0ac145d70 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b2651700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac061280 con 0x7fb0ac05ba80 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51232/0 (socket says 172.21.15.185:51232) 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 -- 172.21.15.185:0/1447540319 learned_addr learned my addr 172.21.15.185:0/1447540319 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b0e4e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb0ac145d70 0x7fb0ac1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b164f700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 0x7fb0ac1414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 -- 172.21.15.185:0/1447540319 >> v2:172.21.15.185:3300/0 conn(0x7fb0ac145d70 msgr2=0x7fb0ac1419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.185:3300/0 conn(0x7fb0ac145d70 0x7fb0ac1419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 -- 172.21.15.185:0/1447540319 >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 msgr2=0x7fb0ac1414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 0x7fb0ac1414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 -- 172.21.15.185:0/1447540319 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac060210 con 0x7fb0ac05ba80 2020-11-16T22:15:41.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.939+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/1447540319 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb0ac0b1570 tx=0x7fb0a4006eb0).ready entity=mon.2 client_cookie=cb8f133f0e382e80 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.943+0000 7fb0b164f700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 0x7fb0ac1414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:41.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.943+0000 7fb0a3fff700 1 -- 172.21.15.185:0/1447540319 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4009810 con 0x7fb0ac05ba80 2020-11-16T22:15:41.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0a3fff700 1 -- 172.21.15.185:0/1447540319 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4009970 con 0x7fb0ac05ba80 2020-11-16T22:15:41.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0a3fff700 1 -- 172.21.15.185:0/1447540319 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4009cf0 con 0x7fb0ac05ba80 2020-11-16T22:15:41.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- 172.21.15.185:0/1447540319 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 msgr2=0x7fb0ac1421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 --2- 172.21.15.185:0/1447540319 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb0ac0b1570 tx=0x7fb0a4006eb0).stop 2020-11-16T22:15:41.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- 172.21.15.185:0/1447540319 shutdown_connections 2020-11-16T22:15:41.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 --2- 172.21.15.185:0/1447540319 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac1421c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.96:3300/0 conn(0x7fb0ac0b1a20 0x7fb0ac1414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 --2- 172.21.15.185:0/1447540319 >> v2:172.21.15.185:3300/0 conn(0x7fb0ac145d70 0x7fb0ac1419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- 172.21.15.185:0/1447540319 >> 172.21.15.185:0/1447540319 conn(0x7fb0ac063df0 msgr2=0x7fb0ac0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- 172.21.15.185:0/1447540319 shutdown_connections 2020-11-16T22:15:41.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- 172.21.15.185:0/1447540319 wait complete. 2020-11-16T22:15:41.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 Processor -- start 2020-11-16T22:15:41.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.947+0000 7fb0b2651700 1 -- start start 2020-11-16T22:15:41.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac10fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ac145d70 0x7fb0ac154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac144520 con 0x7fb0ac145d70 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac139f50 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0b0cc0 con 0x7fb0ac05ba80 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57790/0 (socket says 172.21.15.185:57790) 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac10fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/3424515390 learned_addr learned my addr 172.21.15.185:0/3424515390 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.937 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ac145d70 0x7fb0ac154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 msgr2=0x7fb0ac10fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac10fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/3424515390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ac145d70 msgr2=0x7fb0ac154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ac145d70 0x7fb0ac154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/3424515390 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0afe10 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b164f700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb09c00eba0 tx=0x7fb09c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 -- 172.21.15.185:0/3424515390 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00b3a0 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 -- 172.21.15.185:0/3424515390 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c00c8e0 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 -- 172.21.15.185:0/3424515390 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00cca0 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac13b750 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac145270 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 -- 172.21.15.185:0/3424515390 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb09c012600 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 0x7fb084057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0a17fa700 1 -- 172.21.15.185:0/3424515390 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb09c06e070 con 0x7fb0ac0b1a20 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b164f700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 0x7fb084057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.951+0000 7fb0b164f700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 0x7fb084057550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb0a8009db0 tx=0x7fb0a8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.955+0000 7fb0c2344740 1 Processor -- start 2020-11-16T22:15:41.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.955+0000 7fb0c2344740 1 -- start start 2020-11-16T22:15:41.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.955+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2b9f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.955+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 0x2be3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a2c4b0 con 0x29827b0 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae0930 con 0x2914bb0 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29a06f0 con 0x2be31b0 2020-11-16T22:15:41.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b1e50700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 0x2be3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57800/0 (socket says 172.21.15.185:57800) 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2394397552 learned_addr learned my addr 172.21.15.185:0/2394397552 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b164f700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2b9f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 msgr2=0x2be3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 0x2be3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2394397552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 msgr2=0x2b9f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2b9f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2394397552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x299b5a0 con 0x2914bb0 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 0x2be3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b164f700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2b9f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb09c047890 tx=0x7fb09c014000).ready entity=mon.1 client_cookie=cd05ce8f3c644520 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb08e7fc700 1 -- 172.21.15.185:0/2394397552 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c06c2e0 con 0x2914bb0 2020-11-16T22:15:41.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb08e7fc700 1 -- 172.21.15.185:0/2394397552 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c06c440 con 0x2914bb0 2020-11-16T22:15:41.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb08e7fc700 1 -- 172.21.15.185:0/2394397552 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c047420 con 0x2914bb0 2020-11-16T22:15:41.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 msgr2=0x2ba0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb09c047890 tx=0x7fb09c014000).stop 2020-11-16T22:15:41.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- 172.21.15.185:0/2394397552 shutdown_connections 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2be31b0 0x2be3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2b9f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2394397552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba0a90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- 172.21.15.185:0/2394397552 >> 172.21.15.185:0/2394397552 conn(0x2b50e40 msgr2=0x2b9ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- 172.21.15.185:0/2394397552 shutdown_connections 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- 172.21.15.185:0/2394397552 wait complete. 2020-11-16T22:15:41.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- start start 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 Processor -- start 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 0x2b9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2be31b0 0x2bda1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2acd6e0 con 0x2be31b0 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2acd820 con 0x29827b0 2020-11-16T22:15:41.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2acea90 con 0x2914bb0 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0c2344740 10 client.0 ll_register_callbacks cb 0x2ac3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 0x2b9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51248/0 (socket says 172.21.15.185:51248) 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 0x2b9f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57806/0 (socket says 172.21.15.185:57806) 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b164f700 1 -- 172.21.15.185:0/960352731 learned_addr learned my addr 172.21.15.185:0/960352731 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.959+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2be31b0 0x2bda1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 -- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 msgr2=0x2b9f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 0x2b9f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 -- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2be31b0 msgr2=0x2bda1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2be31b0 0x2bda1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 -- 172.21.15.185:0/960352731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2abe280 con 0x2914bb0 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b164f700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb0a8012390 tx=0x7fb0a8014000).ready entity=mon.2 client_cookie=6e0991da34687021 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:41.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012e70 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2b816c0 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb0a800fd50 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a8020690 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 10 client.4967 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x28f3fd0 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2b816c0 con 0x2914bb0 2020-11-16T22:15:41.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb0a801fa30 con 0x2914bb0 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 10 client.4967 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 10 client.4967 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 20 client.4967 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2bcec30 0x2bec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 4 client.4967 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x29bec50 con 0x2bcec30 2020-11-16T22:15:41.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2bcec30 0x2bec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 10 client.4967 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2bcec30 0x2bec100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=12cfab83b4cc751b server_cookie=67a4b3e43dab5e76 in_seq=0 out_seq=0 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 1 -- 172.21.15.185:0/960352731 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x29bec50 con 0x2bcec30 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 10 client.4967 handle_command_reply: tid=0 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 client.4967 shutdown 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 msgr2=0x2be9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb0a8012390 tx=0x7fb0a8014000).stop 2020-11-16T22:15:41.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 shutdown_connections 2020-11-16T22:15:41.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2be31b0 0x2bda1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be9570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2bcec30 0x2bec100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 --2- 172.21.15.185:0/960352731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29827b0 0x2b9f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb07b7fe700 0 client.4967 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 >> 172.21.15.185:0/960352731 conn(0x2b50e40 msgr2=0x2bd3230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 shutdown_connections 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.963+0000 7fb0c2344740 1 -- 172.21.15.185:0/960352731 wait complete. 2020-11-16T22:15:41.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.967+0000 7fb0c2344740 20 client.4967 trim_cache size 0 max 16384 2020-11-16T22:15:41.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.983+0000 7fb0c2344740 1 Processor -- start 2020-11-16T22:15:41.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.983+0000 7fb0c2344740 1 -- start start 2020-11-16T22:15:41.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2be92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aa5e40 con 0x29827b0 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b9ae40 con 0x2b50e40 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b9d500 con 0x2914bb0 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57814/0 (socket says 172.21.15.185:57814) 2020-11-16T22:15:41.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 -- 172.21.15.185:0/369542974 learned_addr learned my addr 172.21.15.185:0/369542974 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2be92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 -- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 msgr2=0x2be8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 -- 172.21.15.185:0/369542974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 msgr2=0x2be92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2be92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 -- 172.21.15.185:0/369542974 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0800081d0 con 0x2b50e40 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2be92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb0a8013fb0 tx=0x7fb0a8023000).ready entity=mon.1 client_cookie=fef2b5abe5235f5e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb08effd700 1 -- 172.21.15.185:0/369542974 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012e70 con 0x2b50e40 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb08effd700 1 -- 172.21.15.185:0/369542974 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8009af0 con 0x2b50e40 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb08effd700 1 -- 172.21.15.185:0/369542974 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a801e470 con 0x2b50e40 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 msgr2=0x2bce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb0a8013fb0 tx=0x7fb0a8023000).stop 2020-11-16T22:15:41.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- 172.21.15.185:0/369542974 shutdown_connections 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b50e40 0x2bce210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29827b0 0x2be92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- 172.21.15.185:0/369542974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2914bb0 0x2be8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- 172.21.15.185:0/369542974 >> 172.21.15.185:0/369542974 conn(0x7fb080002320 msgr2=0x2bd3230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- 172.21.15.185:0/369542974 shutdown_connections 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- 172.21.15.185:0/369542974 wait complete. 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- start start 2020-11-16T22:15:41.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 Processor -- start 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b50e40 0x2c3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ba3090 con 0x2b50e40 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ba2aa0 con 0x2914bb0 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ba5000 con 0x29827b0 2020-11-16T22:15:41.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0c2344740 10 client.0 ll_register_callbacks cb 0x2bd72e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b0e4e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51262/0 (socket says 172.21.15.185:51262) 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2959506424 learned_addr learned my addr 172.21.15.185:0/2959506424 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b164f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57820/0 (socket says 172.21.15.185:57820) 2020-11-16T22:15:41.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.987+0000 7fb0b1e50700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b50e40 0x2c3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 msgr2=0x2ba3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b50e40 msgr2=0x2c3ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:41.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b50e40 0x2c3ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 -- 172.21.15.185:0/2959506424 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2932720 con 0x29827b0 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b164f700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b0e4e700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb09c028040 tx=0x7fb09c014040).ready entity=mon.2 client_cookie=da3e546c7b3b5d37 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00eeb0 con 0x29827b0 2020-11-16T22:15:41.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb09c033df0 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb09c06c440 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb09c047420 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 10 client.4973 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x293de00 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c06c440 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb09c03c080 con 0x29827b0 2020-11-16T22:15:41.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 10 client.4973 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 10 client.4973 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 20 client.4973 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2be31b0 0x2c3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 4 client.4973 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x29c5f50 con 0x2be31b0 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b164f700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2be31b0 0x2c3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:41.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 10 client.4973 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b164f700 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2be31b0 0x2c3f200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c052d8897cc4085d server_cookie=7e9361c001a14b94 in_seq=0 out_seq=0 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 1 -- 172.21.15.185:0/2959506424 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x29c5f50 con 0x2be31b0 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 10 client.4973 handle_command_reply: tid=0 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 client.4973 shutdown 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 msgr2=0x2ba3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb09c028040 tx=0x7fb09c014040).stop 2020-11-16T22:15:41.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 shutdown_connections 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x2be31b0 0x2c3f200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2914bb0 0x2ba3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b50e40 0x2c3ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 --2- 172.21.15.185:0/2959506424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29827b0 0x2ba3740 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 >> 172.21.15.185:0/2959506424 conn(0x7fb080002320 msgr2=0x2bce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb07b7fe700 0 client.4973 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:41.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 shutdown_connections 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 1 -- 172.21.15.185:0/2959506424 wait complete. 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0c2344740 20 client.4973 trim_cache size 0 max 16384 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.991+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 msgr2=0x7fb084057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 0x7fb084057550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb0a8009db0 tx=0x7fb0a8008040).stop 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 msgr2=0x7fb0ac0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb09c00eba0 tx=0x7fb09c006df0).stop 2020-11-16T22:15:41.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 shutdown_connections 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb084055090 0x7fb084057550 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0ac145d70 0x7fb0ac154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0ac05ba80 0x7fb0ac10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 --2- 172.21.15.185:0/3424515390 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0ac0b1a20 0x7fb0ac0b4b00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 >> 172.21.15.185:0/3424515390 conn(0x7fb0ac063df0 msgr2=0x7fb0ac13ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:41.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 shutdown_connections 2020-11-16T22:15:41.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:41.995+0000 7fb0b2651700 1 -- 172.21.15.185:0/3424515390 wait complete. 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "tag123": { 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "tag123", 2020-11-16T22:15:41.992 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:15:41.993 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:41.993 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:41.993 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:41.997 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:15:42.968 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:42.974 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:42.977 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:15:46.998 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:47.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.143+0000 7f40c0c25700 1 Processor -- start 2020-11-16T22:15:47.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.143+0000 7f40c0c25700 1 -- start start 2020-11-16T22:15:47.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f40bc0662c0 0x7f40bc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f40bc141190 0x7f40bc141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0b3af0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc061e60 con 0x7f40bc0662c0 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0b3d20 con 0x7f40bc141190 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f40bc141190 0x7f40bc141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bb7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f40bc0b2eb0 0x7f40bc13a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51266/0 (socket says 172.21.15.185:51266) 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bb7fe700 1 -- 172.21.15.185:0/518866467 learned_addr learned my addr 172.21.15.185:0/518866467 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:47.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40baffd700 1 --2- 172.21.15.185:0/518866467 >> v2:172.21.15.96:3300/0 conn(0x7f40bc0662c0 0x7f40bc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 -- 172.21.15.185:0/518866467 >> v2:172.21.15.96:3300/0 conn(0x7f40bc0662c0 msgr2=0x7f40bc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 --2- 172.21.15.185:0/518866467 >> v2:172.21.15.96:3300/0 conn(0x7f40bc0662c0 0x7f40bc0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 -- 172.21.15.185:0/518866467 >> v1:172.21.15.185:6789/0 conn(0x7f40bc0b2eb0 legacy=0x7f40bc13a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40ba7fc700 1 -- 172.21.15.185:0/518866467 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994787610 0 0) 0x7f40bc0b3af0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 -- 172.21.15.185:0/518866467 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc140cc0 con 0x7f40bc141190 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40bbfff700 1 --2- 172.21.15.185:0/518866467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc141190 0x7f40bc141620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f40bc062630 tx=0x7f40b40091d0).ready entity=mon.1 client_cookie=fc42801bf2cb30fc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40ba7fc700 1 -- 172.21.15.185:0/518866467 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400ba70 con 0x7f40bc141190 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40ba7fc700 1 -- 172.21.15.185:0/518866467 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400bbd0 con 0x7f40bc141190 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40ba7fc700 1 -- 172.21.15.185:0/518866467 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4011400 con 0x7f40bc141190 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- 172.21.15.185:0/518866467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc141190 msgr2=0x7f40bc141620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 --2- 172.21.15.185:0/518866467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc141190 0x7f40bc141620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f40bc062630 tx=0x7f40b40091d0).stop 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- 172.21.15.185:0/518866467 shutdown_connections 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 --2- 172.21.15.185:0/518866467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc141190 0x7f40bc141620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 --2- 172.21.15.185:0/518866467 >> v2:172.21.15.96:3300/0 conn(0x7f40bc0662c0 0x7f40bc0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- 172.21.15.185:0/518866467 >> 172.21.15.185:0/518866467 conn(0x7f40bc060130 msgr2=0x7f40bc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- 172.21.15.185:0/518866467 shutdown_connections 2020-11-16T22:15:47.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- 172.21.15.185:0/518866467 wait complete. 2020-11-16T22:15:47.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 Processor -- start 2020-11-16T22:15:47.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.147+0000 7f40c0c25700 1 -- start start 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc0662c0 0x7f40bc0aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 0x7f40bc1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1101b0 con 0x7f40bc141190 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0b0160 con 0x7f40bc0662c0 2020-11-16T22:15:47.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc138770 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40bb7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc0662c0 0x7f40bc0aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51278/0 (socket says 172.21.15.185:51278) 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 -- 172.21.15.185:0/945111090 learned_addr learned my addr 172.21.15.185:0/945111090 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40bbfff700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 0x7f40bc1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 -- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc0662c0 msgr2=0x7f40bc0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc0662c0 0x7f40bc0aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 -- 172.21.15.185:0/945111090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 msgr2=0x7f40bc1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 0x7f40bc1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 -- 172.21.15.185:0/945111090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0ae3e0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40bbfff700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 0x7f40bc1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40baffd700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f40b000ebd0 tx=0x7f40b0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000b340 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000c8e0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0ae680 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000cc60 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc1540b0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc111280 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f40b001c460 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 0x7f4090057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f40b006e0b0 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.151+0000 7f40bb7fe700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 0x7f4090057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.155+0000 7f40bb7fe700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 0x7f4090057520 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f40ac0011a0 tx=0x7f40ac007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.163+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f40bc111280 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.363+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40bc111280 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.363+0000 7f40a37fe700 1 -- 172.21.15.185:0/945111090 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7f40bc111280 con 0x7f40bc0b2eb0 2020-11-16T22:15:47.351 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:47.351 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:47.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 msgr2=0x7f4090057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 0x7f4090057520 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f40ac0011a0 tx=0x7f40ac007a00).stop 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 msgr2=0x7f40bc0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f40b000ebd0 tx=0x7f40b0006df0).stop 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 shutdown_connections 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4090055060 0x7f4090057520 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f40bc0662c0 0x7f40bc0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40bc141190 0x7f40bc1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 --2- 172.21.15.185:0/945111090 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f40bc0b2eb0 0x7f40bc0afac0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 >> 172.21.15.185:0/945111090 conn(0x7f40bc060130 msgr2=0x7f40bc0677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 shutdown_connections 2020-11-16T22:15:47.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.367+0000 7f40c0c25700 1 -- 172.21.15.185:0/945111090 wait complete. 2020-11-16T22:15:47.356 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:15:47.370 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:15:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 Processor -- start 2020-11-16T22:15:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- start start 2020-11-16T22:15:47.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 0x7fa5cc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa5cc0b3920 0x7fa5cc148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc14c660 con 0x7fa5cc0b3080 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0ac120 con 0x7fa5cc0b3920 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0ac3c0 con 0x7fa5cc0b13d0 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1c33700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa5cc0b3080 0x7fa5cc0ac860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51278/0 (socket says 172.21.15.185:51278) 2020-11-16T22:15:47.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1c33700 1 -- 172.21.15.185:0/2566863465 learned_addr learned my addr 172.21.15.185:0/2566863465 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:47.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2566863465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 0x7fa5cc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2434700 1 --2- 172.21.15.185:0/2566863465 >> v2:172.21.15.96:3300/0 conn(0x7fa5cc0b3920 0x7fa5cc148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d0c31700 1 -- 172.21.15.185:0/2566863465 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389497976 0 0) 0x7fa5cc14c660 con 0x7fa5cc0b3080 2020-11-16T22:15:47.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d0c31700 1 -- 172.21.15.185:0/2566863465 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003240 con 0x7fa5cc0b3080 2020-11-16T22:15:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 -- 172.21.15.185:0/2566863465 >> v2:172.21.15.96:3300/0 conn(0x7fa5cc0b3920 msgr2=0x7fa5cc148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2566863465 >> v2:172.21.15.96:3300/0 conn(0x7fa5cc0b3920 0x7fa5cc148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 -- 172.21.15.185:0/2566863465 >> v1:172.21.15.185:6789/0 conn(0x7fa5cc0b3080 legacy=0x7fa5cc0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 -- 172.21.15.185:0/2566863465 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0b20e0 con 0x7fa5cc0b13d0 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2566863465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 0x7fa5cc0b1bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa5c800b9a0 tx=0x7fa5c8006df0).ready entity=mon.2 client_cookie=f7081e898ffcffc6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d0c31700 1 -- 172.21.15.185:0/2566863465 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009680 con 0x7fa5cc0b13d0 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d0c31700 1 -- 172.21.15.185:0/2566863465 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c80097e0 con 0x7fa5cc0b13d0 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d0c31700 1 -- 172.21.15.185:0/2566863465 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009b60 con 0x7fa5cc0b13d0 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2566863465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 msgr2=0x7fa5cc0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2566863465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 0x7fa5cc0b1bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa5c800b9a0 tx=0x7fa5c8006df0).stop 2020-11-16T22:15:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2566863465 shutdown_connections 2020-11-16T22:15:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2566863465 >> v2:172.21.15.96:3300/0 conn(0x7fa5cc0b3920 0x7fa5cc148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2566863465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b13d0 0x7fa5cc0b1bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2566863465 >> 172.21.15.185:0/2566863465 conn(0x7fa5cc060130 msgr2=0x7fa5cc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2566863465 shutdown_connections 2020-11-16T22:15:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.515+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2566863465 wait complete. 2020-11-16T22:15:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 Processor -- start 2020-11-16T22:15:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 -- start start 2020-11-16T22:15:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 0x7fa5cc111ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5cc0af250 0x7fa5cc146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc157bd0 con 0x7fa5cc0b3920 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1586d0 con 0x7fa5cc0af250 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc111fd0 con 0x7fa5cc0b3080 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51290/0 (socket says 172.21.15.185:51290) 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 -- 172.21.15.185:0/2294415051 learned_addr learned my addr 172.21.15.185:0/2294415051 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2434700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5cc0af250 0x7fa5cc146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 0x7fa5cc111ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 -- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5cc0af250 msgr2=0x7fa5cc146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5cc0af250 0x7fa5cc146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 -- 172.21.15.185:0/2294415051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 msgr2=0x7fa5cc111ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 0x7fa5cc111ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 -- 172.21.15.185:0/2294415051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc112260 con 0x7fa5cc0b3080 2020-11-16T22:15:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 0x7fa5cc111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa5c000c710 tx=0x7fa5c0007ad0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5bdffb700 1 -- 172.21.15.185:0/2294415051 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00072e0 con 0x7fa5cc0b3080 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5bdffb700 1 -- 172.21.15.185:0/2294415051 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0007440 con 0x7fa5cc0b3080 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc112500 con 0x7fa5cc0b3080 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.519+0000 7fa5bdffb700 1 -- 172.21.15.185:0/2294415051 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005800 con 0x7fa5cc0b3080 2020-11-16T22:15:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc1469f0 con 0x7fa5cc0b3080 2020-11-16T22:15:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5bdffb700 1 -- 172.21.15.185:0/2294415051 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa5c001a060 con 0x7fa5cc0b3080 2020-11-16T22:15:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5bdffb700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 0x7fa5a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5bdffb700 1 -- 172.21.15.185:0/2294415051 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5c006d530 con 0x7fa5cc0b3080 2020-11-16T22:15:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 0x7fa5a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.523+0000 7fa5d1432700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 0x7fa5a4057520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa5cc068950 tx=0x7fa5c8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 Processor -- start 2020-11-16T22:15:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 -- start start 2020-11-16T22:15:47.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d64bb0 0x1feee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e7c4b0 con 0x1dd27b0 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f30930 con 0x1ff0da0 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df06f0 con 0x1d64bb0 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57854/0 (socket says 172.21.15.185:57854) 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 -- 172.21.15.185:0/3193209606 learned_addr learned my addr 172.21.15.185:0/3193209606 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d1432700 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d64bb0 0x1feee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 -- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d64bb0 msgr2=0x1feee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d64bb0 0x1feee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 -- 172.21.15.185:0/3193209606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 msgr2=0x1ff08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.527+0000 7fa5d2434700 1 -- 172.21.15.185:0/3193209606 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1deb5a0 con 0x1ff0da0 2020-11-16T22:15:47.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d2434700 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fa5c4011370 tx=0x7fa5c40091d0).ready entity=mon.1 client_cookie=a346f3920128d2e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5aaffd700 1 -- 172.21.15.185:0/3193209606 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4014440 con 0x1ff0da0 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5aaffd700 1 -- 172.21.15.185:0/3193209606 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40145a0 con 0x1ff0da0 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5aaffd700 1 -- 172.21.15.185:0/3193209606 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4014920 con 0x1ff0da0 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 msgr2=0x1fef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fa5c4011370 tx=0x7fa5c40091d0).stop 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/3193209606 shutdown_connections 2020-11-16T22:15:47.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ff0da0 0x1fef380 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- 172.21.15.185:0/3193209606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d64bb0 0x1feee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/3193209606 >> 172.21.15.185:0/3193209606 conn(0x1fa0e40 msgr2=0x1feddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/3193209606 shutdown_connections 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/3193209606 wait complete. 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- start start 2020-11-16T22:15:47.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 Processor -- start 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 0x201edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f1d6e0 con 0x1dd27b0 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f1d820 con 0x1d64bb0 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f1ea90 con 0x1ff0da0 2020-11-16T22:15:47.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 10 client.0 ll_register_callbacks cb 0x1f13ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57860/0 (socket says 172.21.15.185:57860) 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1860250329 learned_addr learned my addr 172.21.15.185:0/1860250329 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d2434700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 0x201edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 msgr2=0x201edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 0x201edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 msgr2=0x1ff3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1860250329 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f0e280 con 0x1d64bb0 2020-11-16T22:15:47.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d2434700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 0x201edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa5c0046e30 tx=0x7fa5c0010000).ready entity=mon.1 client_cookie=19fe61c4af91b69 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa59ffff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0043790 con 0x1d64bb0 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5c006d2e0 con 0x1d64bb0 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.531+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1fd16c0 con 0x1d64bb0 2020-11-16T22:15:47.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5c0043450 con 0x1d64bb0 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 10 client.5293 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x1d43fd0 con 0x1d64bb0 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1fd16c0 con 0x1d64bb0 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa5c00463b0 con 0x1d64bb0 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 10 client.5293 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 10 client.5293 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:47.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 20 client.5293 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff1d00 0x2034620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 4 client.5293 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1e0ec50 con 0x1ff1d00 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff1d00 0x2034620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 10 client.5293 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff1d00 0x2034620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=79cea7017533d01f server_cookie=4563a4150bc57ad8 in_seq=0 out_seq=0 2020-11-16T22:15:47.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 1 -- 172.21.15.185:0/1860250329 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1e0ec50 con 0x1ff1d00 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 10 client.5293 handle_command_reply: tid=0 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 client.5293 shutdown 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 msgr2=0x2026410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fa5c0046e30 tx=0x7fa5c0010000).stop 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 shutdown_connections 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff1d00 0x2034620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x2026410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff0da0 0x201edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa59ffff700 0 client.5293 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1860250329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x1ff3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 >> 172.21.15.185:0/1860250329 conn(0x1fa0e40 msgr2=0x1feee20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 shutdown_connections 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 1 -- 172.21.15.185:0/1860250329 wait complete. 2020-11-16T22:15:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.535+0000 7fa5e2928740 20 client.5293 trim_cache size 0 max 16384 2020-11-16T22:15:47.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 Processor -- start 2020-11-16T22:15:47.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 -- start start 2020-11-16T22:15:47.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x2020310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2037b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ef5e40 con 0x1dd27b0 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f4d680 con 0x1d64bb0 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1feeaf0 con 0x1fa0e40 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57870/0 (socket says 172.21.15.185:57870) 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x2020310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 -- 172.21.15.185:0/1495187477 learned_addr learned my addr 172.21.15.185:0/1495187477 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d2434700 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2037b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 -- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 msgr2=0x2020310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x2020310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 -- 172.21.15.185:0/1495187477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 msgr2=0x2037b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2037b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 -- 172.21.15.185:0/1495187477 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980081b0 con 0x1d64bb0 2020-11-16T22:15:47.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa5c800c010 tx=0x7fa5c8013000).ready entity=mon.1 client_cookie=986f9d862de1c449 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.555+0000 7fa5ab7fe700 1 -- 172.21.15.185:0/1495187477 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802f770 con 0x1d64bb0 2020-11-16T22:15:47.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5ab7fe700 1 -- 172.21.15.185:0/1495187477 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c802f8d0 con 0x1d64bb0 2020-11-16T22:15:47.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5ab7fe700 1 -- 172.21.15.185:0/1495187477 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c802fc50 con 0x1d64bb0 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 msgr2=0x20254b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa5c800c010 tx=0x7fa5c8013000).stop 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1495187477 shutdown_connections 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2037b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x2020310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1495187477 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20254b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1495187477 >> 172.21.15.185:0/1495187477 conn(0x7fa598002320 msgr2=0x1ff3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1495187477 shutdown_connections 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1495187477 wait complete. 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- start start 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 Processor -- start 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2083ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x20867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x202d140 con 0x1dd27b0 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x202d3e0 con 0x1d64bb0 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x202d680 con 0x1fa0e40 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 10 client.0 ll_register_callbacks cb 0x20238b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57874/0 (socket says 172.21.15.185:57874) 2020-11-16T22:15:47.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1166196903 learned_addr learned my addr 172.21.15.185:0/1166196903 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:47.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d2434700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x20867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2083ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 msgr2=0x20867a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x20867a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 msgr2=0x2083ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2083ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 -- 172.21.15.185:0/1166196903 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d82720 con 0x1d64bb0 2020-11-16T22:15:47.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5d1c33700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa5c0005f70 tx=0x7fa5c006d000).ready entity=mon.1 client_cookie=4853f18cdce008e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c006d3b0 con 0x1d64bb0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5c0043450 con 0x1d64bb0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5c40145a0 con 0x1d64bb0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5c00059c0 con 0x1d64bb0 2020-11-16T22:15:47.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 10 client.5299 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x1d8de00 con 0x1d64bb0 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40145a0 con 0x1d64bb0 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa5c0005b20 con 0x1d64bb0 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 10 client.5299 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 10 client.5299 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 20 client.5299 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:47.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff0da0 0x2086c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:47.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 4 client.5299 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:15:47.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.559+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1e15f50 con 0x1ff0da0 2020-11-16T22:15:47.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff0da0 0x2086c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:47.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa59ffff700 10 client.5299 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:47.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d1432700 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff0da0 0x2086c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c549b587bf362a86 server_cookie=2494d8b7bded345d in_seq=0 out_seq=0 2020-11-16T22:15:47.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa59ffff700 1 -- 172.21.15.185:0/1166196903 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x1e15f50 con 0x1ff0da0 2020-11-16T22:15:47.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa59ffff700 10 client.5299 handle_command_reply: tid=0 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 client.5299 shutdown 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 msgr2=0x20835c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa5c0005f70 tx=0x7fa5c006d000).stop 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 shutdown_connections 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1ff0da0 0x2086c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d64bb0 0x20835c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1fa0e40 0x20867a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 --2- 172.21.15.185:0/1166196903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1dd27b0 0x2083ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa59ffff700 0 client.5299 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 >> 172.21.15.185:0/1166196903 conn(0x7fa598002320 msgr2=0x1fe58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 shutdown_connections 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 1 -- 172.21.15.185:0/1166196903 wait complete. 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5e2928740 20 client.5299 trim_cache size 0 max 16384 2020-11-16T22:15:47.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 msgr2=0x7fa5a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 0x7fa5a4057520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fa5cc068950 tx=0x7fa5c8015000).stop 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 msgr2=0x7fa5cc1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa5c000c710 tx=0x7fa5c0007ad0).stop 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 shutdown_connections 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5a4055060 0x7fa5a4057520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5cc0af250 0x7fa5cc146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5cc0b3080 0x7fa5cc1576e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 --2- 172.21.15.185:0/2294415051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5cc0b3920 0x7fa5cc111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 >> 172.21.15.185:0/2294415051 conn(0x7fa5cc060130 msgr2=0x7fa5cc067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 shutdown_connections 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:47.563+0000 7fa5d2c35700 1 -- 172.21.15.185:0/2294415051 wait complete. 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:47.565 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: "tag123": { 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "tag123", 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:47.566 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:47.567 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:47.567 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:15:52.568 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:52.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 Processor -- start 2020-11-16T22:15:52.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 -- start start 2020-11-16T22:15:52.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd3040b2160 0x7fd304067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd304098cc0 con 0x7fd304066240 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3040b3950 con 0x7fd3040b17e0 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd30c04f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30414c4f0 con 0x7fd3040b2160 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd309deb700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd3040b17e0 0x7fd30414bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51324/0 (socket says 172.21.15.185:51324) 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd309deb700 1 -- 172.21.15.185:0/1290203010 learned_addr learned my addr 172.21.15.185:0/1290203010 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:52.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd3095ea700 1 --2- 172.21.15.185:0/1290203010 >> v2:172.21.15.96:3300/0 conn(0x7fd3040b2160 0x7fd304067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032469784 0 0) 0x7fd3040b3950 con 0x7fd3040b17e0 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ec003240 con 0x7fd3040b17e0 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd3095ea700 1 -- 172.21.15.185:0/1290203010 >> v1:172.21.15.185:6789/0 conn(0x7fd3040b17e0 legacy=0x7fd30414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd3095ea700 1 -- 172.21.15.185:0/1290203010 >> v1:172.21.15.96:6789/0 conn(0x7fd304066240 legacy=0x7fd304065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254895447 0 0) 0x7fd304098cc0 con 0x7fd304066240 2020-11-16T22:15:52.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.711+0000 7fd3095ea700 1 -- 172.21.15.185:0/1290203010 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040b3bf0 con 0x7fd3040b2160 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd3095ea700 1 --2- 172.21.15.185:0/1290203010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b2160 0x7fd304067fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd30000b9a0 tx=0x7fd300006df0).ready entity=mon.0 client_cookie=307a0301f2cf999a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000097b0 con 0x7fd3040b2160 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300009910 con 0x7fd3040b2160 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd308de9700 1 -- 172.21.15.185:0/1290203010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300009c90 con 0x7fd3040b2160 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- 172.21.15.185:0/1290203010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b2160 msgr2=0x7fd304067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1290203010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b2160 0x7fd304067fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd30000b9a0 tx=0x7fd300006df0).stop 2020-11-16T22:15:52.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- 172.21.15.185:0/1290203010 shutdown_connections 2020-11-16T22:15:52.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1290203010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b2160 0x7fd304067fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- 172.21.15.185:0/1290203010 >> 172.21.15.185:0/1290203010 conn(0x7fd3040604a0 msgr2=0x7fd3040608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:52.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- 172.21.15.185:0/1290203010 shutdown_connections 2020-11-16T22:15:52.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- 172.21.15.185:0/1290203010 wait complete. 2020-11-16T22:15:52.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 Processor -- start 2020-11-16T22:15:52.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- start start 2020-11-16T22:15:52.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:52.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b17e0 0x7fd3040af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3040b2160 0x7fd3040ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304091490 con 0x7fd3040b17e0 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304090ab0 con 0x7fd304066240 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30c04f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd304090d50 con 0x7fd3040b2160 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd309deb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd309deb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57888/0 (socket says 172.21.15.185:57888) 2020-11-16T22:15:52.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd309deb700 1 -- 172.21.15.185:0/1195807017 learned_addr learned my addr 172.21.15.185:0/1195807017 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.715+0000 7fd30a5ec700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3040b2160 0x7fd3040ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd3095ea700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b17e0 0x7fd3040af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 -- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3040b2160 msgr2=0x7fd3040ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3040b2160 0x7fd3040ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 -- 172.21.15.185:0/1195807017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b17e0 msgr2=0x7fd3040af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b17e0 0x7fd3040af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 -- 172.21.15.185:0/1195807017 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304090ff0 con 0x7fd304066240 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd309deb700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fd2f80010a0 tx=0x7fd2f8007a40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80072e0 con 0x7fd304066240 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8007440 con 0x7fd304066240 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30408f3f0 con 0x7fd304066240 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8005820 con 0x7fd304066240 2020-11-16T22:15:52.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3040af900 con 0x7fd304066240 2020-11-16T22:15:52.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30405d4e0 con 0x7fd304066240 2020-11-16T22:15:52.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd2f8010070 con 0x7fd304066240 2020-11-16T22:15:52.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 0x7fd2dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:52.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2f806d360 con 0x7fd304066240 2020-11-16T22:15:52.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.719+0000 7fd3095ea700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 0x7fd2dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:52.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.723+0000 7fd3095ea700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 0x7fd2dc057520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd30000be60 tx=0x7fd300015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:52.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.727+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd2f80435c0 con 0x7fd304066240 2020-11-16T22:15:52.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.923+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd304062a50 con 0x7fd304066240 2020-11-16T22:15:52.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.923+0000 7fd2f5ffb700 1 -- 172.21.15.185:0/1195807017 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7fd2f8046be0 con 0x7fd304066240 2020-11-16T22:15:52.909 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:52.909 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 msgr2=0x7fd2dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 0x7fd2dc057520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd30000be60 tx=0x7fd300015000).stop 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 msgr2=0x7fd3040b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fd2f80010a0 tx=0x7fd2f8007a40).stop 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 shutdown_connections 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2dc055060 0x7fd2dc057520 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd304066240 0x7fd3040b0a30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3040b2160 0x7fd3040ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 --2- 172.21.15.185:0/1195807017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3040b17e0 0x7fd3040af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 >> 172.21.15.185:0/1195807017 conn(0x7fd3040604a0 msgr2=0x7fd30409b590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 shutdown_connections 2020-11-16T22:15:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:52.927+0000 7fd30c04f700 1 -- 172.21.15.185:0/1195807017 wait complete. 2020-11-16T22:15:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:15:52.924 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:15:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 Processor -- start 2020-11-16T22:15:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- start start 2020-11-16T22:15:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe6980b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 0x7fe698148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69814c660 con 0x7fe6980b3080 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6980ac120 con 0x7fe6980b3920 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980ac3c0 con 0x7fe6980b13d0 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe6980b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69dee2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe6980b3080 0x7fe6980ac860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51336/0 (socket says 172.21.15.185:51336) 2020-11-16T22:15:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69dee2700 1 -- 172.21.15.185:0/3901826075 learned_addr learned my addr 172.21.15.185:0/3901826075 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69e6e3700 1 --2- 172.21.15.185:0/3901826075 >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 0x7fe698148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 -- 172.21.15.185:0/3901826075 >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 msgr2=0x7fe698148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3901826075 >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 0x7fe698148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 -- 172.21.15.185:0/3901826075 >> v1:172.21.15.185:6789/0 conn(0x7fe6980b3080 legacy=0x7fe6980ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 -- 172.21.15.185:0/3901826075 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980b20e0 con 0x7fe6980b13d0 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69e6e3700 1 --2- 172.21.15.185:0/3901826075 >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 0x7fe698148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3901826075 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe6980b1bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe69400b9a0 tx=0x7fe694006df0).ready entity=mon.2 client_cookie=f79646faf7c6c5a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69cee0700 1 -- 172.21.15.185:0/3901826075 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6940097b0 con 0x7fe6980b13d0 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69cee0700 1 -- 172.21.15.185:0/3901826075 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe694009910 con 0x7fe6980b13d0 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69cee0700 1 -- 172.21.15.185:0/3901826075 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694009c90 con 0x7fe6980b13d0 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- 172.21.15.185:0/3901826075 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 msgr2=0x7fe6980b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 --2- 172.21.15.185:0/3901826075 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe6980b1bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fe69400b9a0 tx=0x7fe694006df0).stop 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- 172.21.15.185:0/3901826075 shutdown_connections 2020-11-16T22:15:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 --2- 172.21.15.185:0/3901826075 >> v2:172.21.15.96:3300/0 conn(0x7fe6980b3920 0x7fe698148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 --2- 172.21.15.185:0/3901826075 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe6980b1bf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- 172.21.15.185:0/3901826075 >> 172.21.15.185:0/3901826075 conn(0x7fe698060130 msgr2=0x7fe698060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- 172.21.15.185:0/3901826075 shutdown_connections 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.067+0000 7fe69eee4700 1 -- 172.21.15.185:0/3901826075 wait complete. 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 Processor -- start 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- start start 2020-11-16T22:15:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6980b3080 0x7fe69815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe6980b3920 0x7fe69815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980b00e0 con 0x7fe6980b3080 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980b6280 con 0x7fe6980b3920 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980b6520 con 0x7fe6980b13d0 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51346/0 (socket says 172.21.15.185:51346) 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 -- 172.21.15.185:0/987137530 learned_addr learned my addr 172.21.15.185:0/987137530 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69e6e3700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe6980b3920 0x7fe69815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6980b3080 0x7fe69815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 -- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe6980b3920 msgr2=0x7fe69815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe6980b3920 0x7fe69815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 -- 172.21.15.185:0/987137530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6980b3080 msgr2=0x7fe69815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6980b3080 0x7fe69815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 -- 172.21.15.185:0/987137530 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69815b1a0 con 0x7fe6980b13d0 2020-11-16T22:15:53.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69dee2700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe68800c710 tx=0x7fe688007c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe68dffb700 1 -- 172.21.15.185:0/987137530 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880072e0 con 0x7fe6980b13d0 2020-11-16T22:15:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe68dffb700 1 -- 172.21.15.185:0/987137530 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688007440 con 0x7fe6980b13d0 2020-11-16T22:15:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69810fcd0 con 0x7fe6980b13d0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe68dffb700 1 -- 172.21.15.185:0/987137530 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688005800 con 0x7fe6980b13d0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.071+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6980b6eb0 con 0x7fe6980b13d0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.075+0000 7fe68dffb700 1 -- 172.21.15.185:0/987137530 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe688010070 con 0x7fe6980b13d0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.075+0000 7fe68dffb700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.075+0000 7fe68dffb700 1 -- 172.21.15.185:0/987137530 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe68806d440 con 0x7fe6980b13d0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.075+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.075+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 0x7fe670057520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fe694006e30 tx=0x7fe694015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 Processor -- start 2020-11-16T22:15:53.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- start start 2020-11-16T22:15:53.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x1976e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x1977380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x18044b0 con 0x16ecbb0 2020-11-16T22:15:53.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x18b8930 con 0x1978da0 2020-11-16T22:15:53.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x17786f0 con 0x175a7b0 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51354/0 (socket says 172.21.15.185:51354) 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69e6e3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x1977380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 -- 172.21.15.185:0/1157348042 learned_addr learned my addr 172.21.15.185:0/1157348042 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x1976e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 -- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 msgr2=0x1977380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x1977380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 -- 172.21.15.185:0/1157348042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 msgr2=0x1976e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x1976e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 -- 172.21.15.185:0/1157348042 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17735a0 con 0x175a7b0 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe69dee2700 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe688047240 tx=0x7fe688010000).ready entity=mon.2 client_cookie=1408fc2863a1f369 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe676ffd700 1 -- 172.21.15.185:0/1157348042 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688045570 con 0x175a7b0 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe676ffd700 1 -- 172.21.15.185:0/1157348042 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688005c30 con 0x175a7b0 2020-11-16T22:15:53.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe676ffd700 1 -- 172.21.15.185:0/1157348042 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68806d2e0 con 0x175a7b0 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 msgr2=0x19788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe688047240 tx=0x7fe688010000).stop 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- 172.21.15.185:0/1157348042 shutdown_connections 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x1977380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19788b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/1157348042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x1976e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- 172.21.15.185:0/1157348042 >> 172.21.15.185:0/1157348042 conn(0x1928e40 msgr2=0x1975dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- 172.21.15.185:0/1157348042 shutdown_connections 2020-11-16T22:15:53.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- 172.21.15.185:0/1157348042 wait complete. 2020-11-16T22:15:53.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 -- start start 2020-11-16T22:15:53.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.079+0000 7fe6aebd7740 1 Processor -- start 2020-11-16T22:15:53.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x19a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x19a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a56e0 con 0x175a7b0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x18a5820 con 0x1978da0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x18a6a90 con 0x16ecbb0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 10 client.0 ll_register_callbacks cb 0x189bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69e6e3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x19a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51360/0 (socket says 172.21.15.185:51360) 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 -- 172.21.15.185:0/3508062534 learned_addr learned my addr 172.21.15.185:0/3508062534 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x19a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 -- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 msgr2=0x19a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x19a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 -- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 msgr2=0x19a6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x19a6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 -- 172.21.15.185:0/3508062534 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1896280 con 0x16ecbb0 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x19a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69e6e3700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x19a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69dee2700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe688043860 tx=0x7fe688014000).ready entity=mon.2 client_cookie=3718f3d9b1ffc6d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68806d2e0 con 0x16ecbb0 2020-11-16T22:15:53.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe688043150 con 0x16ecbb0 2020-11-16T22:15:53.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x19596c0 con 0x16ecbb0 2020-11-16T22:15:53.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6880462c0 con 0x16ecbb0 2020-11-16T22:15:53.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 10 client.5009 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:53.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x16cbfd0 con 0x16ecbb0 2020-11-16T22:15:53.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x19596c0 con 0x16ecbb0 2020-11-16T22:15:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fe68801b020 con 0x16ecbb0 2020-11-16T22:15:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 10 client.5009 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 10 client.5009 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 20 client.5009 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:53.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x19ae390 0x19bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 4 client.5009 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1796c50 con 0x19ae390 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x19ae390 0x19bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe66bfff700 10 client.5009 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.083+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x19ae390 0x19bfbc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=828601d94db0e527 server_cookie=bb4f4b1d385e9c5a in_seq=0 out_seq=0 2020-11-16T22:15:53.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe66bfff700 1 -- 172.21.15.185:0/3508062534 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1796c50 con 0x19ae390 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe66bfff700 10 client.5009 handle_command_reply: tid=0 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 client.5009 shutdown 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 msgr2=0x19745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe688043860 tx=0x7fe688014000).stop 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 shutdown_connections 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x19ae390 0x19bfbc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1978da0 0x19a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16ecbb0 0x19745c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/3508062534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x19a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 >> 172.21.15.185:0/3508062534 conn(0x1928e40 msgr2=0x1976e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe66bfff700 0 client.5009 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 shutdown_connections 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 1 -- 172.21.15.185:0/3508062534 wait complete. 2020-11-16T22:15:53.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.087+0000 7fe6aebd7740 20 client.5009 trim_cache size 0 max 16384 2020-11-16T22:15:53.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 Processor -- start 2020-11-16T22:15:53.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 -- start start 2020-11-16T22:15:53.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x19735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x187de40 con 0x16ecbb0 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1978620 con 0x1928e40 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19bc450 con 0x175a7b0 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57924/0 (socket says 172.21.15.185:57924) 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.107+0000 7fe69dee2700 1 -- 172.21.15.185:0/560436150 learned_addr learned my addr 172.21.15.185:0/560436150 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69e6e3700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 msgr2=0x19a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x19735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/560436150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 msgr2=0x19735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x19735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/560436150 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664008160 con 0x1928e40 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x19735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fe6880456b0 tx=0x7fe68805b000).ready entity=mon.1 client_cookie=25969c7eef9ee3a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6777fe700 1 -- 172.21.15.185:0/560436150 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68806d2e0 con 0x1928e40 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6777fe700 1 -- 172.21.15.185:0/560436150 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688043150 con 0x1928e40 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6777fe700 1 -- 172.21.15.185:0/560436150 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688005960 con 0x1928e40 2020-11-16T22:15:53.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 msgr2=0x1974410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fe6880456b0 tx=0x7fe68805b000).stop 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- 172.21.15.185:0/560436150 shutdown_connections 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1928e40 0x1974410 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x175a7b0 0x19a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/560436150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16ecbb0 0x19735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- 172.21.15.185:0/560436150 >> 172.21.15.185:0/560436150 conn(0x7fe664002320 msgr2=0x19aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- 172.21.15.185:0/560436150 shutdown_connections 2020-11-16T22:15:53.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- 172.21.15.185:0/560436150 wait complete. 2020-11-16T22:15:53.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- start start 2020-11-16T22:15:53.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 Processor -- start 2020-11-16T22:15:53.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x1a0b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1928e40 0x1a0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19b4f00 con 0x175a7b0 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19b51a0 con 0x16ecbb0 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19b5440 con 0x1928e40 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe6aebd7740 10 client.0 ll_register_callbacks cb 0x1972c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69e6e3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1928e40 0x1a0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57930/0 (socket says 172.21.15.185:57930) 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/2484342315 learned_addr learned my addr 172.21.15.185:0/2484342315 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x1a0b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1928e40 msgr2=0x1a0bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1928e40 0x1a0bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 msgr2=0x1a0b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x1a0b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 -- 172.21.15.185:0/2484342315 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x170a720 con 0x16ecbb0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe66bfff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.111+0000 7fe69dee2700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fe688043860 tx=0x7fe688069000).ready entity=mon.1 client_cookie=4aef43a618c8d372 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688022070 con 0x16ecbb0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe688005c30 con 0x16ecbb0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe6880464d0 con 0x16ecbb0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe688046690 con 0x16ecbb0 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 10 client.5323 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:53.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x1715e00 con 0x16ecbb0 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688005c30 con 0x16ecbb0 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fe68801b040 con 0x16ecbb0 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 10 client.5323 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 10 client.5323 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 20 client.5323 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1978da0 0x19b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:53.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 4 client.5323 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x179df50 con 0x1978da0 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1978da0 0x19b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 10 client.5323 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69d6e1700 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1978da0 0x19b2ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1c06870bf0fe8bde server_cookie=1f2bf20c7f90bebd in_seq=0 out_seq=0 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 1 -- 172.21.15.185:0/2484342315 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+207 (crc 0 0 0) 0x179df50 con 0x1978da0 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 10 client.5323 handle_command_reply: tid=0 2020-11-16T22:15:53.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 client.5323 shutdown 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 msgr2=0x196d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fe688043860 tx=0x7fe688069000).stop 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 shutdown_connections 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x1978da0 0x19b2ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16ecbb0 0x196d6c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1928e40 0x1a0bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 --2- 172.21.15.185:0/2484342315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x175a7b0 0x1a0b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe66bfff700 0 client.5323 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 >> 172.21.15.185:0/2484342315 conn(0x7fe664002320 msgr2=0x1979260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 shutdown_connections 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 1 -- 172.21.15.185:0/2484342315 wait complete. 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe6aebd7740 20 client.5323 trim_cache size 0 max 16384 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 msgr2=0x7fe670057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 0x7fe670057520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fe694006e30 tx=0x7fe694015000).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 msgr2=0x7fe69810f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe68800c710 tx=0x7fe688007c40).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 shutdown_connections 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe670055060 0x7fe670057520 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe6980b3920 0x7fe69815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6980b13d0 0x7fe69810f7e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 --2- 172.21.15.185:0/987137530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6980b3080 0x7fe69815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 >> 172.21.15.185:0/987137530 conn(0x7fe698060130 msgr2=0x7fe69814ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 shutdown_connections 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:53.115+0000 7fe69eee4700 1 -- 172.21.15.185:0/987137530 wait complete. 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:15:53.113 INFO:teuthology.orchestra.run.smithi185.stdout: "tag123": { 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "tag123", 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:15:53.114 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:53.119 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:15:58.120 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:15:58.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.259+0000 7f8b2e591700 1 Processor -- start 2020-11-16T22:15:58.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.259+0000 7f8b2e591700 1 -- start start 2020-11-16T22:15:58.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 0x7f8b281445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b280b4220 con 0x7f8b280b1a00 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b28146560 con 0x7f8b28064350 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280b44c0 con 0x7f8b2809cb40 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2cb2e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 0x7f8b281445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51382/0 (socket says 172.21.15.185:51382) 2020-11-16T22:15:58.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2cb2e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 0x7f8b281445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57942/0 (socket says 172.21.15.185:57942) 2020-11-16T22:15:58.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 -- 172.21.15.185:0/915627065 learned_addr learned my addr 172.21.15.185:0/915627065 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b277fe700 1 --1- 172.21.15.185:0/915627065 >> v1:172.21.15.185:6789/0 conn(0x7f8b280b1a00 0x7f8b28063c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51378/0 (socket says 172.21.15.185:51378) 2020-11-16T22:15:58.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 -- 172.21.15.185:0/915627065 >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 msgr2=0x7f8b281445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 --2- 172.21.15.185:0/915627065 >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 0x7f8b281445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 -- 172.21.15.185:0/915627065 >> v1:172.21.15.185:6789/0 conn(0x7f8b280b1a00 legacy=0x7f8b28063c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 -- 172.21.15.185:0/915627065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280b4760 con 0x7f8b2809cb40 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b27fff700 1 --2- 172.21.15.185:0/915627065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f8b18009d50 tx=0x7f8b18009290).ready entity=mon.2 client_cookie=6c26d2c82116fcfe server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b26ffd700 1 -- 172.21.15.185:0/915627065 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800bbc0 con 0x7f8b2809cb40 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b26ffd700 1 -- 172.21.15.185:0/915627065 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800bd20 con 0x7f8b2809cb40 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b26ffd700 1 -- 172.21.15.185:0/915627065 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18011610 con 0x7f8b2809cb40 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- 172.21.15.185:0/915627065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 msgr2=0x7f8b28146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 --2- 172.21.15.185:0/915627065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f8b18009d50 tx=0x7f8b18009290).stop 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- 172.21.15.185:0/915627065 shutdown_connections 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 --2- 172.21.15.185:0/915627065 >> v2:172.21.15.185:3300/0 conn(0x7f8b28064350 0x7f8b281445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 --2- 172.21.15.185:0/915627065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b28146070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- 172.21.15.185:0/915627065 >> 172.21.15.185:0/915627065 conn(0x7f8b28061240 msgr2=0x7f8b28061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- 172.21.15.185:0/915627065 shutdown_connections 2020-11-16T22:15:58.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.263+0000 7f8b2e591700 1 -- 172.21.15.185:0/915627065 wait complete. 2020-11-16T22:15:58.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 Processor -- start 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- start start 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 0x7f8b281515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28154c40 con 0x7f8b280b1a00 2020-11-16T22:15:58.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28154e20 con 0x7f8b28064350 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281550c0 con 0x7f8b2809cb40 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b27fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51390/0 (socket says 172.21.15.185:51390) 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 -- 172.21.15.185:0/2978764836 learned_addr learned my addr 172.21.15.185:0/2978764836 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b27fff700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57944/0 (socket says 172.21.15.185:57944) 2020-11-16T22:15:58.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2cb2e700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 0x7f8b281515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 -- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 msgr2=0x7f8b28143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 -- 172.21.15.185:0/2978764836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 msgr2=0x7f8b281515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 0x7f8b281515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 -- 172.21.15.185:0/2978764836 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28155360 con 0x7f8b2809cb40 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2cb2e700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 0x7f8b281515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:58.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b27fff700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b277fe700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f8b1c00c6e0 tx=0x7f8b1c007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0072e0 con 0x7f8b2809cb40 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c005530 con 0x7f8b2809cb40 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b28143820 con 0x7f8b2809cb40 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0058b0 con 0x7f8b2809cb40 2020-11-16T22:15:58.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.267+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28155a20 con 0x7f8b2809cb40 2020-11-16T22:15:58.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2805d4e0 con 0x7f8b2809cb40 2020-11-16T22:15:58.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8b1c01a060 con 0x7f8b2809cb40 2020-11-16T22:15:58.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b07fff700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 0x7f8b00057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b27fff700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 0x7f8b00057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b1c06d560 con 0x7f8b2809cb40 2020-11-16T22:15:58.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.271+0000 7f8b27fff700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 0x7f8b00057520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8b18009fd0 tx=0x7f8b18016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.279+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8b1c043840 con 0x7f8b2809cb40 2020-11-16T22:15:58.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b28142bf0 con 0x7f8b2809cb40 2020-11-16T22:15:58.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b07fff700 1 -- 172.21.15.185:0/2978764836 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7f8b1c018020 con 0x7f8b2809cb40 2020-11-16T22:15:58.460 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:15:58.461 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:15:58.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 msgr2=0x7f8b00057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 0x7f8b00057520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8b18009fd0 tx=0x7f8b18016000).stop 2020-11-16T22:15:58.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 msgr2=0x7f8b281406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f8b1c00c6e0 tx=0x7f8b1c007c10).stop 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 shutdown_connections 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8b00055060 0x7f8b00057520 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8b28064350 0x7f8b28143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b280b1a00 0x7f8b281515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 --2- 172.21.15.185:0/2978764836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8b2809cb40 0x7f8b281406e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 >> 172.21.15.185:0/2978764836 conn(0x7f8b28061240 msgr2=0x7f8b280b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 shutdown_connections 2020-11-16T22:15:58.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.475+0000 7f8b2e591700 1 -- 172.21.15.185:0/2978764836 wait complete. 2020-11-16T22:15:58.464 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:15:58.475 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:15:58.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.615+0000 7fb27f59e700 1 Processor -- start 2020-11-16T22:15:58.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.615+0000 7fb27f59e700 1 -- start start 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb2800af7f0 0x7fb280141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800b47b0 con 0x7fb280063fd0 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2800b4a50 con 0x7fb2800af7f0 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800b4cf0 con 0x7fb2800b3ec0 2020-11-16T22:15:58.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51398/0 (socket says 172.21.15.185:51398) 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb280063fd0 0x7fb2800b1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51390/0 (socket says 172.21.15.185:51390) 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 -- 172.21.15.185:0/1494883590 learned_addr learned my addr 172.21.15.185:0/1494883590 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/1494883590 >> v2:172.21.15.96:3300/0 conn(0x7fb2800af7f0 0x7fb280141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 -- 172.21.15.185:0/1494883590 >> v2:172.21.15.96:3300/0 conn(0x7fb2800af7f0 msgr2=0x7fb280141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/1494883590 >> v2:172.21.15.96:3300/0 conn(0x7fb2800af7f0 0x7fb280141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 -- 172.21.15.185:0/1494883590 >> v1:172.21.15.185:6789/0 conn(0x7fb280063fd0 legacy=0x7fb2800b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 -- 172.21.15.185:0/1494883590 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800b4430 con 0x7fb2800b3ec0 2020-11-16T22:15:58.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27d59a700 1 -- 172.21.15.185:0/1494883590 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133862711 0 0) 0x7fb2800b47b0 con 0x7fb280063fd0 2020-11-16T22:15:58.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/1494883590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb270006fd0 tx=0x7fb270006eb0).ready entity=mon.2 client_cookie=dd5b1d4d9c5b66a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27d59a700 1 -- 172.21.15.185:0/1494883590 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700098d0 con 0x7fb2800b3ec0 2020-11-16T22:15:58.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27d59a700 1 -- 172.21.15.185:0/1494883590 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb270009a30 con 0x7fb2800b3ec0 2020-11-16T22:15:58.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27d59a700 1 -- 172.21.15.185:0/1494883590 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270009db0 con 0x7fb2800b3ec0 2020-11-16T22:15:58.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- 172.21.15.185:0/1494883590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 msgr2=0x7fb280141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 --2- 172.21.15.185:0/1494883590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb270006fd0 tx=0x7fb270006eb0).stop 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- 172.21.15.185:0/1494883590 shutdown_connections 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 --2- 172.21.15.185:0/1494883590 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b3ec0 0x7fb280141740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 --2- 172.21.15.185:0/1494883590 >> v2:172.21.15.96:3300/0 conn(0x7fb2800af7f0 0x7fb280141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- 172.21.15.185:0/1494883590 >> 172.21.15.185:0/1494883590 conn(0x7fb280067940 msgr2=0x7fb280067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- 172.21.15.185:0/1494883590 shutdown_connections 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.619+0000 7fb27f59e700 1 -- 172.21.15.185:0/1494883590 wait complete. 2020-11-16T22:15:58.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 Processor -- start 2020-11-16T22:15:58.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- start start 2020-11-16T22:15:58.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280063fd0 0x7fb28012d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800af7f0 0x7fb2800b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280134cf0 con 0x7fb280063fd0 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28012ea70 con 0x7fb2800b3ec0 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800b37b0 con 0x7fb2800af7f0 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800af7f0 0x7fb2800b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57962/0 (socket says 172.21.15.185:57962) 2020-11-16T22:15:58.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3958131834 learned_addr learned my addr 172.21.15.185:0/3958131834 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280063fd0 0x7fb28012d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800af7f0 msgr2=0x7fb2800b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800af7f0 0x7fb2800b8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3958131834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280063fd0 msgr2=0x7fb28012d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280063fd0 0x7fb28012d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3958131834 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28012d650 con 0x7fb2800b3ec0 2020-11-16T22:15:58.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fb27000b1b0 tx=0x7fb270015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb2767fc700 1 -- 172.21.15.185:0/3958131834 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270006b20 con 0x7fb2800b3ec0 2020-11-16T22:15:58.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb2767fc700 1 -- 172.21.15.185:0/3958131834 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27001d5d0 con 0x7fb2800b3ec0 2020-11-16T22:15:58.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb280145050 con 0x7fb2800b3ec0 2020-11-16T22:15:58.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb2767fc700 1 -- 172.21.15.185:0/3958131834 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27001d970 con 0x7fb2800b3ec0 2020-11-16T22:15:58.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.623+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2800b8d40 con 0x7fb2800b3ec0 2020-11-16T22:15:58.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.627+0000 7fb2767fc700 1 -- 172.21.15.185:0/3958131834 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb270025030 con 0x7fb2800b3ec0 2020-11-16T22:15:58.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.627+0000 7fb2767fc700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 0x7fb250057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.627+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 0x7fb250057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.627+0000 7fb2767fc700 1 -- 172.21.15.185:0/3958131834 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb27006f700 con 0x7fb2800b3ec0 2020-11-16T22:15:58.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.627+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 0x7fb250057520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fb2800b5970 tx=0x7fb278007cd0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 Processor -- start 2020-11-16T22:15:58.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 -- start start 2020-11-16T22:15:58.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x13b4da0 0x13b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x12404b0 con 0x1128bb0 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x12f4930 con 0x11967b0 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x11b46f0 con 0x13b4da0 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57966/0 (socket says 172.21.15.185:57966) 2020-11-16T22:15:58.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 -- 172.21.15.185:0/3018570164 learned_addr learned my addr 172.21.15.185:0/3018570164 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x13b4da0 0x13b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 -- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x13b4da0 msgr2=0x13b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x13b4da0 0x13b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 -- 172.21.15.185:0/3018570164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 msgr2=0x13b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 -- 172.21.15.185:0/3018570164 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11af5a0 con 0x11967b0 2020-11-16T22:15:58.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb278001d80 tx=0x7fb27800a000).ready entity=mon.1 client_cookie=52dc09db0ca745e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb26f7fe700 1 -- 172.21.15.185:0/3018570164 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278016a00 con 0x11967b0 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb26f7fe700 1 -- 172.21.15.185:0/3018570164 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278016b60 con 0x11967b0 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb26f7fe700 1 -- 172.21.15.185:0/3018570164 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278016ee0 con 0x11967b0 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.631+0000 7fb29347b740 1 -- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 msgr2=0x13b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb278001d80 tx=0x7fb27800a000).stop 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- 172.21.15.185:0/3018570164 shutdown_connections 2020-11-16T22:15:58.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13b48b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x13b4da0 0x13b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- 172.21.15.185:0/3018570164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- 172.21.15.185:0/3018570164 >> 172.21.15.185:0/3018570164 conn(0x1364e40 msgr2=0x13b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- 172.21.15.185:0/3018570164 shutdown_connections 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- 172.21.15.185:0/3018570164 wait complete. 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- start start 2020-11-16T22:15:58.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 Processor -- start 2020-11-16T22:15:58.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x13b4da0 0x13fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x12e16e0 con 0x13b4da0 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x12e1820 con 0x11967b0 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x12e2a90 con 0x1128bb0 2020-11-16T22:15:58.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb29347b740 10 client.0 ll_register_callbacks cb 0x12d7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51416/0 (socket says 172.21.15.185:51416) 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 -- 172.21.15.185:0/1630068467 learned_addr learned my addr 172.21.15.185:0/1630068467 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x13b4da0 0x13fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 -- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 msgr2=0x13fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 -- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x13b4da0 msgr2=0x13fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x13b4da0 0x13fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 -- 172.21.15.185:0/1630068467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x12d2280 con 0x1128bb0 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb27e59c700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fb278008fd0 tx=0x7fb278010000).ready entity=mon.2 client_cookie=bb9ba902e27e317f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb26cff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780178f0 con 0x1128bb0 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.635+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb278017a50 con 0x1128bb0 2020-11-16T22:15:58.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13956c0 con 0x1128bb0 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb278017e70 con 0x1128bb0 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 10 client.5033 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x1107fd0 con 0x1128bb0 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13956c0 con 0x1128bb0 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb27802a070 con 0x1128bb0 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 10 client.5033 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:58.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 10 client.5033 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 20 client.5033 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13e9d60 0x13e3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 4 client.5033 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x11d2c50 con 0x13e9d60 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13e9d60 0x13e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 10 client.5033 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:58.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13e9d60 0x13e3110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ad10971f84238c51 server_cookie=5770a9b16d684219 in_seq=0 out_seq=0 2020-11-16T22:15:58.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 1 -- 172.21.15.185:0/1630068467 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x11d2c50 con 0x13e9d60 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 10 client.5033 handle_command_reply: tid=0 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 client.5033 shutdown 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 msgr2=0x13b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fb278008fd0 tx=0x7fb278010000).stop 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 shutdown_connections 2020-11-16T22:15:58.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x13b4da0 0x13fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1128bb0 0x13b0410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13e9d60 0x13e3110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb26cff9700 0 client.5033 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 --2- 172.21.15.185:0/1630068467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x11967b0 0x13fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 >> 172.21.15.185:0/1630068467 conn(0x1364e40 msgr2=0x13b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 shutdown_connections 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 1 -- 172.21.15.185:0/1630068467 wait complete. 2020-11-16T22:15:58.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.639+0000 7fb29347b740 20 client.5033 trim_cache size 0 max 16384 2020-11-16T22:15:58.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 Processor -- start 2020-11-16T22:15:58.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 -- start start 2020-11-16T22:15:58.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1364e40 0x13eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x13b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x12b9e40 con 0x1364e40 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1311680 con 0x1128bb0 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x13b72d0 con 0x11967b0 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x13b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57980/0 (socket says 172.21.15.185:57980) 2020-11-16T22:15:58.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x13b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51426/0 (socket says 172.21.15.185:51426) 2020-11-16T22:15:58.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27dd9b700 1 -- 172.21.15.185:0/4119435501 learned_addr learned my addr 172.21.15.185:0/4119435501 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.663+0000 7fb27e59c700 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1364e40 0x13eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 -- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 msgr2=0x13b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x13b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 -- 172.21.15.185:0/4119435501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1364e40 msgr2=0x13eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1364e40 0x13eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 -- 172.21.15.185:0/4119435501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13f9430 con 0x1128bb0 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb280145be0 tx=0x7fb268008040).ready entity=mon.1 client_cookie=21048775dca0eda1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb26ffff700 1 -- 172.21.15.185:0/4119435501 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268015700 con 0x1128bb0 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb26ffff700 1 -- 172.21.15.185:0/4119435501 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268015860 con 0x1128bb0 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb26ffff700 1 -- 172.21.15.185:0/4119435501 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268015c00 con 0x1128bb0 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 msgr2=0x13b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb280145be0 tx=0x7fb268008040).stop 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- 172.21.15.185:0/4119435501 shutdown_connections 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x13b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1364e40 0x13eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- 172.21.15.185:0/4119435501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1128bb0 0x13b7d20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- 172.21.15.185:0/4119435501 >> 172.21.15.185:0/4119435501 conn(0x7fb248002320 msgr2=0x13b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- 172.21.15.185:0/4119435501 shutdown_connections 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- 172.21.15.185:0/4119435501 wait complete. 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- start start 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 Processor -- start 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x144bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1415b80 con 0x1128bb0 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x144f660 con 0x1364e40 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x144fb30 con 0x11967b0 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb29347b740 10 client.0 ll_register_callbacks cb 0x13e7790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x144bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:57988/0 (socket says 172.21.15.185:57988) 2020-11-16T22:15:58.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3566290214 learned_addr learned my addr 172.21.15.185:0/3566290214 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 msgr2=0x144bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x144bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 msgr2=0x13f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 -- 172.21.15.185:0/3566290214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb248008160 con 0x1364e40 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb27ed9d700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb27000bfb0 tx=0x7fb27006f040).ready entity=mon.1 client_cookie=fb9d486437862ec8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.667+0000 7fb26cff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700456f0 con 0x1364e40 2020-11-16T22:15:58.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb270048580 con 0x1364e40 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb278016b60 con 0x1364e40 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb270048740 con 0x1364e40 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 10 client.5350 fetch_fsmap learned FSMap version 26 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x1146720 con 0x1364e40 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278016b60 con 0x1364e40 2020-11-16T22:15:58.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 26) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb270030040 con 0x1364e40 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 10 client.5350 fetch_fsmap finished waiting for FSMap version 26 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 10 client.5350 resolve_mds: resolved ID 'b' to GID 4932 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 20 client.5350 populate_metadata read hostname 'smithi185' 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13b4da0 0x1445790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 4 client.5350 mds_command: new command op to 4932 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 --> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x11d9f50 con 0x13b4da0 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13b4da0 0x1445790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 10 client.5350 ms_handle_connect on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27e59c700 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13b4da0 0x1445790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7655394250f15659 server_cookie=c876ce453de84d0f in_seq=0 out_seq=0 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 1 -- 172.21.15.185:0/3566290214 <== mds.0 v2:172.21.15.96:6834/2155443116 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x11d9f50 con 0x13b4da0 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 10 client.5350 handle_command_reply: tid=0 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 client.5350 shutdown 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 msgr2=0x14452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb27000bfb0 tx=0x7fb27006f040).stop 2020-11-16T22:15:58.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 shutdown_connections 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:6834/2155443116,v1:172.21.15.96:6835/2155443116] conn(0x13b4da0 0x1445790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb26cff9700 0 client.5350 ms_handle_reset on v2:172.21.15.96:6834/2155443116 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1364e40 0x14452a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1128bb0 0x13f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 --2- 172.21.15.185:0/3566290214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x11967b0 0x144bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 >> 172.21.15.185:0/3566290214 conn(0x7fb248002320 msgr2=0x13b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 shutdown_connections 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 1 -- 172.21.15.185:0/3566290214 wait complete. 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb29347b740 20 client.5350 trim_cache size 0 max 16384 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 msgr2=0x7fb250057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 0x7fb250057520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fb2800b5970 tx=0x7fb278007cd0).stop 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 msgr2=0x7fb280144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fb27000b1b0 tx=0x7fb270015000).stop 2020-11-16T22:15:58.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 shutdown_connections 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb250055060 0x7fb250057520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2800b3ec0 0x7fb280144670 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280063fd0 0x7fb28012d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 --2- 172.21.15.185:0/3958131834 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800af7f0 0x7fb2800b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.671+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 >> 172.21.15.185:0/3958131834 conn(0x7fb280067940 msgr2=0x7fb2800624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.675+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 shutdown_connections 2020-11-16T22:15:58.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.675+0000 7fb27f59e700 1 -- 172.21.15.185:0/3958131834 wait complete. 2020-11-16T22:15:58.668 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:15:58.668 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:15:58.668 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:15:58.668 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:15:58.675 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s 2020-11-16T22:15:58.675 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2020-11-16T22:15:58.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.719+0000 7febbb1ffc40 1 Processor -- start 2020-11-16T22:15:58.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.719+0000 7febbb1ffc40 1 -- start start 2020-11-16T22:15:58.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa12fa00 0x5557fa164210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 0x5557fa169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557fa0dc1f0 con 0x5557fa12fa00 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557fa0aabe0 con 0x5557fa12f130 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557fa1fa6f0 con 0x5557fa165180 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febac805700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 0x5557fa169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febac805700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58000/0 (socket says 172.21.15.185:58000) 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febac805700 1 -- 172.21.15.185:0/962783574 learned_addr learned my addr 172.21.15.185:0/962783574 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febac004700 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa12fa00 0x5557fa164210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 -- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 msgr2=0x5557fa1efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 -- 172.21.15.185:0/962783574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa12fa00 msgr2=0x5557fa164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa12fa00 0x5557fa164210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 -- 172.21.15.185:0/962783574 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557fa0d48a0 con 0x5557fa165180 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febad006700 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 0x5557fa169ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x5557fa1323d0 tx=0x7feb94013000).ready entity=mon.2 client_cookie=569d99abaf33d5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febab002700 1 -- 172.21.15.185:0/962783574 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94008cb0 con 0x5557fa165180 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febab002700 1 -- 172.21.15.185:0/962783574 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb94008e10 con 0x5557fa165180 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febab002700 1 -- 172.21.15.185:0/962783574 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94012830 con 0x5557fa165180 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 msgr2=0x5557fa169ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 0x5557fa169ee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x5557fa1323d0 tx=0x7feb94013000).stop 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- 172.21.15.185:0/962783574 shutdown_connections 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa165180 0x5557fa169ee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/962783574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa12fa00 0x5557fa164210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- 172.21.15.185:0/962783574 >> 172.21.15.185:0/962783574 conn(0x5557fa168370 msgr2=0x5557fa166280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- 172.21.15.185:0/962783574 shutdown_connections 2020-11-16T22:15:58.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- 172.21.15.185:0/962783574 wait complete. 2020-11-16T22:15:58.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 Processor -- start 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.723+0000 7febbb1ffc40 1 -- start start 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa12fa00 0x5557fa208240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa165180 0x5557fa208730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557fa119ac0 con 0x5557fa165180 2020-11-16T22:15:58.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac805700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac805700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58006/0 (socket says 172.21.15.185:58006) 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac805700 1 -- 172.21.15.185:0/4038880506 learned_addr learned my addr 172.21.15.185:0/4038880506 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557fa167990 con 0x5557fa12f130 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557fa1f6f20 con 0x5557fa12fa00 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa12fa00 0x5557fa208240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febad006700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa165180 0x5557fa208730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 -- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 msgr2=0x5557fa1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5557fa12f130 0x5557fa1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 -- 172.21.15.185:0/4038880506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa165180 msgr2=0x5557fa208730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557fa165180 0x5557fa208730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557fa0d0280 con 0x5557fa12fa00 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febac004700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa12fa00 0x5557fa208240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7feb9c004130 tx=0x7feb9c001890).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7feba97ff700 1 -- 172.21.15.185:0/4038880506 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00f410 con 0x5557fa12fa00 2020-11-16T22:15:58.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7feba97ff700 1 -- 172.21.15.185:0/4038880506 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c00f570 con 0x5557fa12fa00 2020-11-16T22:15:58.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557fa0ae620 con 0x5557fa12fa00 2020-11-16T22:15:58.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7feba97ff700 1 -- 172.21.15.185:0/4038880506 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00f8f0 con 0x5557fa12fa00 2020-11-16T22:15:58.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.727+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557fa1f09a0 con 0x5557fa12fa00 2020-11-16T22:15:58.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7feba97ff700 1 -- 172.21.15.185:0/4038880506 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feb9c01f070 con 0x5557fa12fa00 2020-11-16T22:15:58.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7feba97ff700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feb8c053ce0 0x7feb8c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7feba97ff700 1 -- 172.21.15.185:0/4038880506 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb9c06d0f0 con 0x5557fa12fa00 2020-11-16T22:15:58.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febac805700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feb8c053ce0 0x7feb8c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5557fa208c20 0x5557fa1efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 --> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] -- osd_op(unknown.0.0:1 5.2 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x5557fa1f24d0 con 0x5557fa208c20 2020-11-16T22:15:58.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febad006700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5557fa208c20 0x5557fa1efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febac805700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feb8c053ce0 0x7feb8c0561a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7feba400a8f0 tx=0x7feba4007040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.731+0000 7febad006700 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5557fa208c20 0x5557fa1efd20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febad006700 1 -- 172.21.15.185:0/4038880506 <== osd.0 v2:172.21.15.185:6808/15202 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7feb94022020 con 0x5557fa208c20 2020-11-16T22:15:58.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5557fa208c20 msgr2=0x5557fa1efd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5557fa208c20 0x5557fa1efd20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feb8c053ce0 msgr2=0x7feb8c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feb8c053ce0 0x7feb8c0561a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7feba400a8f0 tx=0x7feba4007040).stop 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa12fa00 msgr2=0x5557fa208240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 --2- 172.21.15.185:0/4038880506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5557fa12fa00 0x5557fa208240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7feb9c004130 tx=0x7feb9c001890).stop 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febad006700 1 -- 172.21.15.185:0/4038880506 reap_dead start 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 shutdown_connections 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 >> 172.21.15.185:0/4038880506 conn(0x5557fa168370 msgr2=0x5557fa1f8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 shutdown_connections 2020-11-16T22:15:58.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.735+0000 7febbb1ffc40 1 -- 172.21.15.185:0/4038880506 wait complete. 2020-11-16T22:15:58.722 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:58.723 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2020-11-16T22:15:58.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.811+0000 7f4818ca1c40 1 Processor -- start 2020-11-16T22:15:58.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.811+0000 7f4818ca1c40 1 -- start start 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 0x5586c2119810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c21167b0 0x5586c2117750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c2059be0 con 0x5586c21167b0 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c2114d50 con 0x5586c20fdb60 2020-11-16T22:15:58.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c20ffc70 con 0x5586c2100730 2020-11-16T22:15:58.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4809aa6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 0x5586c2119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4809aa6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 0x5586c2119810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51470/0 (socket says 172.21.15.185:51470) 2020-11-16T22:15:58.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4809aa6700 1 -- 172.21.15.185:0/3700943739 learned_addr learned my addr 172.21.15.185:0/3700943739 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480aaa8700 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c21167b0 0x5586c2117750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 -- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 msgr2=0x5586c2119810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 0x5586c2119810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 -- 172.21.15.185:0/3700943739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c21167b0 msgr2=0x5586c2117750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c21167b0 0x5586c2117750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 -- 172.21.15.185:0/3700943739 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c2082480 con 0x5586c20fdb60 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f480a2a7700 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21a4790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f47f4009c20 tx=0x7f47f400c040).ready entity=mon.1 client_cookie=39fceca5b03618c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4808aa4700 1 -- 172.21.15.185:0/3700943739 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f400f040 con 0x5586c20fdb60 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4808aa4700 1 -- 172.21.15.185:0/3700943739 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4008a50 con 0x5586c20fdb60 2020-11-16T22:15:58.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4808aa4700 1 -- 172.21.15.185:0/3700943739 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4008dd0 con 0x5586c20fdb60 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 msgr2=0x5586c21a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21a4790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f47f4009c20 tx=0x7f47f400c040).stop 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- 172.21.15.185:0/3700943739 shutdown_connections 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21a4790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c21167b0 0x5586c2117750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/3700943739 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c2100730 0x5586c2119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- 172.21.15.185:0/3700943739 >> 172.21.15.185:0/3700943739 conn(0x5586c2117370 msgr2=0x5586c210ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- 172.21.15.185:0/3700943739 shutdown_connections 2020-11-16T22:15:58.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.815+0000 7f4818ca1c40 1 -- 172.21.15.185:0/3700943739 wait complete. 2020-11-16T22:15:58.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 Processor -- start 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- start start 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2100730 0x5586c20de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 0x5586c21b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c21b84d0 con 0x5586c2100730 2020-11-16T22:15:58.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c20f1390 con 0x5586c20fdb60 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c210fdb0 con 0x5586c21167b0 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 0x5586c21b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480a2a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 0x5586c21b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51474/0 (socket says 172.21.15.185:51474) 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 -- 172.21.15.185:0/1764319568 learned_addr learned my addr 172.21.15.185:0/1764319568 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480a2a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58028/0 (socket says 172.21.15.185:58028) 2020-11-16T22:15:58.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4809aa6700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2100730 0x5586c20de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 -- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 msgr2=0x5586c21b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 -- 172.21.15.185:0/1764319568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2100730 msgr2=0x5586c20de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2100730 0x5586c20de3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 -- 172.21.15.185:0/1764319568 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c20838a0 con 0x5586c21167b0 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480a2a7700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586c20fdb60 0x5586c21b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:58.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f480aaa8700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 0x5586c21b8ba0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f4804008fd0 tx=0x7f4804014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4802ffd700 1 -- 172.21.15.185:0/1764319568 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804010ac0 con 0x5586c21167b0 2020-11-16T22:15:58.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4802ffd700 1 -- 172.21.15.185:0/1764319568 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4804021070 con 0x5586c21167b0 2020-11-16T22:15:58.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5586c207f280 con 0x5586c21167b0 2020-11-16T22:15:58.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4802ffd700 1 -- 172.21.15.185:0/1764319568 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804010c20 con 0x5586c21167b0 2020-11-16T22:15:58.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.819+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586c205d620 con 0x5586c21167b0 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4802ffd700 1 -- 172.21.15.185:0/1764319568 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f480402d070 con 0x5586c21167b0 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4802ffd700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f47ec053d10 0x7f47ec0561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4802ffd700 1 -- 172.21.15.185:0/1764319568 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f480406ecc0 con 0x5586c21167b0 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f480a2a7700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f47ec053d10 0x7f47ec0561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5586c21ac270 0x5586c21c1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 --> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] -- osd_op(unknown.0.0:1 5.2 5:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x5586c21b9090 con 0x5586c21ac270 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4809aa6700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5586c21ac270 0x5586c21c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f480a2a7700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f47ec053d10 0x7f47ec0561d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f47f4009b10 tx=0x7f47f4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.823+0000 7f4809aa6700 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5586c21ac270 0x5586c21c1880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4809aa6700 1 -- 172.21.15.185:0/1764319568 <== osd.0 v2:172.21.15.185:6808/15202 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f47fc012040 con 0x5586c21ac270 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5586c21ac270 msgr2=0x5586c21c1880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6808/15202,v1:172.21.15.185:6809/15202] conn(0x5586c21ac270 0x5586c21c1880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f47ec053d10 msgr2=0x7f47ec0561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f47ec053d10 0x7f47ec0561d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f47f4009b10 tx=0x7f47f4015000).stop 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 msgr2=0x5586c21b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 --2- 172.21.15.185:0/1764319568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586c21167b0 0x5586c21b8ba0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f4804008fd0 tx=0x7f4804014000).stop 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f480aaa8700 1 -- 172.21.15.185:0/1764319568 reap_dead start 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 shutdown_connections 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 >> 172.21.15.185:0/1764319568 conn(0x5586c2117370 msgr2=0x5586c21a44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 shutdown_connections 2020-11-16T22:15:58.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.827+0000 7f4818ca1c40 1 -- 172.21.15.185:0/1764319568 wait complete. 2020-11-16T22:15:58.815 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:58.816 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2020-11-16T22:15:58.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 Processor -- start 2020-11-16T22:15:58.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 -- start start 2020-11-16T22:15:58.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f2632a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f262bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f262d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f25a51f0 con 0x5617f26307b0 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2573be0 con 0x5617f261a730 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.903+0000 7fca2c522c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f2634350 con 0x5617f2617b60 2020-11-16T22:15:58.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1db28700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f2632a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f262bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1db28700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f2632a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51488/0 (socket says 172.21.15.185:51488) 2020-11-16T22:15:58.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1db28700 1 -- 172.21.15.185:0/3877564339 learned_addr learned my addr 172.21.15.185:0/3877564339 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1e329700 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f262d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 -- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 msgr2=0x5617f2632a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f2632a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 -- 172.21.15.185:0/3877564339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 msgr2=0x5617f262d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f262d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 -- 172.21.15.185:0/3877564339 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f259d8a0 con 0x5617f261a730 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca1d327700 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f262bea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x5617f25fb890 tx=0x7fca1000c040).ready entity=mon.1 client_cookie=85d23db6c45c8f10 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca0ffff700 1 -- 172.21.15.185:0/3877564339 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000f040 con 0x5617f261a730 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca0ffff700 1 -- 172.21.15.185:0/3877564339 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10008b10 con 0x5617f261a730 2020-11-16T22:15:58.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca0ffff700 1 -- 172.21.15.185:0/3877564339 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10008e90 con 0x5617f261a730 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 msgr2=0x5617f262bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f262bea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x5617f25fb890 tx=0x7fca1000c040).stop 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- 172.21.15.185:0/3877564339 shutdown_connections 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f262d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f2632a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 --2- 172.21.15.185:0/3877564339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f262bea0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- 172.21.15.185:0/3877564339 >> 172.21.15.185:0/3877564339 conn(0x5617f2631370 msgr2=0x5617f2631750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- 172.21.15.185:0/3877564339 shutdown_connections 2020-11-16T22:15:58.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- 172.21.15.185:0/3877564339 wait complete. 2020-11-16T22:15:58.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 Processor -- start 2020-11-16T22:15:58.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.907+0000 7fca2c522c40 1 -- start start 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f260a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f26bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f26bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f260bef0 con 0x5617f26307b0 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f260a820 con 0x5617f261a730 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f260c120 con 0x5617f2617b60 2020-11-16T22:15:58.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1d327700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f26bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f260a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1d327700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f26bcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58054/0 (socket says 172.21.15.185:58054) 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1d327700 1 -- 172.21.15.185:0/1105602144 learned_addr learned my addr 172.21.15.185:0/1105602144 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1e329700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f26bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 -- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 msgr2=0x5617f26bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5617f261a730 0x5617f26bcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 -- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 msgr2=0x5617f26bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5617f26307b0 0x5617f26bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 -- 172.21.15.185:0/1105602144 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f2599280 con 0x5617f2617b60 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca1db28700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f260a330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fca140080b0 tx=0x7fca14015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca0e7fc700 1 -- 172.21.15.185:0/1105602144 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14010a70 con 0x5617f2617b60 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5617f2577620 con 0x5617f2617b60 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca0e7fc700 1 -- 172.21.15.185:0/1105602144 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca14010bd0 con 0x5617f2617b60 2020-11-16T22:15:58.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca0e7fc700 1 -- 172.21.15.185:0/1105602144 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14010d60 con 0x5617f2617b60 2020-11-16T22:15:58.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.911+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f26bcfe0 con 0x5617f2617b60 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca0e7fc700 1 -- 172.21.15.185:0/1105602144 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fca1402c070 con 0x5617f2617b60 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca0e7fc700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fca00053ce0 0x7fca000561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca0e7fc700 1 -- 172.21.15.185:0/1105602144 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca1406ee30 con 0x5617f2617b60 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca1d327700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fca00053ce0 0x7fca000561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca2c522c40 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5617f2617030 0x5617f26d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- osd_op(unknown.0.0:1 5.5 5:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x5617f26ddfe0 con 0x5617f2617030 2020-11-16T22:15:58.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca1d327700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fca00053ce0 0x7fca000561a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x5617f2632420 tx=0x7fca10018000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca1e329700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5617f2617030 0x5617f26d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.915+0000 7fca1e329700 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5617f2617030 0x5617f26d1b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca1e329700 1 -- 172.21.15.185:0/1105602144 <== osd.7 v2:172.21.15.96:6811/15252 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7fca1800e3a0 con 0x5617f2617030 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5617f2617030 msgr2=0x5617f26d1b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5617f2617030 0x5617f26d1b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fca00053ce0 msgr2=0x7fca000561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fca00053ce0 0x7fca000561a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x5617f2632420 tx=0x7fca10018000).stop 2020-11-16T22:15:58.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 msgr2=0x5617f260a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 --2- 172.21.15.185:0/1105602144 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5617f2617b60 0x5617f260a330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fca140080b0 tx=0x7fca14015000).stop 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca1e329700 1 -- 172.21.15.185:0/1105602144 reap_dead start 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 shutdown_connections 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 >> 172.21.15.185:0/1105602144 conn(0x5617f2631370 msgr2=0x5617f2633590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 shutdown_connections 2020-11-16T22:15:58.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.919+0000 7fca2c522c40 1 -- 172.21.15.185:0/1105602144 wait complete. 2020-11-16T22:15:58.906 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:58.906 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2020-11-16T22:15:58.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:58.999+0000 7f3937161c40 1 Processor -- start 2020-11-16T22:15:58.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 -- start start 2020-11-16T22:15:58.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24e60 0x560f93bc31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93b29cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f93a9d190 con 0x560f93b28620 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f93a9e1f0 con 0x560f93b24a80 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3937161c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f93a6cbe0 con 0x560f93b24e60 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58066/0 (socket says 172.21.15.185:58066) 2020-11-16T22:15:58.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 -- 172.21.15.185:0/1424696936 learned_addr learned my addr 172.21.15.185:0/1424696936 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3927f66700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24e60 0x560f93bc31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928f68700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93b29cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 -- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24e60 msgr2=0x560f93bc31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24e60 0x560f93bc31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 -- 172.21.15.185:0/1424696936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 msgr2=0x560f93b29cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93b29cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 -- 172.21.15.185:0/1424696936 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f93a92280 con 0x560f93b24a80 2020-11-16T22:15:58.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928f68700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93b29cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:15:58.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3928767700 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f391000b730 tx=0x7f39100017e0).ready entity=mon.1 client_cookie=3bd6a8b024632ab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3926f64700 1 -- 172.21.15.185:0/1424696936 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000be00 con 0x560f93b24a80 2020-11-16T22:15:58.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3926f64700 1 -- 172.21.15.185:0/1424696936 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391000d5a0 con 0x560f93b24a80 2020-11-16T22:15:58.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.003+0000 7f3926f64700 1 -- 172.21.15.185:0/1424696936 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000d920 con 0x560f93b24a80 2020-11-16T22:15:58.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 msgr2=0x560f93b269a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f391000b730 tx=0x7f39100017e0).stop 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- 172.21.15.185:0/1424696936 shutdown_connections 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24a80 0x560f93b269a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93b29cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 --2- 172.21.15.185:0/1424696936 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24e60 0x560f93bc31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- 172.21.15.185:0/1424696936 >> 172.21.15.185:0/1424696936 conn(0x560f93b30be0 msgr2=0x560f93bbb250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:58.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- 172.21.15.185:0/1424696936 shutdown_connections 2020-11-16T22:15:58.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- 172.21.15.185:0/1424696936 wait complete. 2020-11-16T22:15:58.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 Processor -- start 2020-11-16T22:15:58.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.007+0000 7f3937161c40 1 -- start start 2020-11-16T22:15:58.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 0x560f93b2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24e60 0x560f93ba3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93bc2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f93b2d6d0 con 0x560f93b28620 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f93bc0a20 con 0x560f93b24e60 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f93bc0cc0 con 0x560f93b24a80 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 0x560f93b2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 0x560f93b2d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51516/0 (socket says 172.21.15.185:51516) 2020-11-16T22:15:58.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 -- 172.21.15.185:0/3818402289 learned_addr learned my addr 172.21.15.185:0/3818402289 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3927f66700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24e60 0x560f93ba3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928f68700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93bc2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 -- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24e60 msgr2=0x560f93ba3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x560f93b24e60 0x560f93ba3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 -- 172.21.15.185:0/3818402289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 msgr2=0x560f93bc2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f93b28620 0x560f93bc2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:58.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 -- 172.21.15.185:0/3818402289 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f93a70620 con 0x560f93b24a80 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3928767700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 0x560f93b2d1e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f391000b700 tx=0x560f93b2a940).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 -- 172.21.15.185:0/3818402289 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000eaa0 con 0x560f93b24a80 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 -- 172.21.15.185:0/3818402289 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391000ec00 con 0x560f93b24a80 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f93bc0f60 con 0x560f93b24a80 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 -- 172.21.15.185:0/3818402289 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391001b440 con 0x560f93b24a80 2020-11-16T22:15:58.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f93ba3940 con 0x560f93b24a80 2020-11-16T22:15:59.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 -- 172.21.15.185:0/3818402289 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f391001b7f0 con 0x560f93b24a80 2020-11-16T22:15:59.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f390c053ce0 0x7f390c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3925761700 1 -- 172.21.15.185:0/3818402289 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f391006f340 con 0x560f93b24a80 2020-11-16T22:15:59.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.011+0000 7f3927f66700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f390c053ce0 0x7f390c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x560f93bd49c0 0x560f93bd6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 5.1 5:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x560f93bc1440 con 0x560f93bd49c0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3927f66700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f390c053ce0 0x7f390c0561a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x560f93bbe5e0 tx=0x7f3918002840).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3928f68700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x560f93bd49c0 0x560f93bd6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3928f68700 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x560f93bd49c0 0x560f93bd6db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3928f68700 1 -- 172.21.15.185:0/3818402289 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f392000a500 con 0x560f93bd49c0 2020-11-16T22:15:59.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x560f93bd49c0 msgr2=0x560f93bd6db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x560f93bd49c0 0x560f93bd6db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f390c053ce0 msgr2=0x7f390c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f390c053ce0 0x7f390c0561a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x560f93bbe5e0 tx=0x7f3918002840).stop 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 msgr2=0x560f93b2d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 --2- 172.21.15.185:0/3818402289 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x560f93b24a80 0x560f93b2d1e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f391000b700 tx=0x560f93b2a940).stop 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3928f68700 1 -- 172.21.15.185:0/3818402289 reap_dead start 2020-11-16T22:15:59.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 shutdown_connections 2020-11-16T22:15:59.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 >> 172.21.15.185:0/3818402289 conn(0x560f93b30be0 msgr2=0x560f93b269a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 shutdown_connections 2020-11-16T22:15:59.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.015+0000 7f3937161c40 1 -- 172.21.15.185:0/3818402289 wait complete. 2020-11-16T22:15:59.005 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:59.005 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2020-11-16T22:15:59.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 Processor -- start 2020-11-16T22:15:59.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 -- start start 2020-11-16T22:15:59.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 0x5642862f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5642862f9c20 0x56428638b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642862681f0 con 0x5642862f2620 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x564286236be0 con 0x5642862ba1e0 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7fad09f70c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x56428637f220 con 0x5642862f9c20 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfbd77700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5642862f9c20 0x56428638b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58086/0 (socket says 172.21.15.185:58086) 2020-11-16T22:15:59.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 -- 172.21.15.185:0/600514455 learned_addr learned my addr 172.21.15.185:0/600514455 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfad75700 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 0x5642862f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 -- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5642862f9c20 msgr2=0x56428638b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5642862f9c20 0x56428638b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 -- 172.21.15.185:0/600514455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 msgr2=0x5642862f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 0x5642862f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 -- 172.21.15.185:0/600514455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642862608a0 con 0x5642862ba1e0 2020-11-16T22:15:59.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfad75700 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 0x5642862f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:59.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.095+0000 7facfb576700 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x5642862a8300 tx=0x7face4015000).ready entity=mon.1 client_cookie=bf7db7eeaf7949b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facf9d73700 1 -- 172.21.15.185:0/600514455 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4000b80 con 0x5642862ba1e0 2020-11-16T22:15:59.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facf9d73700 1 -- 172.21.15.185:0/600514455 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face400a5e0 con 0x5642862ba1e0 2020-11-16T22:15:59.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facf9d73700 1 -- 172.21.15.185:0/600514455 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40129b0 con 0x5642862ba1e0 2020-11-16T22:15:59.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 msgr2=0x5642862fa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x5642862a8300 tx=0x7face4015000).stop 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- 172.21.15.185:0/600514455 shutdown_connections 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862ba1e0 0x5642862fa1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5642862f9c20 0x56428638b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- 172.21.15.185:0/600514455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862f2620 0x5642862f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- 172.21.15.185:0/600514455 >> 172.21.15.185:0/600514455 conn(0x5642862fabe0 msgr2=0x564286381cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- 172.21.15.185:0/600514455 shutdown_connections 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- 172.21.15.185:0/600514455 wait complete. 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 Processor -- start 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- start start 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862ba1e0 0x5642863864a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862f2620 0x56428638c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 0x56428636af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x564286399390 con 0x5642862ba1e0 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x564286383860 con 0x5642862f2620 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7fad09f70c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642862f7e20 con 0x56428638c8f0 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facfbd77700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 0x56428636af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facfad75700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862f2620 0x56428638c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facfbd77700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 0x56428636af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51540/0 (socket says 172.21.15.185:51540) 2020-11-16T22:15:59.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facfbd77700 1 -- 172.21.15.185:0/3644272366 learned_addr learned my addr 172.21.15.185:0/3644272366 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:59.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.099+0000 7facfb576700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862ba1e0 0x5642863864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 -- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862f2620 msgr2=0x56428638c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5642862f2620 0x56428638c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 -- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862ba1e0 msgr2=0x5642863864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642862ba1e0 0x5642863864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 -- 172.21.15.185:0/3644272366 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56428625c280 con 0x56428638c8f0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facfbd77700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 0x56428636af40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7facf4010d10 tx=0x7facf4005d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facf3fff700 1 -- 172.21.15.185:0/3644272366 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4002510 con 0x56428638c8f0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facf3fff700 1 -- 172.21.15.185:0/3644272366 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4010ed0 con 0x56428638c8f0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56428623a620 con 0x56428638c8f0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facf3fff700 1 -- 172.21.15.185:0/3644272366 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400ee20 con 0x56428638c8f0 2020-11-16T22:15:59.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56428638ce30 con 0x56428638c8f0 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.103+0000 7facf3fff700 1 -- 172.21.15.185:0/3644272366 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7facf4023070 con 0x56428638c8f0 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facf3fff700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facdc053ce0 0x7facdc0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfb576700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facdc053ce0 0x7facdc0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facf3fff700 1 -- 172.21.15.185:0/3644272366 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7facf406d890 con 0x56428638c8f0 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5642863825c0 0x5642863a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfb576700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facdc053ce0 0x7facdc0561a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x5642862a8300 tx=0x7face401c000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- osd_op(unknown.0.0:1 5.5 5:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x564286398e40 con 0x5642863825c0 2020-11-16T22:15:59.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfad75700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5642863825c0 0x5642863a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfad75700 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5642863825c0 0x5642863a1cd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfad75700 1 -- 172.21.15.185:0/3644272366 <== osd.7 v2:172.21.15.96:6811/15252 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7facec00c6f0 con 0x5642863825c0 2020-11-16T22:15:59.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5642863825c0 msgr2=0x5642863a1cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x5642863825c0 0x5642863a1cd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facdc053ce0 msgr2=0x7facdc0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facdc053ce0 0x7facdc0561a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x5642862a8300 tx=0x7face401c000).stop 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 msgr2=0x56428636af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 --2- 172.21.15.185:0/3644272366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x56428638c8f0 0x56428636af40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7facf4010d10 tx=0x7facf4005d00).stop 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7facfbd77700 1 -- 172.21.15.185:0/3644272366 reap_dead start 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 shutdown_connections 2020-11-16T22:15:59.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.107+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 >> 172.21.15.185:0/3644272366 conn(0x5642862fabe0 msgr2=0x564286381cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.111+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 shutdown_connections 2020-11-16T22:15:59.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.111+0000 7fad09f70c40 1 -- 172.21.15.185:0/3644272366 wait complete. 2020-11-16T22:15:59.298 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:59.298 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2020-11-16T22:15:59.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.387+0000 7f23be12ac40 1 Processor -- start 2020-11-16T22:15:59.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.387+0000 7f23be12ac40 1 -- start start 2020-11-16T22:15:59.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.387+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726ae89c0 0x561726ae2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617269cd190 con 0x561726ae89c0 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617269ce1f0 con 0x561726a5a950 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x56172699cbe0 con 0x561726a41fa0 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aef2f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51552/0 (socket says 172.21.15.185:51552) 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aef2f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58108/0 (socket says 172.21.15.185:58108) 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aef2f700 1 -- 172.21.15.185:0/1581565815 learned_addr learned my addr 172.21.15.185:0/1581565815 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aff31700 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726ae89c0 0x561726ae2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 -- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 msgr2=0x561726ae2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 -- 172.21.15.185:0/1581565815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726ae89c0 msgr2=0x561726ae2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:15:59.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726ae89c0 0x561726ae2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 -- 172.21.15.185:0/1581565815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617269c2280 con 0x561726a41fa0 2020-11-16T22:15:59.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aef2f700 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:59.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23af730700 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f23a0001aa0 tx=0x7f23a00069b0).ready entity=mon.2 client_cookie=3e1d9cc71bca2b2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23adf2d700 1 -- 172.21.15.185:0/1581565815 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00062e0 con 0x561726a41fa0 2020-11-16T22:15:59.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23adf2d700 1 -- 172.21.15.185:0/1581565815 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a0004500 con 0x561726a41fa0 2020-11-16T22:15:59.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23adf2d700 1 -- 172.21.15.185:0/1581565815 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00048a0 con 0x561726a41fa0 2020-11-16T22:15:59.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 msgr2=0x561726a1f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f23a0001aa0 tx=0x7f23a00069b0).stop 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- 172.21.15.185:0/1581565815 shutdown_connections 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726ae89c0 0x561726ae2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726a41fa0 0x561726a1f550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- 172.21.15.185:0/1581565815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- 172.21.15.185:0/1581565815 >> 172.21.15.185:0/1581565815 conn(0x561726a5bfb0 msgr2=0x561726ae8260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- 172.21.15.185:0/1581565815 shutdown_connections 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- 172.21.15.185:0/1581565815 wait complete. 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 Processor -- start 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- start start 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726a41fa0 0x561726aebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 0x561726ae0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x561726ae4050 con 0x561726a41fa0 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x561726afb320 con 0x561726a5a950 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23be12ac40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x561726ae1de0 con 0x561726ae89c0 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aff31700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 0x561726ae0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aef2f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aff31700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 0x561726ae0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51562/0 (socket says 172.21.15.185:51562) 2020-11-16T22:15:59.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.391+0000 7f23aff31700 1 -- 172.21.15.185:0/4088952265 learned_addr learned my addr 172.21.15.185:0/4088952265 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23af730700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726a41fa0 0x561726aebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 -- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 msgr2=0x561726ae08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 -- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726a41fa0 msgr2=0x561726aebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561726a41fa0 0x561726aebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 -- 172.21.15.185:0/4088952265 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617269a0620 con 0x561726ae89c0 2020-11-16T22:15:59.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aef2f700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x561726a5a950 0x561726ae08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23aff31700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 0x561726ae0da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f23a800c010 tx=0x7f23a8012000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23a7fff700 1 -- 172.21.15.185:0/4088952265 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8014320 con 0x561726ae89c0 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23a7fff700 1 -- 172.21.15.185:0/4088952265 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a801b040 con 0x561726ae89c0 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561726ae3910 con 0x561726ae89c0 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23a7fff700 1 -- 172.21.15.185:0/4088952265 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8012760 con 0x561726ae89c0 2020-11-16T22:15:59.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.395+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561726ae1b40 con 0x561726ae89c0 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23a7fff700 1 -- 172.21.15.185:0/4088952265 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f23a8012b10 con 0x561726ae89c0 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23a7fff700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2394053ce0 0x7f23940561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23a7fff700 1 -- 172.21.15.185:0/4088952265 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f23a806e350 con 0x561726ae89c0 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23af730700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2394053ce0 0x7f23940561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x561726b025b0 0x561726b049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23af730700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2394053ce0 0x7f23940561a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f23a0002210 tx=0x7f23a0012000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- osd_op(unknown.0.0:1 5.5 5:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected e32) v8 -- 0x561726b04e90 con 0x561726b025b0 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23aef2f700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x561726b025b0 0x561726b049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23aef2f700 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x561726b025b0 0x561726b049a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23aef2f700 1 -- 172.21.15.185:0/4088952265 <== osd.7 v2:172.21.15.96:6811/15252 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+10 (crc 0 0 0) 0x7f239800ec50 con 0x561726b025b0 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x561726b025b0 msgr2=0x561726b049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x561726b025b0 0x561726b049a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2394053ce0 msgr2=0x7f23940561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2394053ce0 0x7f23940561a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f23a0002210 tx=0x7f23a0012000).stop 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 msgr2=0x561726ae0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 --2- 172.21.15.185:0/4088952265 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x561726ae89c0 0x561726ae0da0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f23a800c010 tx=0x7f23a8012000).stop 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23aff31700 1 -- 172.21.15.185:0/4088952265 reap_dead start 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 shutdown_connections 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 >> 172.21.15.185:0/4088952265 conn(0x561726a5bfb0 msgr2=0x561726a1f550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 shutdown_connections 2020-11-16T22:15:59.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.399+0000 7f23be12ac40 1 -- 172.21.15.185:0/4088952265 wait complete. 2020-11-16T22:15:59.390 INFO:teuthology.orchestra.run.smithi185.stdout:tag123 2020-11-16T22:15:59.391 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:15:59.391 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:15:59.431 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:15:59.448 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:15:59.464 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:15:59.464 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:15:59.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:15:59.465 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:15:59.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:15:59.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:15:59.544 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:15:59.564 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:15:59.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 Processor -- start 2020-11-16T22:15:59.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 -- start start 2020-11-16T22:15:59.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4cf80ba800 0x7f4cf80b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80b7290 con 0x7f4cf80b5d70 2020-11-16T22:15:59.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80b4960 con 0x7f4cf8143470 2020-11-16T22:15:59.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfef38700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf80b4bf0 con 0x7f4cf80ba800 2020-11-16T22:15:59.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfe737700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4cf8143470 0x7f4cf81438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51566/0 (socket says 172.21.15.185:51566) 2020-11-16T22:15:59.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfe737700 1 -- 172.21.15.185:0/3985397606 learned_addr learned my addr 172.21.15.185:0/3985397606 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:15:59.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfd735700 1 --2- 172.21.15.185:0/3985397606 >> v2:172.21.15.96:3300/0 conn(0x7f4cf80ba800 0x7f4cf80b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453778181 0 0) 0x7f4cf80b4960 con 0x7f4cf8143470 2020-11-16T22:15:59.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce0003240 con 0x7f4cf8143470 2020-11-16T22:15:59.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651755663 0 0) 0x7f4cf80b7290 con 0x7f4cf80b5d70 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf80b4960 con 0x7f4cf80b5d70 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfd735700 1 -- 172.21.15.185:0/3985397606 >> v1:172.21.15.185:6789/0 conn(0x7f4cf8143470 legacy=0x7f4cf81438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfd735700 1 -- 172.21.15.185:0/3985397606 >> v1:172.21.15.96:6789/0 conn(0x7f4cf80b5d70 legacy=0x7f4cf80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfd735700 1 -- 172.21.15.185:0/3985397606 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80b74c0 con 0x7f4cf80ba800 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfd735700 1 --2- 172.21.15.185:0/3985397606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80ba800 0x7f4cf80b4470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4ce800b9a0 tx=0x7f4ce8006df0).ready entity=mon.0 client_cookie=ba410f8fa1c7fbde server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8009680 con 0x7f4cf80ba800 2020-11-16T22:15:59.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80097e0 con 0x7f4cf80ba800 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.719+0000 7f4cfcf34700 1 -- 172.21.15.185:0/3985397606 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8009b60 con 0x7f4cf80ba800 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/3985397606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80ba800 msgr2=0x7f4cf80b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 --2- 172.21.15.185:0/3985397606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80ba800 0x7f4cf80b4470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4ce800b9a0 tx=0x7f4ce8006df0).stop 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/3985397606 shutdown_connections 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 --2- 172.21.15.185:0/3985397606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80ba800 0x7f4cf80b4470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/3985397606 >> 172.21.15.185:0/3985397606 conn(0x7f4cf805ba80 msgr2=0x7f4cf805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/3985397606 shutdown_connections 2020-11-16T22:15:59.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/3985397606 wait complete. 2020-11-16T22:15:59.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 Processor -- start 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- start start 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 0x7f4cf8141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80afe20 0x7f4cf813e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80af310 con 0x7f4cf80afe20 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8153890 con 0x7f4cf8143470 2020-11-16T22:15:59.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80af500 con 0x7f4cf80b5d70 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58130/0 (socket says 172.21.15.185:58130) 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 -- 172.21.15.185:0/1242253961 learned_addr learned my addr 172.21.15.185:0/1242253961 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfdf36700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 0x7f4cf8141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfe737700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80afe20 0x7f4cf813e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 -- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 msgr2=0x7f4cf8141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 0x7f4cf8141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 -- 172.21.15.185:0/1242253961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80afe20 msgr2=0x7f4cf813e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80afe20 0x7f4cf813e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 -- 172.21.15.185:0/1242253961 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80af7a0 con 0x7f4cf8143470 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfdf36700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 0x7f4cf8141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfd735700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f4ce8010000 tx=0x7f4ce8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8011b70 con 0x7f4cf8143470 2020-11-16T22:15:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf8141770 con 0x7f4cf8143470 2020-11-16T22:15:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8011cd0 con 0x7f4cf8143470 2020-11-16T22:15:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801e670 con 0x7f4cf8143470 2020-11-16T22:15:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.723+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf80b0d50 con 0x7f4cf8143470 2020-11-16T22:15:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf80af930 con 0x7f4cf8143470 2020-11-16T22:15:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4ce8025030 con 0x7f4cf8143470 2020-11-16T22:15:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4ce5ffb700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 0x7f4cd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:15:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4ce806f130 con 0x7f4cf8143470 2020-11-16T22:15:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4cfdf36700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 0x7f4cd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:15:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.727+0000 7f4cfdf36700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 0x7f4cd0057520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4cf0007fd0 tx=0x7f4cf0007d10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:15:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.735+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4ce8045440 con 0x7f4cf8143470 2020-11-16T22:15:59.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.887+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4cf805d200 con 0x7f4cf8143470 2020-11-16T22:15:59.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4ce5ffb700 1 -- 172.21.15.185:0/1242253961 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ce80230a0 con 0x7f4cf8143470 2020-11-16T22:15:59.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 msgr2=0x7f4cd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 0x7f4cd0057520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4cf0007fd0 tx=0x7f4cf0007d10).stop 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 msgr2=0x7f4cf813e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f4ce8010000 tx=0x7f4ce8015000).stop 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 shutdown_connections 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4cd0055060 0x7f4cd0057520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cf80afe20 0x7f4cf813e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4cf80b5d70 0x7f4cf8141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 --2- 172.21.15.185:0/1242253961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4cf8143470 0x7f4cf813e180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:15:59.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 >> 172.21.15.185:0/1242253961 conn(0x7f4cf805ba80 msgr2=0x7f4cf8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:15:59.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 shutdown_connections 2020-11-16T22:15:59.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:15:59.899+0000 7f4cfef38700 1 -- 172.21.15.185:0/1242253961 wait complete. 2020-11-16T22:15:59.901 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2020-11-16T22:16:00.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 Processor -- start 2020-11-16T22:16:00.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 -- start start 2020-11-16T22:16:00.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fceac0b7470 0x7fceac0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 0x7fceac13e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac066ad0 con 0x7fceac13e720 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac140d10 con 0x7fceac0b7470 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceb0efd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac144d90 con 0x7fceac0b2920 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceabfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fceac13e720 0x7fceac141100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51578/0 (socket says 172.21.15.185:51578) 2020-11-16T22:16:00.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceab7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fceac0b7470 0x7fceac0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceabfff700 1 -- 172.21.15.185:0/3225112483 learned_addr learned my addr 172.21.15.185:0/3225112483 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:00.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 0x7fceac13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceab7fe700 1 -- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 msgr2=0x7fceac13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:00.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceab7fe700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 0x7fceac13e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceab7fe700 1 -- 172.21.15.185:0/3225112483 >> v1:172.21.15.185:6789/0 conn(0x7fceac13e720 legacy=0x7fceac141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceaa7fc700 1 -- 172.21.15.185:0/3225112483 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860525699 0 0) 0x7fceac066ad0 con 0x7fceac13e720 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceab7fe700 1 -- 172.21.15.185:0/3225112483 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac144ff0 con 0x7fceac0b7470 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.039+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 0x7fceac13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceab7fe700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b7470 0x7fceac0b7850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fce9c00b9a0 tx=0x7fce9c006df0).ready entity=mon.1 client_cookie=25ff2c3c8f67b19 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceaa7fc700 1 -- 172.21.15.185:0/3225112483 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c009840 con 0x7fceac0b7470 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceaa7fc700 1 -- 172.21.15.185:0/3225112483 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0099a0 con 0x7fceac0b7470 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceaa7fc700 1 -- 172.21.15.185:0/3225112483 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c009d20 con 0x7fceac0b7470 2020-11-16T22:16:00.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b7470 msgr2=0x7fceac0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b7470 0x7fceac0b7850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fce9c00b9a0 tx=0x7fce9c006df0).stop 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- 172.21.15.185:0/3225112483 shutdown_connections 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b7470 0x7fceac0b7850 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3225112483 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac0b2920 0x7fceac13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- 172.21.15.185:0/3225112483 >> 172.21.15.185:0/3225112483 conn(0x7fceac0637b0 msgr2=0x7fceac0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- 172.21.15.185:0/3225112483 shutdown_connections 2020-11-16T22:16:00.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- 172.21.15.185:0/3225112483 wait complete. 2020-11-16T22:16:00.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 Processor -- start 2020-11-16T22:16:00.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.043+0000 7fceb0efd700 1 -- start start 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 0x7fceac0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac13e720 0x7fceac0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0af900 con 0x7fceac0b7470 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac143790 con 0x7fceac0b2920 2020-11-16T22:16:00.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac144390 con 0x7fceac13e720 2020-11-16T22:16:00.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceabfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac13e720 0x7fceac0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58144/0 (socket says 172.21.15.185:58144) 2020-11-16T22:16:00.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 -- 172.21.15.185:0/3886982630 learned_addr learned my addr 172.21.15.185:0/3886982630 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:00.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 0x7fceac0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 -- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac13e720 msgr2=0x7fceac0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac13e720 0x7fceac0ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 -- 172.21.15.185:0/3886982630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 msgr2=0x7fceac0ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 0x7fceac0ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 -- 172.21.15.185:0/3886982630 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac144630 con 0x7fceac0b2920 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 0x7fceac0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceab7fe700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fce9c006e30 tx=0x7fce9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:00.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c020070 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c011e50 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac0af640 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c019660 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac142af0 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac0ae2b0 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce9c00f070 con 0x7fceac0b2920 2020-11-16T22:16:00.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.047+0000 7fce8b7fe700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 0x7fce84057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:00.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.051+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce9c070450 con 0x7fceac0b2920 2020-11-16T22:16:00.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.051+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 0x7fce84057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:00.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.051+0000 7fceaaffd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 0x7fce84057520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fcea0006fd0 tx=0x7fcea0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:00.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.055+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fce9c0466b0 con 0x7fceac0b2920 2020-11-16T22:16:00.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.263+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) v1 -- 0x7fceac1430f0 con 0x7fceac0b2920 2020-11-16T22:16:00.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fce8b7fe700 1 -- 172.21.15.185:0/3886982630 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v114) v1 ==== 136+0+0 (secure 0 0 0) 0x7fce9c049cd0 con 0x7fceac0b2920 2020-11-16T22:16:00.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 msgr2=0x7fce84057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 0x7fce84057520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fcea0006fd0 tx=0x7fcea0008040).stop 2020-11-16T22:16:00.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 msgr2=0x7fceac0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fce9c006e30 tx=0x7fce9c015000).stop 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 shutdown_connections 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce84055060 0x7fce84057520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fceac0b2920 0x7fceac0af150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fceac13e720 0x7fceac0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 --2- 172.21.15.185:0/3886982630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceac0b7470 0x7fceac0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:00.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 >> 172.21.15.185:0/3886982630 conn(0x7fceac0637b0 msgr2=0x7fceac13c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:00.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 shutdown_connections 2020-11-16T22:16:00.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:00.979+0000 7fceb0efd700 1 -- 172.21.15.185:0/3886982630 wait complete. 2020-11-16T22:16:00.982 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:16:00.983 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:16:00.983 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2020-11-16T22:16:01.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.119+0000 7efce3fff700 1 Processor -- start 2020-11-16T22:16:01.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.119+0000 7efce3fff700 1 -- start start 2020-11-16T22:16:01.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efce40b30e0 0x7efce40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:01.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce40af130 con 0x7efce40b8820 2020-11-16T22:16:01.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce40af600 con 0x7efce4141b30 2020-11-16T22:16:01.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce40af360 con 0x7efce40b30e0 2020-11-16T22:16:01.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efcda7fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7efce4141b30 0x7efce41454e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51590/0 (socket says 172.21.15.185:51590) 2020-11-16T22:16:01.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efcda7fc700 1 -- 172.21.15.185:0/391617261 learned_addr learned my addr 172.21.15.185:0/391617261 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce37fe700 1 --2- 172.21.15.185:0/391617261 >> v2:172.21.15.96:3300/0 conn(0x7efce40b30e0 0x7efce40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716229244 0 0) 0x7efce40af600 con 0x7efce4141b30 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc4003240 con 0x7efce4141b30 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce37fe700 1 -- 172.21.15.185:0/391617261 >> v1:172.21.15.185:6789/0 conn(0x7efce4141b30 legacy=0x7efce41454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce37fe700 1 -- 172.21.15.185:0/391617261 >> v1:172.21.15.96:6789/0 conn(0x7efce40b8820 legacy=0x7efce4141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919485210 0 0) 0x7efce40af130 con 0x7efce40b8820 2020-11-16T22:16:01.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce37fe700 1 -- 172.21.15.185:0/391617261 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce40b3dc0 con 0x7efce40b30e0 2020-11-16T22:16:01.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce37fe700 1 --2- 172.21.15.185:0/391617261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce40af7e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efcdc009c90 tx=0x7efcdc0091d0).ready entity=mon.0 client_cookie=cdcda2dbb7ca87f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:01.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00ba70 con 0x7efce40b30e0 2020-11-16T22:16:01.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc00bbd0 con 0x7efce40b30e0 2020-11-16T22:16:01.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce27fc700 1 -- 172.21.15.185:0/391617261 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc011400 con 0x7efce40b30e0 2020-11-16T22:16:01.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- 172.21.15.185:0/391617261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 msgr2=0x7efce40af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 --2- 172.21.15.185:0/391617261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce40af7e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efcdc009c90 tx=0x7efcdc0091d0).stop 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- 172.21.15.185:0/391617261 shutdown_connections 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 --2- 172.21.15.185:0/391617261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce40af7e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- 172.21.15.185:0/391617261 >> 172.21.15.185:0/391617261 conn(0x7efce40645d0 msgr2=0x7efce40649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- 172.21.15.185:0/391617261 shutdown_connections 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.123+0000 7efce3fff700 1 -- 172.21.15.185:0/391617261 wait complete. 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 Processor -- start 2020-11-16T22:16:01.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- start start 2020-11-16T22:16:01.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce4143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:01.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:01.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 0x7efce41572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:01.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce4061760 con 0x7efce40b30e0 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce4143c00 con 0x7efce4062fa0 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce4156b10 con 0x7efce40b8820 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 0x7efce41572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 0x7efce41572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58162/0 (socket says 172.21.15.185:58162) 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51604/0 (socket says 172.21.15.185:51604) 2020-11-16T22:16:01.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce37fe700 1 -- 172.21.15.185:0/3880823179 learned_addr learned my addr 172.21.15.185:0/3880823179 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce2ffd700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce4143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 -- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 msgr2=0x7efce41572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 0x7efce41572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 -- 172.21.15.185:0/3880823179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 msgr2=0x7efce4143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce4143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 -- 172.21.15.185:0/3880823179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4157790 con 0x7efce40b8820 2020-11-16T22:16:01.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcda7fc700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7efce40639a0 tx=0x7efccc0079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:01.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0072e0 con 0x7efce40b8820 2020-11-16T22:16:01.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc007460 con 0x7efce40b8820 2020-11-16T22:16:01.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce4062c80 con 0x7efce40b8820 2020-11-16T22:16:01.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc005840 con 0x7efce40b8820 2020-11-16T22:16:01.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.127+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce41439d0 con 0x7efce40b8820 2020-11-16T22:16:01.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce40b3fa0 con 0x7efce40b8820 2020-11-16T22:16:01.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efccc010070 con 0x7efce40b8820 2020-11-16T22:16:01.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efcdb7fe700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 0x7efcc0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:01.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efccc06d340 con 0x7efce40b8820 2020-11-16T22:16:01.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efce2ffd700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 0x7efcc0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:01.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.131+0000 7efce2ffd700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 0x7efcc0057520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efcd40011a0 tx=0x7efcd4007970).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:01.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.139+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efccc043490 con 0x7efce40b8820 2020-11-16T22:16:01.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.347+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7efce40618d0 con 0x7efce40b8820 2020-11-16T22:16:01.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.983+0000 7efcdb7fe700 1 -- 172.21.15.185:0/3880823179 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v115) v1 ==== 146+0+0 (secure 0 0 0) 0x7efccc018020 con 0x7efce40b8820 2020-11-16T22:16:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 msgr2=0x7efcc0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 0x7efcc0057520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efcd40011a0 tx=0x7efcd4007970).stop 2020-11-16T22:16:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 msgr2=0x7efce4156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7efce40639a0 tx=0x7efccc0079e0).stop 2020-11-16T22:16:01.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 shutdown_connections 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efcc0055060 0x7efcc0057520 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efce4062fa0 0x7efce41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efce40b30e0 0x7efce4143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 --2- 172.21.15.185:0/3880823179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efce40b8820 0x7efce4156db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 >> 172.21.15.185:0/3880823179 conn(0x7efce40645d0 msgr2=0x7efce40b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 shutdown_connections 2020-11-16T22:16:01.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:01.987+0000 7efce3fff700 1 -- 172.21.15.185:0/3880823179 wait complete. 2020-11-16T22:16:01.990 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:16:02.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.139+0000 7fa96df6d700 1 Processor -- start 2020-11-16T22:16:02.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.139+0000 7fa96df6d700 1 -- start start 2020-11-16T22:16:02.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa968063fd0 0x7fa968061240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 0x7fa968143310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9680b4350 con 0x7fa9680b3c40 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680b60a0 con 0x7fa9680b4c70 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680b40d0 con 0x7fa968063fd0 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa968063fd0 0x7fa968061240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa968063fd0 0x7fa968061240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58168/0 (socket says 172.21.15.185:58168) 2020-11-16T22:16:02.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96d76c700 1 --2- 172.21.15.185:0/3276449696 >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 0x7fa968143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 -- 172.21.15.185:0/3276449696 learned_addr learned my addr 172.21.15.185:0/3276449696 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 -- 172.21.15.185:0/3276449696 >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 msgr2=0x7fa968143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 --2- 172.21.15.185:0/3276449696 >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 0x7fa968143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 -- 172.21.15.185:0/3276449696 >> v1:172.21.15.96:6789/0 conn(0x7fa9680b3c40 legacy=0x7fa9680b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa9677fe700 1 -- 172.21.15.185:0/3276449696 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151002609 0 0) 0x7fa9680b4350 con 0x7fa9680b3c40 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 -- 172.21.15.185:0/3276449696 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9680629a0 con 0x7fa968063fd0 2020-11-16T22:16:02.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96d76c700 1 --2- 172.21.15.185:0/3276449696 >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 0x7fa968143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96cf6b700 1 --2- 172.21.15.185:0/3276449696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa968061240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa96805c620 tx=0x7fa958006eb0).ready entity=mon.1 client_cookie=279f5ed48198d2de server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa9677fe700 1 -- 172.21.15.185:0/3276449696 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958009810 con 0x7fa968063fd0 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa9677fe700 1 -- 172.21.15.185:0/3276449696 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958009970 con 0x7fa968063fd0 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa9677fe700 1 -- 172.21.15.185:0/3276449696 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958009cf0 con 0x7fa968063fd0 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- 172.21.15.185:0/3276449696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 msgr2=0x7fa968061240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 --2- 172.21.15.185:0/3276449696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa968061240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa96805c620 tx=0x7fa958006eb0).stop 2020-11-16T22:16:02.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- 172.21.15.185:0/3276449696 shutdown_connections 2020-11-16T22:16:02.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 --2- 172.21.15.185:0/3276449696 >> v2:172.21.15.96:3300/0 conn(0x7fa9680b4c70 0x7fa968143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 --2- 172.21.15.185:0/3276449696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa968061240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- 172.21.15.185:0/3276449696 >> 172.21.15.185:0/3276449696 conn(0x7fa968067940 msgr2=0x7fa968067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:02.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- 172.21.15.185:0/3276449696 shutdown_connections 2020-11-16T22:16:02.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.143+0000 7fa96df6d700 1 -- 172.21.15.185:0/3276449696 wait complete. 2020-11-16T22:16:02.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 Processor -- start 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 -- start start 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa96813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 0x7fa9680afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9680b4c70 0x7fa9680b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968064ef0 con 0x7fa9680b4c70 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680b1b00 con 0x7fa968063fd0 2020-11-16T22:16:02.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680b2610 con 0x7fa9680b3c40 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96cf6b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa96813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 0x7fa9680afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96cf6b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa96813fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58172/0 (socket says 172.21.15.185:58172) 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96cf6b700 1 -- 172.21.15.185:0/2484759044 learned_addr learned my addr 172.21.15.185:0/2484759044 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96d76c700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9680b4c70 0x7fa9680b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 -- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 msgr2=0x7fa96813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa96813fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 -- 172.21.15.185:0/2484759044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9680b4c70 msgr2=0x7fa9680b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9680b4c70 0x7fa9680b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 -- 172.21.15.185:0/2484759044 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9680b0d80 con 0x7fa9680b3c40 2020-11-16T22:16:02.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa967fff700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 0x7fa9680afc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa95000c730 tx=0x7fa950007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000f470 con 0x7fa9680b3c40 2020-11-16T22:16:02.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000f5f0 con 0x7fa9680b3c40 2020-11-16T22:16:02.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9680b20d0 con 0x7fa9680b3c40 2020-11-16T22:16:02.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.147+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000f9b0 con 0x7fa9680b3c40 2020-11-16T22:16:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968140320 con 0x7fa9680b3c40 2020-11-16T22:16:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96814a170 con 0x7fa9680b3c40 2020-11-16T22:16:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa950019030 con 0x7fa9680b3c40 2020-11-16T22:16:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa964ff9700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 0x7fa940057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa96cf6b700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 0x7fa940057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa95006d230 con 0x7fa9680b3c40 2020-11-16T22:16:02.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.151+0000 7fa96cf6b700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 0x7fa940057520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fa95800beb0 tx=0x7fa958015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.159+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa950043490 con 0x7fa9680b3c40 2020-11-16T22:16:02.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.315+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fa96805d200 con 0x7fa9680b3c40 2020-11-16T22:16:02.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.323+0000 7fa964ff9700 1 -- 172.21.15.185:0/2484759044 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa950017020 con 0x7fa9680b3c40 2020-11-16T22:16:02.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 msgr2=0x7fa940057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 0x7fa940057520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fa95800beb0 tx=0x7fa958015000).stop 2020-11-16T22:16:02.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 msgr2=0x7fa9680afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 0x7fa9680afc80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa95000c730 tx=0x7fa950007a00).stop 2020-11-16T22:16:02.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 shutdown_connections 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa940055060 0x7fa940057520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa968063fd0 0x7fa96813fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9680b4c70 0x7fa9680b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 --2- 172.21.15.185:0/2484759044 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9680b3c40 0x7fa9680afc80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 >> 172.21.15.185:0/2484759044 conn(0x7fa968067940 msgr2=0x7fa9680b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 shutdown_connections 2020-11-16T22:16:02.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.327+0000 7fa96df6d700 1 -- 172.21.15.185:0/2484759044 wait complete. 2020-11-16T22:16:02.324 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:16:02.325 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:16:02.364 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:16:02.381 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:16:02.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:16:02.396 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:16:02.397 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:02.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 Processor -- start 2020-11-16T22:16:02.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 -- start start 2020-11-16T22:16:02.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 0x7fd2dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 0x7fd2dc1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc061fe0 con 0x7fd2dc0b3090 2020-11-16T22:16:02.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b2b60 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2e095f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc14c060 con 0x7fd2dc0ad110 2020-11-16T22:16:02.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2da59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 0x7fd2dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2db59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 0x7fd2dc1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2da59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 0x7fd2dc0b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58182/0 (socket says 172.21.15.185:58182) 2020-11-16T22:16:02.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2da59c700 1 -- 172.21.15.185:0/1772604331 learned_addr learned my addr 172.21.15.185:0/1772604331 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:02.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2db59e700 1 -- 172.21.15.185:0/1772604331 >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 msgr2=0x7fd2dc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:02.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2db59e700 1 --2- 172.21.15.185:0/1772604331 >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 0x7fd2dc0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2db59e700 1 -- 172.21.15.185:0/1772604331 >> v1:172.21.15.96:6789/0 conn(0x7fd2dc0b3090 legacy=0x7fd2dc14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.583+0000 7fd2db59e700 1 -- 172.21.15.185:0/1772604331 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0b0fe0 con 0x7fd2dc0ad110 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2db59e700 1 --2- 172.21.15.185:0/1772604331 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 0x7fd2dc1477b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fd2d4009c90 tx=0x7fd2d40091d0).ready entity=mon.2 client_cookie=8c7dbb46732f9188 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2d9d9b700 1 -- 172.21.15.185:0/1772604331 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400ba70 con 0x7fd2dc0ad110 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2d9d9b700 1 -- 172.21.15.185:0/1772604331 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d400bbd0 con 0x7fd2dc0ad110 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2d9d9b700 1 -- 172.21.15.185:0/1772604331 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4011400 con 0x7fd2dc0ad110 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- 172.21.15.185:0/1772604331 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 msgr2=0x7fd2dc1477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- 172.21.15.185:0/1772604331 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 0x7fd2dc1477b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fd2d4009c90 tx=0x7fd2d40091d0).stop 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- 172.21.15.185:0/1772604331 shutdown_connections 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- 172.21.15.185:0/1772604331 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0ad110 0x7fd2dc1477b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- 172.21.15.185:0/1772604331 >> v2:172.21.15.185:3300/0 conn(0x7fd2dc0b1d30 0x7fd2dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- 172.21.15.185:0/1772604331 >> 172.21.15.185:0/1772604331 conn(0x7fd2dc060130 msgr2=0x7fd2dc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- 172.21.15.185:0/1772604331 shutdown_connections 2020-11-16T22:16:02.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- 172.21.15.185:0/1772604331 wait complete. 2020-11-16T22:16:02.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 Processor -- start 2020-11-16T22:16:02.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- start start 2020-11-16T22:16:02.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2dc0ad110 0x7fd2dc0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2dc0b3090 0x7fd2dc15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc14d690 con 0x7fd2dc0b3090 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc110ec0 con 0x7fd2dc0ad110 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2e095f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc14d8c0 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2da59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2da59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51632/0 (socket says 172.21.15.185:51632) 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2da59c700 1 -- 172.21.15.185:0/773800791 learned_addr learned my addr 172.21.15.185:0/773800791 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:02.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2db59e700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2dc0b3090 0x7fd2dc15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.587+0000 7fd2dad9d700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2dc0ad110 0x7fd2dc0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 -- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2dc0ad110 msgr2=0x7fd2dc0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2dc0ad110 0x7fd2dc0af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 -- 172.21.15.185:0/773800791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2dc0b3090 msgr2=0x7fd2dc15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2dc0b3090 0x7fd2dc15b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 -- 172.21.15.185:0/773800791 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc14db60 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2da59c700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd2d000f8b0 tx=0x7fd2d0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0014670 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00147d0 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc14de60 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0014b70 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc0afed0 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc14f9f0 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd2d0024070 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 0x7fd2ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2d006e030 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2dad9d700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 0x7fd2ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.591+0000 7fd2dad9d700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 0x7fd2ac057520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fd2cc003300 tx=0x7fd2cc007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.599+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd2d0044290 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2dc05d200 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2c2ffd700 1 -- 172.21.15.185:0/773800791 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7fd2d00478b0 con 0x7fd2dc0b1d30 2020-11-16T22:16:02.784 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:02.784 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:16:02.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 msgr2=0x7fd2ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 0x7fd2ac057520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fd2cc003300 tx=0x7fd2cc007b90).stop 2020-11-16T22:16:02.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 msgr2=0x7fd2dc15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fd2d000f8b0 tx=0x7fd2d0008040).stop 2020-11-16T22:16:02.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 shutdown_connections 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2ac055060 0x7fd2ac057520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2dc0ad110 0x7fd2dc0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2dc0b3090 0x7fd2dc15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 --2- 172.21.15.185:0/773800791 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2dc0b1d30 0x7fd2dc15ae60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 >> 172.21.15.185:0/773800791 conn(0x7fd2dc060130 msgr2=0x7fd2dc149180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 shutdown_connections 2020-11-16T22:16:02.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.799+0000 7fd2e095f700 1 -- 172.21.15.185:0/773800791 wait complete. 2020-11-16T22:16:02.788 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:16:02.799 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 Processor -- start 2020-11-16T22:16:02.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- start start 2020-11-16T22:16:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7eff5c0b17e0 0x7eff5c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7eff5c0b2160 0x7eff5c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c098cc0 con 0x7eff5c066240 2020-11-16T22:16:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c0b3950 con 0x7eff5c0b17e0 2020-11-16T22:16:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c0b3bf0 con 0x7eff5c0b2160 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff61432700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7eff5c066240 0x7eff5c065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51632/0 (socket says 172.21.15.185:51632) 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff61432700 1 -- 172.21.15.185:0/2509107391 learned_addr learned my addr 172.21.15.185:0/2509107391 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 --2- 172.21.15.185:0/2509107391 >> v2:172.21.15.185:3300/0 conn(0x7eff5c0b2160 0x7eff5c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff60c31700 1 --2- 172.21.15.185:0/2509107391 >> v2:172.21.15.96:3300/0 conn(0x7eff5c0b17e0 0x7eff5c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5b7fe700 1 -- 172.21.15.185:0/2509107391 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260576104 0 0) 0x7eff5c098cc0 con 0x7eff5c066240 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5b7fe700 1 -- 172.21.15.185:0/2509107391 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff48003240 con 0x7eff5c066240 2020-11-16T22:16:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 -- 172.21.15.185:0/2509107391 >> v2:172.21.15.96:3300/0 conn(0x7eff5c0b17e0 msgr2=0x7eff5c067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:02.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 --2- 172.21.15.185:0/2509107391 >> v2:172.21.15.96:3300/0 conn(0x7eff5c0b17e0 0x7eff5c067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 -- 172.21.15.185:0/2509107391 >> v1:172.21.15.185:6789/0 conn(0x7eff5c066240 legacy=0x7eff5c065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 -- 172.21.15.185:0/2509107391 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c067b10 con 0x7eff5c0b2160 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5bfff700 1 --2- 172.21.15.185:0/2509107391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b2160 0x7eff5c14bde0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7eff5c09c7e0 tx=0x7eff50006df0).ready entity=mon.1 client_cookie=b51c2cc2c50f8055 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5b7fe700 1 -- 172.21.15.185:0/2509107391 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50009640 con 0x7eff5c0b2160 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5b7fe700 1 -- 172.21.15.185:0/2509107391 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff500097a0 con 0x7eff5c0b2160 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff5b7fe700 1 -- 172.21.15.185:0/2509107391 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50009b20 con 0x7eff5c0b2160 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- 172.21.15.185:0/2509107391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b2160 msgr2=0x7eff5c14bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 --2- 172.21.15.185:0/2509107391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b2160 0x7eff5c14bde0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7eff5c09c7e0 tx=0x7eff50006df0).stop 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- 172.21.15.185:0/2509107391 shutdown_connections 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 --2- 172.21.15.185:0/2509107391 >> v2:172.21.15.96:3300/0 conn(0x7eff5c0b17e0 0x7eff5c067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 --2- 172.21.15.185:0/2509107391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b2160 0x7eff5c14bde0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- 172.21.15.185:0/2509107391 >> 172.21.15.185:0/2509107391 conn(0x7eff5c0604a0 msgr2=0x7eff5c0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- 172.21.15.185:0/2509107391 shutdown_connections 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.935+0000 7eff62e95700 1 -- 172.21.15.185:0/2509107391 wait complete. 2020-11-16T22:16:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 Processor -- start 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- start start 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff5c066240 0x7eff5c14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff5c0b2160 0x7eff5c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c15b250 con 0x7eff5c066240 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c15b4f0 con 0x7eff5c0b17e0 2020-11-16T22:16:02.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c15b790 con 0x7eff5c0b2160 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58202/0 (socket says 172.21.15.185:58202) 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 -- 172.21.15.185:0/187440039 learned_addr learned my addr 172.21.15.185:0/187440039 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff61432700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff5c0b2160 0x7eff5c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff60c31700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff5c066240 0x7eff5c14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 -- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff5c0b2160 msgr2=0x7eff5c14a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff5c0b2160 0x7eff5c14a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 -- 172.21.15.185:0/187440039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff5c066240 msgr2=0x7eff5c14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff5c066240 0x7eff5c14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 -- 172.21.15.185:0/187440039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c15ba30 con 0x7eff5c0b17e0 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff5bfff700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7eff50009340 tx=0x7eff50015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50011c40 con 0x7eff5c0b17e0 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff50011da0 con 0x7eff5c0b17e0 2020-11-16T22:16:02.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c15bcd0 con 0x7eff5c0b17e0 2020-11-16T22:16:02.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5001e670 con 0x7eff5c0b17e0 2020-11-16T22:16:02.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.939+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c14aa30 con 0x7eff5c0b17e0 2020-11-16T22:16:02.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c05d4e0 con 0x7eff5c0b17e0 2020-11-16T22:16:02.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7eff50025030 con 0x7eff5c0b17e0 2020-11-16T22:16:02.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff58ff9700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 0x7eff34057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:02.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff5006f4d0 con 0x7eff5c0b17e0 2020-11-16T22:16:02.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff60c31700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 0x7eff34057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:02.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.943+0000 7eff60c31700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 0x7eff34057520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7eff4c006fd0 tx=0x7eff4c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:02.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:02.951+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7eff500456d0 con 0x7eff5c0b17e0 2020-11-16T22:16:03.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.107+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5c14ece0 con 0x7eff5c0b17e0 2020-11-16T22:16:03.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff58ff9700 1 -- 172.21.15.185:0/187440039 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16826 (secure 0 0 0) 0x7eff500311d0 con 0x7eff5c0b17e0 2020-11-16T22:16:03.095 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:03.095 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":32,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:07.653440+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:03.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 msgr2=0x7eff34057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 0x7eff34057520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7eff4c006fd0 tx=0x7eff4c006df0).stop 2020-11-16T22:16:03.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 msgr2=0x7eff5c14a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7eff50009340 tx=0x7eff50015000).stop 2020-11-16T22:16:03.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 shutdown_connections 2020-11-16T22:16:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7eff34055060 0x7eff34057520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff5c0b2160 0x7eff5c14a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff5c066240 0x7eff5c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 --2- 172.21.15.185:0/187440039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff5c0b17e0 0x7eff5c14a050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 >> 172.21.15.185:0/187440039 conn(0x7eff5c0604a0 msgr2=0x7eff5c062c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:03.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 shutdown_connections 2020-11-16T22:16:03.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.111+0000 7eff62e95700 1 -- 172.21.15.185:0/187440039 wait complete. 2020-11-16T22:16:03.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:15:03.444879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:15:04.633004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:16:03.301 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:16:03.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 Processor -- start 2020-11-16T22:16:03.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 -- start start 2020-11-16T22:16:03.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd6340b3020 0x7fd6340b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd634141150 0x7fd6341415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6340acdf0 con 0x7fd6340abef0 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6341462b0 con 0x7fd634141150 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.451+0000 7fd63a12d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6340ad2c0 con 0x7fd6340b3020 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd6340b3020 0x7fd6340b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd6340b3020 0x7fd6340b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58208/0 (socket says 172.21.15.185:58208) 2020-11-16T22:16:03.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 -- 172.21.15.185:0/2685719592 learned_addr learned my addr 172.21.15.185:0/2685719592 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:03.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63992c700 1 --2- 172.21.15.185:0/2685719592 >> v2:172.21.15.96:3300/0 conn(0x7fd634141150 0x7fd6341415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 -- 172.21.15.185:0/2685719592 >> v2:172.21.15.96:3300/0 conn(0x7fd634141150 msgr2=0x7fd6341415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:03.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 --2- 172.21.15.185:0/2685719592 >> v2:172.21.15.96:3300/0 conn(0x7fd634141150 0x7fd6341415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 -- 172.21.15.185:0/2685719592 >> v1:172.21.15.96:6789/0 conn(0x7fd6340abef0 legacy=0x7fd634145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 -- 172.21.15.185:0/2685719592 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340ad020 con 0x7fd6340b3020 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63912b700 1 --2- 172.21.15.185:0/2685719592 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd6340b17c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fd634064930 tx=0x7fd62c006eb0).ready entity=mon.1 client_cookie=ed0cb5920699f77e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd633fff700 1 -- 172.21.15.185:0/2685719592 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c009810 con 0x7fd6340b3020 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd633fff700 1 -- 172.21.15.185:0/2685719592 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c009970 con 0x7fd6340b3020 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd633fff700 1 -- 172.21.15.185:0/2685719592 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c009cf0 con 0x7fd6340b3020 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 -- 172.21.15.185:0/2685719592 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 msgr2=0x7fd6340b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2685719592 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd6340b17c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fd634064930 tx=0x7fd62c006eb0).stop 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 -- 172.21.15.185:0/2685719592 shutdown_connections 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2685719592 >> v2:172.21.15.96:3300/0 conn(0x7fd634141150 0x7fd6341415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2685719592 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd6340b17c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 -- 172.21.15.185:0/2685719592 >> 172.21.15.185:0/2685719592 conn(0x7fd634060130 msgr2=0x7fd634060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 -- 172.21.15.185:0/2685719592 shutdown_connections 2020-11-16T22:16:03.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.455+0000 7fd63a12d700 1 -- 172.21.15.185:0/2685719592 wait complete. 2020-11-16T22:16:03.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 Processor -- start 2020-11-16T22:16:03.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 -- start start 2020-11-16T22:16:03.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6340abef0 0x7fd6340ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd634141150 0x7fd634151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340b0740 con 0x7fd6340abef0 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634144f00 con 0x7fd6340b3020 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63a12d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd634144c60 con 0x7fd634141150 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63892a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63992c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd634141150 0x7fd634151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63892a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58216/0 (socket says 172.21.15.185:58216) 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63892a700 1 -- 172.21.15.185:0/2849742245 learned_addr learned my addr 172.21.15.185:0/2849742245 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.459+0000 7fd63912b700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6340abef0 0x7fd6340ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 -- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd634141150 msgr2=0x7fd634151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd634141150 0x7fd634151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 -- 172.21.15.185:0/2849742245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6340abef0 msgr2=0x7fd6340ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6340abef0 0x7fd6340ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 -- 172.21.15.185:0/2849742245 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634151370 con 0x7fd6340b3020 2020-11-16T22:16:03.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63892a700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fd62400c6e0 tx=0x7fd624007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624007020 con 0x7fd6340b3020 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd624007180 con 0x7fd6340b3020 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634151610 con 0x7fd6340b3020 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400e560 con 0x7fd6340b3020 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634143140 con 0x7fd6340b3020 2020-11-16T22:16:03.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6341517a0 con 0x7fd6340b3020 2020-11-16T22:16:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd624010070 con 0x7fd6340b3020 2020-11-16T22:16:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 0x7fd610057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63912b700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 0x7fd610057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd62406d0c0 con 0x7fd6340b3020 2020-11-16T22:16:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.463+0000 7fd63912b700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 0x7fd610057520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fd62c006ef0 tx=0x7fd62c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.471+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd624043320 con 0x7fd6340b3020 2020-11-16T22:16:03.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd63405d200 con 0x7fd6340b3020 2020-11-16T22:16:03.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd6317fa700 1 -- 172.21.15.185:0/2849742245 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v32) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd624046940 con 0x7fd6340b3020 2020-11-16T22:16:03.612 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:16:03.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 msgr2=0x7fd610057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 0x7fd610057520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fd62c006ef0 tx=0x7fd62c015000).stop 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 msgr2=0x7fd63413ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fd62400c6e0 tx=0x7fd624007c50).stop 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 shutdown_connections 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd610055060 0x7fd610057520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd634141150 0x7fd634151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6340abef0 0x7fd6340ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 --2- 172.21.15.185:0/2849742245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd6340b3020 0x7fd63413ee90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.627+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 >> 172.21.15.185:0/2849742245 conn(0x7fd634060130 msgr2=0x7fd6340b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:03.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.631+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 shutdown_connections 2020-11-16T22:16:03.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.631+0000 7fd63a12d700 1 -- 172.21.15.185:0/2849742245 wait complete. 2020-11-16T22:16:03.627 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:16:03.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 Processor -- start 2020-11-16T22:16:03.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 -- start start 2020-11-16T22:16:03.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1bf00b5d70 0x7f1bf00ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf00b7290 con 0x7f1bf01475d0 2020-11-16T22:16:03.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00b4bf0 con 0x7f1bf00b5d70 2020-11-16T22:16:03.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf64e2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b74c0 con 0x7f1bf00b4470 2020-11-16T22:16:03.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf4cdf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf4cdf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51666/0 (socket says 172.21.15.185:51666) 2020-11-16T22:16:03.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/2599395981 learned_addr learned my addr 172.21.15.185:0/2599395981 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:03.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.767+0000 7f1bf54e0700 1 --2- 172.21.15.185:0/2599395981 >> v2:172.21.15.185:3300/0 conn(0x7f1bf00b5d70 0x7f1bf00ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/2599395981 >> v2:172.21.15.185:3300/0 conn(0x7f1bf00b5d70 msgr2=0x7f1bf00ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf4cdf700 1 --2- 172.21.15.185:0/2599395981 >> v2:172.21.15.185:3300/0 conn(0x7f1bf00b5d70 0x7f1bf00ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/2599395981 >> v1:172.21.15.96:6789/0 conn(0x7f1bf01475d0 legacy=0x7f1bf0147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/2599395981 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00b7760 con 0x7f1bf00b4470 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf4cdf700 1 --2- 172.21.15.185:0/2599395981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1bec006fd0 tx=0x7f1bec006eb0).ready entity=mon.2 client_cookie=21e1a7b9440ecc87 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1be7fff700 1 -- 172.21.15.185:0/2599395981 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec009810 con 0x7f1bf00b4470 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1be7fff700 1 -- 172.21.15.185:0/2599395981 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec009970 con 0x7f1bf00b4470 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1be7fff700 1 -- 172.21.15.185:0/2599395981 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec009cf0 con 0x7f1bf00b4470 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- 172.21.15.185:0/2599395981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 msgr2=0x7f1bf00b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/2599395981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1bec006fd0 tx=0x7f1bec006eb0).stop 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- 172.21.15.185:0/2599395981 shutdown_connections 2020-11-16T22:16:03.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/2599395981 >> v2:172.21.15.185:3300/0 conn(0x7f1bf00b5d70 0x7f1bf00ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/2599395981 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf00b4470 0x7f1bf00b7c00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- 172.21.15.185:0/2599395981 >> 172.21.15.185:0/2599395981 conn(0x7f1bf005ba80 msgr2=0x7f1bf005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:03.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- 172.21.15.185:0/2599395981 shutdown_connections 2020-11-16T22:16:03.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- 172.21.15.185:0/2599395981 wait complete. 2020-11-16T22:16:03.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 Processor -- start 2020-11-16T22:16:03.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.771+0000 7f1bf64e2700 1 -- start start 2020-11-16T22:16:03.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 0x7f1bf0146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf01475d0 0x7f1bf00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00adf90 con 0x7f1bf00b4470 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf0153f90 con 0x7f1bf00b5d70 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00ad110 con 0x7f1bf01475d0 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58230/0 (socket says 172.21.15.185:58230) 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/863072905 learned_addr learned my addr 172.21.15.185:0/863072905 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:03.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf5ce1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf01475d0 0x7f1bf00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf54e0700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 0x7f1bf0146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf01475d0 msgr2=0x7f1bf00adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf01475d0 0x7f1bf00adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/863072905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 msgr2=0x7f1bf0146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 0x7f1bf0146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 -- 172.21.15.185:0/863072905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0146d50 con 0x7f1bf00b5d70 2020-11-16T22:16:03.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf54e0700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 0x7f1bf0146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf4cdf700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f1bf00b7600 tx=0x7f1bec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec011d50 con 0x7f1bf00b5d70 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec01e460 con 0x7f1bf00b5d70 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf00af460 con 0x7f1bf00b5d70 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01e820 con 0x7f1bf00b5d70 2020-11-16T22:16:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf00aeba0 con 0x7f1bf00b5d70 2020-11-16T22:16:03.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1bec025030 con 0x7f1bf00b5d70 2020-11-16T22:16:03.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf00ad2d0 con 0x7f1bf00b5d70 2020-11-16T22:16:03.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.775+0000 7f1be57fa700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 0x7f1bc8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.779+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1bec06f300 con 0x7f1bf00b5d70 2020-11-16T22:16:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.779+0000 7f1bf54e0700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 0x7f1bc8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.779+0000 7f1bf54e0700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 0x7f1bc8057520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f1be0006fd0 tx=0x7f1be0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:03.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.783+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1bec045610 con 0x7f1bf00b5d70 2020-11-16T22:16:03.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.935+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1bf005d200 con 0x7f1bf00b5d70 2020-11-16T22:16:03.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.935+0000 7f1be57fa700 1 -- 172.21.15.185:0/863072905 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1bec023070 con 0x7f1bf00b5d70 2020-11-16T22:16:03.921 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 msgr2=0x7f1bc8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 0x7f1bc8057520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f1be0006fd0 tx=0x7f1be0008040).stop 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 msgr2=0x7f1bf00ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f1bf00b7600 tx=0x7f1bec015000).stop 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 shutdown_connections 2020-11-16T22:16:03.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1bc8055060 0x7f1bc8057520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1bf01475d0 0x7f1bf00adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bf00b4470 0x7f1bf0146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 --2- 172.21.15.185:0/863072905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1bf00b5d70 0x7f1bf00ad5b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 >> 172.21.15.185:0/863072905 conn(0x7f1bf005ba80 msgr2=0x7f1bf00b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 shutdown_connections 2020-11-16T22:16:03.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:03.939+0000 7f1bf64e2700 1 -- 172.21.15.185:0/863072905 wait complete. 2020-11-16T22:16:03.941 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:16:04.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.083+0000 7f62110b7700 1 Processor -- start 2020-11-16T22:16:04.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.083+0000 7f62110b7700 1 -- start start 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f620c0ba800 0x7f620c144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c1446e0 con 0x7f620c13dd30 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c1444f0 con 0x7f620c0ba800 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c144980 con 0x7f620c0b5c40 2020-11-16T22:16:04.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51680/0 (socket says 172.21.15.185:51680) 2020-11-16T22:16:04.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 -- 172.21.15.185:0/2109809278 learned_addr learned my addr 172.21.15.185:0/2109809278 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:04.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620b7fe700 1 --2- 172.21.15.185:0/2109809278 >> v2:172.21.15.96:3300/0 conn(0x7f620c0ba800 0x7f620c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 -- 172.21.15.185:0/2109809278 >> v2:172.21.15.96:3300/0 conn(0x7f620c0ba800 msgr2=0x7f620c144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 --2- 172.21.15.185:0/2109809278 >> v2:172.21.15.96:3300/0 conn(0x7f620c0ba800 0x7f620c144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 -- 172.21.15.185:0/2109809278 >> v1:172.21.15.96:6789/0 conn(0x7f620c13dd30 legacy=0x7f620c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 -- 172.21.15.185:0/2109809278 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c144c20 con 0x7f620c0b5c40 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620bfff700 1 --2- 172.21.15.185:0/2109809278 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f61fc006fd0 tx=0x7f61fc006eb0).ready entity=mon.2 client_cookie=b493e380602d4b10 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620affd700 1 -- 172.21.15.185:0/2109809278 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0098d0 con 0x7f620c0b5c40 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620affd700 1 -- 172.21.15.185:0/2109809278 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc009a30 con 0x7f620c0b5c40 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f620affd700 1 -- 172.21.15.185:0/2109809278 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc009db0 con 0x7f620c0b5c40 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- 172.21.15.185:0/2109809278 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 msgr2=0x7f620c0b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 --2- 172.21.15.185:0/2109809278 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f61fc006fd0 tx=0x7f61fc006eb0).stop 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- 172.21.15.185:0/2109809278 shutdown_connections 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 --2- 172.21.15.185:0/2109809278 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c0b5c40 0x7f620c0b6020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 --2- 172.21.15.185:0/2109809278 >> v2:172.21.15.96:3300/0 conn(0x7f620c0ba800 0x7f620c144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- 172.21.15.185:0/2109809278 >> 172.21.15.185:0/2109809278 conn(0x7f620c05ba80 msgr2=0x7f620c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- 172.21.15.185:0/2109809278 shutdown_connections 2020-11-16T22:16:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.087+0000 7f62110b7700 1 -- 172.21.15.185:0/2109809278 wait complete. 2020-11-16T22:16:04.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 Processor -- start 2020-11-16T22:16:04.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 -- start start 2020-11-16T22:16:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f620c0b5c40 0x7f620c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 0x7f620c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c156ae0 con 0x7f620c0b5c40 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c156d80 con 0x7f620c0ba800 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62110b7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c157020 con 0x7f620c13dd30 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62108b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 0x7f620c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58242/0 (socket says 172.21.15.185:58242) 2020-11-16T22:16:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 -- 172.21.15.185:0/3094714765 learned_addr learned my addr 172.21.15.185:0/3094714765 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620bfff700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f620c0b5c40 0x7f620c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 -- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 msgr2=0x7f620c0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 0x7f620c0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 -- 172.21.15.185:0/3094714765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f620c0b5c40 msgr2=0x7f620c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f620c0b5c40 0x7f620c1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f620b7fe700 1 -- 172.21.15.185:0/3094714765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c149f70 con 0x7f620c0ba800 2020-11-16T22:16:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.091+0000 7f62108b6700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 0x7f620c0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f620b7fe700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f61f400d910 tx=0x7f61f4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400f5e0 con 0x7f620c0ba800 2020-11-16T22:16:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f400f740 con 0x7f620c0ba800 2020-11-16T22:16:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0b0eb0 con 0x7f620c0ba800 2020-11-16T22:16:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400fae0 con 0x7f620c0ba800 2020-11-16T22:16:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c145b10 con 0x7f620c0ba800 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c05d200 con 0x7f620c0ba800 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f61f4012420 con 0x7f620c0ba800 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 0x7f61e8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f620bfff700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 0x7f61e8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f61f406d930 con 0x7f620c0ba800 2020-11-16T22:16:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.095+0000 7f620bfff700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 0x7f61e8057550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f61fc00b1b0 tx=0x7f61fc015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.103+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f61f4043b30 con 0x7f620c0ba800 2020-11-16T22:16:04.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.255+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f620c05c720 con 0x7f620c0ba800 2020-11-16T22:16:04.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.255+0000 7f6203fff700 1 -- 172.21.15.185:0/3094714765 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+10 (secure 0 0 0) 0x7f61f4047150 con 0x7f620c0ba800 2020-11-16T22:16:04.241 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:04.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 msgr2=0x7f61e8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 0x7f61e8057550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f61fc00b1b0 tx=0x7f61fc015000).stop 2020-11-16T22:16:04.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 msgr2=0x7f620c145400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f61f400d910 tx=0x7f61f4008040).stop 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 shutdown_connections 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f61e8055090 0x7f61e8057550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f620c13dd30 0x7f620c0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f620c0b5c40 0x7f620c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 --2- 172.21.15.185:0/3094714765 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f620c0ba800 0x7f620c145400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 >> 172.21.15.185:0/3094714765 conn(0x7f620c05ba80 msgr2=0x7f620c141460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 shutdown_connections 2020-11-16T22:16:04.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.259+0000 7f62110b7700 1 -- 172.21.15.185:0/3094714765 wait complete. 2020-11-16T22:16:04.256 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:16:04.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 Processor -- start 2020-11-16T22:16:04.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 -- start start 2020-11-16T22:16:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 0x7ff0a00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a00b02e0 con 0x7ff0a00b74b0 2020-11-16T22:16:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a0143fa0 con 0x7ff0a0144c00 2020-11-16T22:16:04.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a7e8c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00b0580 con 0x7ff0a00ade00 2020-11-16T22:16:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51696/0 (socket says 172.21.15.185:51696) 2020-11-16T22:16:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 -- 172.21.15.185:0/2464668048 learned_addr learned my addr 172.21.15.185:0/2464668048 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:04.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a5427700 1 --2- 172.21.15.185:0/2464668048 >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 0x7ff0a00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 -- 172.21.15.185:0/2464668048 >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 msgr2=0x7ff0a00afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 --2- 172.21.15.185:0/2464668048 >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 0x7ff0a00afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 -- 172.21.15.185:0/2464668048 >> v1:172.21.15.96:6789/0 conn(0x7ff0a00b74b0 legacy=0x7ff0a00ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 -- 172.21.15.185:0/2464668048 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00ad250 con 0x7ff0a00ade00 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a5427700 1 --2- 172.21.15.185:0/2464668048 >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 0x7ff0a00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a6429700 1 --2- 172.21.15.185:0/2464668048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7ff098009d50 tx=0x7ff098009290).ready entity=mon.2 client_cookie=af265d1b42be7ba7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a4c26700 1 -- 172.21.15.185:0/2464668048 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800bc80 con 0x7ff0a00ade00 2020-11-16T22:16:04.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a4c26700 1 -- 172.21.15.185:0/2464668048 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09800bde0 con 0x7ff0a00ade00 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.403+0000 7ff0a4c26700 1 -- 172.21.15.185:0/2464668048 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098011610 con 0x7ff0a00ade00 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/2464668048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 msgr2=0x7ff0a0142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/2464668048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7ff098009d50 tx=0x7ff098009290).stop 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/2464668048 shutdown_connections 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/2464668048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a00ade00 0x7ff0a0142ef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/2464668048 >> v2:172.21.15.96:3300/0 conn(0x7ff0a0144c00 0x7ff0a00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/2464668048 >> 172.21.15.185:0/2464668048 conn(0x7ff0a0067860 msgr2=0x7ff0a0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/2464668048 shutdown_connections 2020-11-16T22:16:04.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/2464668048 wait complete. 2020-11-16T22:16:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 Processor -- start 2020-11-16T22:16:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- start start 2020-11-16T22:16:04.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a00b74b0 0x7ff0a00b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 0x7ff0a00b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0145770 con 0x7ff0a00b74b0 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00b1a00 con 0x7ff0a00ade00 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a7e8c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01469a0 con 0x7ff0a0144c00 2020-11-16T22:16:04.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a6429700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 0x7ff0a00b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58256/0 (socket says 172.21.15.185:58256) 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 -- 172.21.15.185:0/908725892 learned_addr learned my addr 172.21.15.185:0/908725892 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5427700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a00b74b0 0x7ff0a00b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 -- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 msgr2=0x7ff0a00b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 0x7ff0a00b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 -- 172.21.15.185:0/908725892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a00b74b0 msgr2=0x7ff0a00b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a00b74b0 0x7ff0a00b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a5c28700 1 -- 172.21.15.185:0/908725892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0145d00 con 0x7ff0a00ade00 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.407+0000 7ff0a6429700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 0x7ff0a00b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a5c28700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff09c00c770 tx=0x7ff09c007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00e560 con 0x7ff0a00ade00 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00e6e0 con 0x7ff0a00ade00 2020-11-16T22:16:04.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a0140330 con 0x7ff0a00ade00 2020-11-16T22:16:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00eaa0 con 0x7ff0a00ade00 2020-11-16T22:16:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a0146690 con 0x7ff0a00ade00 2020-11-16T22:16:04.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a005d4e0 con 0x7ff0a00ade00 2020-11-16T22:16:04.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff09c019030 con 0x7ff0a00ade00 2020-11-16T22:16:04.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 0x7ff078057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a5427700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 0x7ff078057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff09c0470c0 con 0x7ff0a00ade00 2020-11-16T22:16:04.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.411+0000 7ff0a5427700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 0x7ff078057550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff090006fd0 tx=0x7ff090008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.423+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff09c043bf0 con 0x7ff0a00ade00 2020-11-16T22:16:04.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.567+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0a0145e90 con 0x7ff0a00ade00 2020-11-16T22:16:04.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.567+0000 7ff08dffb700 1 -- 172.21.15.185:0/908725892 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff09c017030 con 0x7ff0a00ade00 2020-11-16T22:16:04.555 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:04.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 msgr2=0x7ff078057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 0x7ff078057550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff090006fd0 tx=0x7ff090008040).stop 2020-11-16T22:16:04.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 msgr2=0x7ff0a0147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff09c00c770 tx=0x7ff09c007c10).stop 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 shutdown_connections 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff078055090 0x7ff078057550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff0a00ade00 0x7ff0a0147460 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff0a0144c00 0x7ff0a00b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 --2- 172.21.15.185:0/908725892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a00b74b0 0x7ff0a00b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 >> 172.21.15.185:0/908725892 conn(0x7ff0a0067860 msgr2=0x7ff0a01414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 shutdown_connections 2020-11-16T22:16:04.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.571+0000 7ff0a7e8c700 1 -- 172.21.15.185:0/908725892 wait complete. 2020-11-16T22:16:04.568 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:04.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 Processor -- start 2020-11-16T22:16:04.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 -- start start 2020-11-16T22:16:04.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 0x7f6c8c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c0abf90 con 0x7f6c8c0669a0 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c0ac230 con 0x7f6c8c061e60 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c9136c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0ac4d0 con 0x7f6c8c0acc10 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c90b6b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6c8c061e60 0x7f6c8c1437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51706/0 (socket says 172.21.15.185:51706) 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c90b6b700 1 -- 172.21.15.185:0/1649669875 learned_addr learned my addr 172.21.15.185:0/1649669875 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:04.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c83fff700 1 --2- 172.21.15.185:0/1649669875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 0x7f6c8c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c8b7fe700 1 -- 172.21.15.185:0/1649669875 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332686104 0 0) 0x7f6c8c0ac230 con 0x7f6c8c061e60 2020-11-16T22:16:04.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c8b7fe700 1 -- 172.21.15.185:0/1649669875 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c6c003240 con 0x7f6c8c061e60 2020-11-16T22:16:04.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c83fff700 1 -- 172.21.15.185:0/1649669875 >> v1:172.21.15.185:6789/0 conn(0x7f6c8c061e60 legacy=0x7f6c8c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c83fff700 1 -- 172.21.15.185:0/1649669875 >> v1:172.21.15.96:6789/0 conn(0x7f6c8c0669a0 legacy=0x7f6c8c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c83fff700 1 -- 172.21.15.185:0/1649669875 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0ac770 con 0x7f6c8c0acc10 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c83fff700 1 --2- 172.21.15.185:0/1649669875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 0x7f6c8c1432e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6c7400b9d0 tx=0x7f6c74006df0).ready entity=mon.2 client_cookie=1998b14388d923eb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c8b7fe700 1 -- 172.21.15.185:0/1649669875 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740097b0 con 0x7f6c8c0acc10 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c8b7fe700 1 -- 172.21.15.185:0/1649669875 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c74009910 con 0x7f6c8c0acc10 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.703+0000 7f6c8b7fe700 1 -- 172.21.15.185:0/1649669875 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74009c90 con 0x7f6c8c0acc10 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/1649669875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 msgr2=0x7f6c8c1432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 --2- 172.21.15.185:0/1649669875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 0x7f6c8c1432e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6c7400b9d0 tx=0x7f6c74006df0).stop 2020-11-16T22:16:04.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/1649669875 shutdown_connections 2020-11-16T22:16:04.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 --2- 172.21.15.185:0/1649669875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0acc10 0x7f6c8c1432e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/1649669875 >> 172.21.15.185:0/1649669875 conn(0x7f6c8c060130 msgr2=0x7f6c8c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/1649669875 shutdown_connections 2020-11-16T22:16:04.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/1649669875 wait complete. 2020-11-16T22:16:04.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 Processor -- start 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- start start 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0669a0 0x7f6c8c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 0x7f6c8c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0b0490 con 0x7f6c8c153a30 2020-11-16T22:16:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0af780 con 0x7f6c8c061e60 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0afa20 con 0x7f6c8c0669a0 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c83fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0669a0 0x7f6c8c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58270/0 (socket says 172.21.15.185:58270) 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 -- 172.21.15.185:0/4039810283 learned_addr learned my addr 172.21.15.185:0/4039810283 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c90b6b700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 0x7f6c8c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 -- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0669a0 msgr2=0x7f6c8c0ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0669a0 0x7f6c8c0ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 -- 172.21.15.185:0/4039810283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 msgr2=0x7f6c8c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 0x7f6c8c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 -- 172.21.15.185:0/4039810283 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0afcc0 con 0x7f6c8c061e60 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c90b6b700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 0x7f6c8c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c8bfff700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6c7c00c710 tx=0x7f6c7c007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00e560 con 0x7f6c8c061e60 2020-11-16T22:16:04.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c00e6c0 con 0x7f6c8c061e60 2020-11-16T22:16:04.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c0aff60 con 0x7f6c8c061e60 2020-11-16T22:16:04.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00ea80 con 0x7f6c8c061e60 2020-11-16T22:16:04.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.707+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c0b6060 con 0x7f6c8c061e60 2020-11-16T22:16:04.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c0ac970 con 0x7f6c8c061e60 2020-11-16T22:16:04.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6c7c019030 con 0x7f6c8c061e60 2020-11-16T22:16:04.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c88ff9700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 0x7f6c68057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:04.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6c7c06d1d0 con 0x7f6c8c061e60 2020-11-16T22:16:04.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c83fff700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 0x7f6c68057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:04.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.711+0000 7f6c83fff700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 0x7f6c68057520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6c7400b150 tx=0x7f6c74015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:04.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.719+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6c8c0ac970 con 0x7f6c8c061e60 2020-11-16T22:16:04.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.911+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c8c0ac970 con 0x7f6c8c061e60 2020-11-16T22:16:04.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c88ff9700 1 -- 172.21.15.185:0/4039810283 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2519 (secure 0 0 0) 0x7f6c8c0ac970 con 0x7f6c8c061e60 2020-11-16T22:16:04.899 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:04.899 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:15:05.635355+0000","modified":"2020-11-16T22:15:39.164387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4932,"mds_1":4927,"mds_2":4682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4682":{"gid":4682,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/959337378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":959337378},{"type":"v1","addr":"172.21.15.185:6835","nonce":959337378}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_4927":{"gid":4927,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6833/404895834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":404895834},{"type":"v1","addr":"172.21.15.96:6833","nonce":404895834}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_4932":{"gid":4932,"name":"b","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/2155443116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2155443116},{"type":"v1","addr":"172.21.15.96:6835","nonce":2155443116}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-16T22:16:04.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 msgr2=0x7f6c68057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 0x7f6c68057520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6c7400b150 tx=0x7f6c74015000).stop 2020-11-16T22:16:04.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 msgr2=0x7f6c8c0b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:04.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6c7c00c710 tx=0x7f6c7c007c10).stop 2020-11-16T22:16:04.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 shutdown_connections 2020-11-16T22:16:04.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6c68055060 0x7f6c68057520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6c8c061e60 0x7f6c8c0b57a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c153a30 0x7f6c8c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 --2- 172.21.15.185:0/4039810283 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6c8c0669a0 0x7f6c8c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:04.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 >> 172.21.15.185:0/4039810283 conn(0x7f6c8c060130 msgr2=0x7f6c8c141690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:04.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 shutdown_connections 2020-11-16T22:16:04.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:04.915+0000 7f6c9136c700 1 -- 172.21.15.185:0/4039810283 wait complete. 2020-11-16T22:16:04.903 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 26 2020-11-16T22:16:04.913 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:05.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 Processor -- start 2020-11-16T22:16:05.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 -- start start 2020-11-16T22:16:05.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8400b2920 0x7fb8400b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb840066ad0 con 0x7fb8400b7470 2020-11-16T22:16:05.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb840140d10 con 0x7fb8400b2920 2020-11-16T22:16:05.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb844c5c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb840144d90 con 0x7fb840141100 2020-11-16T22:16:05.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51726/0 (socket says 172.21.15.185:51726) 2020-11-16T22:16:05.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83ffff700 1 -- 172.21.15.185:0/3296382316 learned_addr learned my addr 172.21.15.185:0/3296382316 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:05.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83effd700 1 --2- 172.21.15.185:0/3296382316 >> v2:172.21.15.96:3300/0 conn(0x7fb8400b2920 0x7fb8400b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83ffff700 1 -- 172.21.15.185:0/3296382316 >> v2:172.21.15.96:3300/0 conn(0x7fb8400b2920 msgr2=0x7fb8400b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:05.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.051+0000 7fb83ffff700 1 --2- 172.21.15.185:0/3296382316 >> v2:172.21.15.96:3300/0 conn(0x7fb8400b2920 0x7fb8400b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83ffff700 1 -- 172.21.15.185:0/3296382316 >> v1:172.21.15.96:6789/0 conn(0x7fb8400b7470 legacy=0x7fb84013e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83ffff700 1 -- 172.21.15.185:0/3296382316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840144ff0 con 0x7fb840141100 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83ffff700 1 --2- 172.21.15.185:0/3296382316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fb838009d50 tx=0x7fb838009290).ready entity=mon.2 client_cookie=22fec2df7b5d5a4d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83e7fc700 1 -- 172.21.15.185:0/3296382316 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800bc50 con 0x7fb840141100 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83e7fc700 1 -- 172.21.15.185:0/3296382316 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800bdb0 con 0x7fb840141100 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb83e7fc700 1 -- 172.21.15.185:0/3296382316 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838011610 con 0x7fb840141100 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 -- 172.21.15.185:0/3296382316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 msgr2=0x7fb840141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3296382316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fb838009d50 tx=0x7fb838009290).stop 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 -- 172.21.15.185:0/3296382316 shutdown_connections 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3296382316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb840141100 0x7fb840141570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3296382316 >> v2:172.21.15.96:3300/0 conn(0x7fb8400b2920 0x7fb8400b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 -- 172.21.15.185:0/3296382316 >> 172.21.15.185:0/3296382316 conn(0x7fb8400637b0 msgr2=0x7fb8400b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 -- 172.21.15.185:0/3296382316 shutdown_connections 2020-11-16T22:16:05.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.055+0000 7fb844c5c700 1 -- 172.21.15.185:0/3296382316 wait complete. 2020-11-16T22:16:05.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 Processor -- start 2020-11-16T22:16:05.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- start start 2020-11-16T22:16:05.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 0x7fb8400af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb840141100 0x7fb84013a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb840060e70 con 0x7fb840141100 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400ad810 con 0x7fb8400b7470 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400adab0 con 0x7fb8400b2920 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58284/0 (socket says 172.21.15.185:58284) 2020-11-16T22:16:05.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 -- 172.21.15.185:0/3538046512 learned_addr learned my addr 172.21.15.185:0/3538046512 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:05.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83f7fe700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 0x7fb8400af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83ffff700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb840141100 0x7fb84013a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 -- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 msgr2=0x7fb8400af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 0x7fb8400af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 -- 172.21.15.185:0/3538046512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb840141100 msgr2=0x7fb84013a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb840141100 0x7fb84013a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 -- 172.21.15.185:0/3538046512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840154610 con 0x7fb8400b7470 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83f7fe700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 0x7fb8400af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb83effd700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb82800eba0 tx=0x7fb828006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800ca60 con 0x7fb8400b7470 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800cbc0 con 0x7fb8400b7470 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb840060af0 con 0x7fb8400b7470 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8280123f0 con 0x7fb8400b7470 2020-11-16T22:16:05.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.059+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84013e030 con 0x7fb8400b7470 2020-11-16T22:16:05.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8400adc70 con 0x7fb8400b7470 2020-11-16T22:16:05.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb828012760 con 0x7fb8400b7470 2020-11-16T22:16:05.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb8377fe700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 0x7fb81c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb83f7fe700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 0x7fb81c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb82806e270 con 0x7fb8400b7470 2020-11-16T22:16:05.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.063+0000 7fb83f7fe700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 0x7fb81c057520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fb8300010d0 tx=0x7fb830009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.067+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb8280444d0 con 0x7fb8400b7470 2020-11-16T22:16:05.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.219+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb840060ce0 con 0x7fb8400b7470 2020-11-16T22:16:05.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.219+0000 7fb8377fe700 1 -- 172.21.15.185:0/3538046512 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fb828047af0 con 0x7fb8400b7470 2020-11-16T22:16:05.206 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:05.206 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":32,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:15:07.653440+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:15:03.444879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-16T22:15:04.633004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 msgr2=0x7fb81c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 0x7fb81c057520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fb8300010d0 tx=0x7fb830009040).stop 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 msgr2=0x7fb84013d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fb82800eba0 tx=0x7fb828006df0).stop 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 shutdown_connections 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb81c055060 0x7fb81c057520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb840141100 0x7fb84013a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8400b2920 0x7fb8400af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 --2- 172.21.15.185:0/3538046512 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8400b7470 0x7fb84013d4f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 >> 172.21.15.185:0/3538046512 conn(0x7fb8400637b0 msgr2=0x7fb84013c810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 shutdown_connections 2020-11-16T22:16:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.223+0000 7fb844c5c700 1 -- 172.21.15.185:0/3538046512 wait complete. 2020-11-16T22:16:05.220 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:16:05.221 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:16:05.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.351+0000 7f3907156700 1 Processor -- start 2020-11-16T22:16:05.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.351+0000 7f3907156700 1 -- start start 2020-11-16T22:16:05.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3900065ab0 0x7f39000b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39000b5110 con 0x7f39001476d0 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000b38a0 con 0x7f3900065ab0 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b3b40 con 0x7f3900061240 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3904ef2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3900065ab0 0x7f39000b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51736/0 (socket says 172.21.15.185:51736) 2020-11-16T22:16:05.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 -- 172.21.15.185:0/3204718470 learned_addr learned my addr 172.21.15.185:0/3204718470 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:05.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 -- 172.21.15.185:0/3204718470 >> v2:172.21.15.185:3300/0 conn(0x7f3900065ab0 msgr2=0x7f39000b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:05.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 --2- 172.21.15.185:0/3204718470 >> v2:172.21.15.185:3300/0 conn(0x7f3900065ab0 0x7f39000b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 -- 172.21.15.185:0/3204718470 >> v1:172.21.15.96:6789/0 conn(0x7f39001476d0 legacy=0x7f39001436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 -- 172.21.15.185:0/3204718470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000b5340 con 0x7f3900061240 2020-11-16T22:16:05.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38fffff700 1 --2- 172.21.15.185:0/3204718470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f38f4006fd0 tx=0x7f38f4006eb0).ready entity=mon.2 client_cookie=e3c0395d32f1b9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38ff7fe700 1 -- 172.21.15.185:0/3204718470 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4009810 con 0x7f3900061240 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38ff7fe700 1 -- 172.21.15.185:0/3204718470 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4009970 con 0x7f3900061240 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f38ff7fe700 1 -- 172.21.15.185:0/3204718470 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4009cf0 con 0x7f3900061240 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- 172.21.15.185:0/3204718470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 msgr2=0x7f39000b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 --2- 172.21.15.185:0/3204718470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f38f4006fd0 tx=0x7f38f4006eb0).stop 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- 172.21.15.185:0/3204718470 shutdown_connections 2020-11-16T22:16:05.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 --2- 172.21.15.185:0/3204718470 >> v2:172.21.15.185:3300/0 conn(0x7f3900065ab0 0x7f39000b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 --2- 172.21.15.185:0/3204718470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900061240 0x7f39000b3200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- 172.21.15.185:0/3204718470 >> 172.21.15.185:0/3204718470 conn(0x7f3900060130 msgr2=0x7f3900060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:05.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- 172.21.15.185:0/3204718470 shutdown_connections 2020-11-16T22:16:05.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.355+0000 7f3907156700 1 -- 172.21.15.185:0/3204718470 wait complete. 2020-11-16T22:16:05.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 Processor -- start 2020-11-16T22:16:05.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- start start 2020-11-16T22:16:05.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 0x7f3900099160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39001476d0 0x7f3900142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b6810 con 0x7f39001476d0 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b6ab0 con 0x7f3900061240 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b6d50 con 0x7f3900065ab0 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3904ef2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 0x7f3900099160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3904ef2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 0x7f3900099160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58298/0 (socket says 172.21.15.185:58298) 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51744/0 (socket says 172.21.15.185:51744) 2020-11-16T22:16:05.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3904ef2700 1 -- 172.21.15.185:0/3725119837 learned_addr learned my addr 172.21.15.185:0/3725119837 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f39056f3700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39001476d0 0x7f3900142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 -- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 msgr2=0x7f3900099160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 0x7f3900099160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 -- 172.21.15.185:0/3725119837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39001476d0 msgr2=0x7f3900142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39001476d0 0x7f3900142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 -- 172.21.15.185:0/3725119837 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900099770 con 0x7f3900065ab0 2020-11-16T22:16:05.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fffff700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f38f4006fd0 tx=0x7f38f4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011c80 con 0x7f3900065ab0 2020-11-16T22:16:05.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4011de0 con 0x7f3900065ab0 2020-11-16T22:16:05.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f401e670 con 0x7f3900065ab0 2020-11-16T22:16:05.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000b6ff0 con 0x7f3900065ab0 2020-11-16T22:16:05.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.359+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3900098170 con 0x7f3900065ab0 2020-11-16T22:16:05.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390005d4e0 con 0x7f3900065ab0 2020-11-16T22:16:05.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f38f401a040 con 0x7f3900065ab0 2020-11-16T22:16:05.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f38fcff9700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 0x7f38d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:05.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f38f406f100 con 0x7f3900065ab0 2020-11-16T22:16:05.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f3904ef2700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 0x7f38d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:05.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.363+0000 7f3904ef2700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 0x7f38d8057520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f38f0006a90 tx=0x7f38f0006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:05.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.367+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f38f4045300 con 0x7f3900065ab0 2020-11-16T22:16:05.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:05.571+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3900063850 con 0x7f3900065ab0 2020-11-16T22:16:05.980 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:16:05.998+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:16:05.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.011+0000 7f38fcff9700 1 -- 172.21.15.185:0/3725119837 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38f4023070 con 0x7f3900065ab0 2020-11-16T22:16:05.998 INFO:tasks.ceph.mds.b.smithi096.stderr: -9719> 2020-11-16T22:15:15.070+0000 7f9813ff6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:16:06.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 msgr2=0x7f38d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 0x7f38d8057520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f38f0006a90 tx=0x7f38f0006df0).stop 2020-11-16T22:16:06.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 msgr2=0x7f39000b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f38f4006fd0 tx=0x7f38f4015000).stop 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 shutdown_connections 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f38d8055060 0x7f38d8057520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3900061240 0x7f3900099160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39001476d0 0x7f3900142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 --2- 172.21.15.185:0/3725119837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3900065ab0 0x7f39000b8140 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 >> 172.21.15.185:0/3725119837 conn(0x7f3900060130 msgr2=0x7f390009df50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:06.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 shutdown_connections 2020-11-16T22:16:06.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.015+0000 7f3907156700 1 -- 172.21.15.185:0/3725119837 wait complete. 2020-11-16T22:16:06.002 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:16:06.016 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:16:06.022 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:16:06.023 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:16:06.030 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:16:06.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.147+0000 7ff5cf59e700 1 Processor -- start 2020-11-16T22:16:06.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.147+0000 7ff5cf59e700 1 -- start start 2020-11-16T22:16:06.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff5d00b5d70 0x7ff5d00ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff5d0143470 0x7ff5d0143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d00b7290 con 0x7ff5d00b4470 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d00b4bf0 con 0x7ff5d0143470 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d00b74c0 con 0x7ff5d00b5d70 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff5d00b5d70 0x7ff5d00ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff5d00b5d70 0x7ff5d00ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58320/0 (socket says 172.21.15.185:58320) 2020-11-16T22:16:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 -- 172.21.15.185:0/1419038943 learned_addr learned my addr 172.21.15.185:0/1419038943 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ced9d700 1 --2- 172.21.15.185:0/1419038943 >> v2:172.21.15.96:3300/0 conn(0x7ff5d0143470 0x7ff5d0143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 -- 172.21.15.185:0/1419038943 >> v2:172.21.15.96:3300/0 conn(0x7ff5d0143470 msgr2=0x7ff5d0143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 --2- 172.21.15.185:0/1419038943 >> v2:172.21.15.96:3300/0 conn(0x7ff5d0143470 0x7ff5d0143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 -- 172.21.15.185:0/1419038943 >> v1:172.21.15.96:6789/0 conn(0x7ff5d00b4470 legacy=0x7ff5d00b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 -- 172.21.15.185:0/1419038943 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d00b7760 con 0x7ff5d00b5d70 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5ce59c700 1 --2- 172.21.15.185:0/1419038943 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ba800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff5d005c5f0 tx=0x7ff5c4006eb0).ready entity=mon.1 client_cookie=d81116c0b398c4e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cd59a700 1 -- 172.21.15.185:0/1419038943 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4009810 con 0x7ff5d00b5d70 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cd59a700 1 -- 172.21.15.185:0/1419038943 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4009970 con 0x7ff5d00b5d70 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cd59a700 1 -- 172.21.15.185:0/1419038943 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4009cf0 con 0x7ff5d00b5d70 2020-11-16T22:16:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- 172.21.15.185:0/1419038943 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 msgr2=0x7ff5d00ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/1419038943 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ba800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff5d005c5f0 tx=0x7ff5c4006eb0).stop 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- 172.21.15.185:0/1419038943 shutdown_connections 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/1419038943 >> v2:172.21.15.96:3300/0 conn(0x7ff5d0143470 0x7ff5d0143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/1419038943 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ba800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- 172.21.15.185:0/1419038943 >> 172.21.15.185:0/1419038943 conn(0x7ff5d005ba80 msgr2=0x7ff5d005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- 172.21.15.185:0/1419038943 shutdown_connections 2020-11-16T22:16:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.151+0000 7ff5cf59e700 1 -- 172.21.15.185:0/1419038943 wait complete. 2020-11-16T22:16:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 Processor -- start 2020-11-16T22:16:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- start start 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 0x7ff5d0141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d0143470 0x7ff5d00aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00ad6b0 con 0x7ff5d0143470 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00ad950 con 0x7ff5d00b5d70 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00adbf0 con 0x7ff5d00b4470 2020-11-16T22:16:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5ce59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 0x7ff5d0141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58326/0 (socket says 172.21.15.185:58326) 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5ce59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 0x7ff5d0141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51768/0 (socket says 172.21.15.185:51768) 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 -- 172.21.15.185:0/427741079 learned_addr learned my addr 172.21.15.185:0/427741079 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5ced9d700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d0143470 0x7ff5d00aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 -- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 msgr2=0x7ff5d0141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 0x7ff5d0141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 -- 172.21.15.185:0/427741079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d0143470 msgr2=0x7ff5d00aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d0143470 0x7ff5d00aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 -- 172.21.15.185:0/427741079 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0141980 con 0x7ff5d00b5d70 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cdd9b700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff5c000c710 tx=0x7ff5c0007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000e490 con 0x7ff5d00b5d70 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c000e5f0 con 0x7ff5d00b5d70 2020-11-16T22:16:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d00afd30 con 0x7ff5d00b5d70 2020-11-16T22:16:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000e990 con 0x7ff5d00b5d70 2020-11-16T22:16:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.155+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d0154120 con 0x7ff5d00b5d70 2020-11-16T22:16:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d00addb0 con 0x7ff5d00b5d70 2020-11-16T22:16:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff5c0019030 con 0x7ff5d00b5d70 2020-11-16T22:16:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5be7fc700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 0x7ff5a4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6331+0+0 (secure 0 0 0) 0x7ff5c006d230 con 0x7ff5d00b5d70 2020-11-16T22:16:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5ce59c700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 0x7ff5a4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.159+0000 7ff5ce59c700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 0x7ff5a4057520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff5c4006ef0 tx=0x7ff5c4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:06.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.163+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff5c0043390 con 0x7ff5d00b5d70 2020-11-16T22:16:06.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:06.367+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5d005d200 con 0x7ff5d00b5d70 2020-11-16T22:16:06.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.011+0000 7ff5be7fc700 1 -- 172.21.15.185:0/427741079 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5c00170a0 con 0x7ff5d00b5d70 2020-11-16T22:16:07.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 msgr2=0x7ff5a4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:07.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 0x7ff5a4057520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff5c4006ef0 tx=0x7ff5c4015000).stop 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 msgr2=0x7ff5d00ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7ff5c000c710 tx=0x7ff5c0007b80).stop 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 shutdown_connections 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff5a4055060 0x7ff5a4057520 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d0143470 0x7ff5d00aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff5d00b4470 0x7ff5d0141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 --2- 172.21.15.185:0/427741079 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff5d00b5d70 0x7ff5d00ae800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 >> 172.21.15.185:0/427741079 conn(0x7ff5d005ba80 msgr2=0x7ff5d0143920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:07.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 shutdown_connections 2020-11-16T22:16:07.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.015+0000 7ff5cf59e700 1 -- 172.21.15.185:0/427741079 wait complete. 2020-11-16T22:16:07.020 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:16:07.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 Processor -- start 2020-11-16T22:16:07.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 -- start start 2020-11-16T22:16:07.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f51dc0b7550 0x7f51dc148160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51dc0acee0 0x7f51dc0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc0ad370 con 0x7f51dc0b7170 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc144f80 con 0x7f51dc0b7550 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e12ca700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc144170 con 0x7f51dc0acee0 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e0ac9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51dc0acee0 0x7f51dc0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e0ac9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51dc0acee0 0x7f51dc0b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58336/0 (socket says 172.21.15.185:58336) 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51e0ac9700 1 -- 172.21.15.185:0/3668852453 learned_addr learned my addr 172.21.15.185:0/3668852453 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:07.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.215+0000 7f51db7fe700 1 --2- 172.21.15.185:0/3668852453 >> v2:172.21.15.96:3300/0 conn(0x7f51dc0b7550 0x7f51dc148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e0ac9700 1 -- 172.21.15.185:0/3668852453 >> v2:172.21.15.96:3300/0 conn(0x7f51dc0b7550 msgr2=0x7f51dc148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:07.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e0ac9700 1 --2- 172.21.15.185:0/3668852453 >> v2:172.21.15.96:3300/0 conn(0x7f51dc0b7550 0x7f51dc148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e0ac9700 1 -- 172.21.15.185:0/3668852453 >> v1:172.21.15.185:6789/0 conn(0x7f51dc0b7170 legacy=0x7f51dc144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:07.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e0ac9700 1 -- 172.21.15.185:0/3668852453 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0b6ed0 con 0x7f51dc0acee0 2020-11-16T22:16:07.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e0ac9700 1 --2- 172.21.15.185:0/3668852453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0acee0 0x7f51dc0b64b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f51dc141bf0 tx=0x7f51d4009290).ready entity=mon.1 client_cookie=13325cd852158c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:07.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51daffd700 1 -- 172.21.15.185:0/3668852453 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400bbc0 con 0x7f51dc0acee0 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51daffd700 1 -- 172.21.15.185:0/3668852453 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400bd20 con 0x7f51dc0acee0 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51daffd700 1 -- 172.21.15.185:0/3668852453 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4011610 con 0x7f51dc0acee0 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- 172.21.15.185:0/3668852453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0acee0 msgr2=0x7f51dc0b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 --2- 172.21.15.185:0/3668852453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0acee0 0x7f51dc0b64b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f51dc141bf0 tx=0x7f51d4009290).stop 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- 172.21.15.185:0/3668852453 shutdown_connections 2020-11-16T22:16:07.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 --2- 172.21.15.185:0/3668852453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0acee0 0x7f51dc0b64b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 --2- 172.21.15.185:0/3668852453 >> v2:172.21.15.96:3300/0 conn(0x7f51dc0b7550 0x7f51dc148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- 172.21.15.185:0/3668852453 >> 172.21.15.185:0/3668852453 conn(0x7f51dc067860 msgr2=0x7f51dc067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- 172.21.15.185:0/3668852453 shutdown_connections 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- 172.21.15.185:0/3668852453 wait complete. 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 Processor -- start 2020-11-16T22:16:07.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.219+0000 7f51e12ca700 1 -- start start 2020-11-16T22:16:07.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 0x7f51dc147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0b7550 0x7f51dc13ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc13e8e0 con 0x7f51dc0acee0 2020-11-16T22:16:07.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc147670 con 0x7f51dc0b7550 2020-11-16T22:16:07.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc147900 con 0x7f51dc0b7170 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e0ac9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0b7550 0x7f51dc13ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51784/0 (socket says 172.21.15.185:51784) 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 -- 172.21.15.185:0/1220387126 learned_addr learned my addr 172.21.15.185:0/1220387126 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51dbfff700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 0x7f51dc147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 -- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0b7550 msgr2=0x7f51dc13ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:07.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0b7550 0x7f51dc13ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 -- 172.21.15.185:0/1220387126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 msgr2=0x7f51dc147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 0x7f51dc147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 -- 172.21.15.185:0/1220387126 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc13fc00 con 0x7f51dc0b7170 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51dbfff700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 0x7f51dc147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51db7fe700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f51c400d910 tx=0x7f51c4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:07.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4009640 con 0x7f51dc0b7170 2020-11-16T22:16:07.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40097a0 con 0x7f51dc0b7170 2020-11-16T22:16:07.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc13fea0 con 0x7f51dc0b7170 2020-11-16T22:16:07.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4009b40 con 0x7f51dc0b7170 2020-11-16T22:16:07.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc146bb0 con 0x7f51dc0b7170 2020-11-16T22:16:07.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc05d200 con 0x7f51dc0b7170 2020-11-16T22:16:07.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51c401c460 con 0x7f51dc0b7170 2020-11-16T22:16:07.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 0x7f51b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:07.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.223+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f51c406dea0 con 0x7f51dc0b7170 2020-11-16T22:16:07.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.227+0000 7f51dbfff700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 0x7f51b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:07.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.227+0000 7f51dbfff700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 0x7f51b8057520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f51cc007f20 tx=0x7f51cc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:07.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.231+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f51c4043fa0 con 0x7f51dc0b7170 2020-11-16T22:16:07.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:07.387+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51dc0b7a50 con 0x7f51dc0b7170 2020-11-16T22:16:08.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51d3fff700 1 -- 172.21.15.185:0/1220387126 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7f51c40475c0 con 0x7f51dc0b7170 2020-11-16T22:16:08.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 msgr2=0x7f51b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 0x7f51b8057520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f51cc007f20 tx=0x7f51cc009040).stop 2020-11-16T22:16:08.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 msgr2=0x7f51dc145e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f51c400d910 tx=0x7f51c4008040).stop 2020-11-16T22:16:08.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 shutdown_connections 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51b8055060 0x7f51b8057520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51dc0b7550 0x7f51dc13ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51dc0acee0 0x7f51dc147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 --2- 172.21.15.185:0/1220387126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51dc0b7170 0x7f51dc145e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 >> 172.21.15.185:0/1220387126 conn(0x7f51dc067860 msgr2=0x7f51dc0b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 shutdown_connections 2020-11-16T22:16:08.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.039+0000 7f51e12ca700 1 -- 172.21.15.185:0/1220387126 wait complete. 2020-11-16T22:16:08.028 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:16:08.043 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:16:08.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.191+0000 7f2c9759e700 1 Processor -- start 2020-11-16T22:16:08.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.191+0000 7f2c9759e700 1 -- start start 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98144a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c980b7570 0x7f2c98143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c980b7cb0 con 0x7f2c98143810 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c980b7a10 con 0x7f2c980b7570 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98067470 con 0x7f2c980602d0 2020-11-16T22:16:08.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c980b7570 0x7f2c98143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c96d9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2c98143810 0x7f2c98067bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51788/0 (socket says 172.21.15.185:51788) 2020-11-16T22:16:08.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2c980b7570 0x7f2c98143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58346/0 (socket says 172.21.15.185:58346) 2020-11-16T22:16:08.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c96d9d700 1 -- 172.21.15.185:0/1367735472 learned_addr learned my addr 172.21.15.185:0/1367735472 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:08.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98144a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51792/0 (socket says 172.21.15.185:51792) 2020-11-16T22:16:08.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 -- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 msgr2=0x7f2c98144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:08.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 --2- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 -- 172.21.15.185:0/1367735472 >> v1:172.21.15.185:6789/0 conn(0x7f2c98143810 legacy=0x7f2c98067bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9559a700 1 -- 172.21.15.185:0/1367735472 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793240902 0 0) 0x7f2c980b7cb0 con 0x7f2c98143810 2020-11-16T22:16:08.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 -- 172.21.15.185:0/1367735472 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98067710 con 0x7f2c980b7570 2020-11-16T22:16:08.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c95d9b700 1 --2- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98143320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f2c8000b9a0 tx=0x7f2c80006df0).ready entity=mon.1 client_cookie=324bd707d8f5f734 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9559a700 1 -- 172.21.15.185:0/1367735472 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800097b0 con 0x7f2c980b7570 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9559a700 1 -- 172.21.15.185:0/1367735472 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80009910 con 0x7f2c980b7570 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9559a700 1 -- 172.21.15.185:0/1367735472 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80009c90 con 0x7f2c980b7570 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 msgr2=0x7f2c98143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98143320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f2c8000b9a0 tx=0x7f2c80006df0).stop 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- 172.21.15.185:0/1367735472 shutdown_connections 2020-11-16T22:16:08.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1367735472 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98143320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- 172.21.15.185:0/1367735472 >> 172.21.15.185:0/1367735472 conn(0x7f2c9805ba80 msgr2=0x7f2c9805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- 172.21.15.185:0/1367735472 shutdown_connections 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- 172.21.15.185:0/1367735472 wait complete. 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 Processor -- start 2020-11-16T22:16:08.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.195+0000 7f2c9759e700 1 -- start start 2020-11-16T22:16:08.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98115820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c98143810 0x7f2c98142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98153d90 con 0x7f2c98143810 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98115d10 con 0x7f2c980b7570 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c980af140 con 0x7f2c980602d0 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c95d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98115820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51796/0 (socket says 172.21.15.185:51796) 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 -- 172.21.15.185:0/1570811071 learned_addr learned my addr 172.21.15.185:0/1570811071 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c96d9d700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c98143810 0x7f2c98142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 -- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 msgr2=0x7f2c98115820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98115820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 -- 172.21.15.185:0/1570811071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c98143810 msgr2=0x7f2c98142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:08.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c98143810 0x7f2c98142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 -- 172.21.15.185:0/1570811071 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c981479c0 con 0x7f2c980602d0 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c95d9b700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98115820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9659c700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f2c8800ebd0 tx=0x7f2c88006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800b3a0 con 0x7f2c980602d0 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8800c8e0 con 0x7f2c980602d0 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c98153f90 con 0x7f2c980602d0 2020-11-16T22:16:08.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800cca0 con 0x7f2c980602d0 2020-11-16T22:16:08.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c980ae8d0 con 0x7f2c980602d0 2020-11-16T22:16:08.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.199+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c980af280 con 0x7f2c980602d0 2020-11-16T22:16:08.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.203+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2c8801c460 con 0x7f2c980602d0 2020-11-16T22:16:08.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.203+0000 7f2c8e7fc700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 0x7f2c74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:08.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.203+0000 7f2c95d9b700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 0x7f2c74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:08.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.203+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f2c8806dfb0 con 0x7f2c980602d0 2020-11-16T22:16:08.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.203+0000 7f2c95d9b700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 0x7f2c74057520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2c980b85e0 tx=0x7f2c80015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:08.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.211+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2c88044260 con 0x7f2c980602d0 2020-11-16T22:16:08.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:08.363+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c980aebb0 con 0x7f2c980602d0 2020-11-16T22:16:09.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.047+0000 7f2c8e7fc700 1 -- 172.21.15.185:0/1570811071 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c88047880 con 0x7f2c980602d0 2020-11-16T22:16:09.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 msgr2=0x7f2c74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 0x7f2c74057520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2c980b85e0 tx=0x7f2c80015000).stop 2020-11-16T22:16:09.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 msgr2=0x7f2c98115330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f2c8800ebd0 tx=0x7f2c88006df0).stop 2020-11-16T22:16:09.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 shutdown_connections 2020-11-16T22:16:09.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c74055060 0x7f2c74057520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c98143810 0x7f2c98142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2c980602d0 0x7f2c98115330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 --2- 172.21.15.185:0/1570811071 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2c980b7570 0x7f2c98115820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 >> 172.21.15.185:0/1570811071 conn(0x7f2c9805ba80 msgr2=0x7f2c98143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 shutdown_connections 2020-11-16T22:16:09.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.051+0000 7f2c9759e700 1 -- 172.21.15.185:0/1570811071 wait complete. 2020-11-16T22:16:09.038 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:16:09.055 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:16:09.055 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:16:09.056 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:16:09.056 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:16:09.056 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:16:09.057 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:16:09.057 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:16:09.057 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:16:09.057 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:16:09.058 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:16:09.058 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:16:09.058 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:16:09.069 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:16:09.070 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:16:09.071 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:16:09.071 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:16:09.072 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:16:09.072 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:16:09.112 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:16:09.112 DEBUG:teuthology.parallel:result is None 2020-11-16T22:16:09.115 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:16:09.115 DEBUG:teuthology.parallel:result is None 2020-11-16T22:16:09.116 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:16:09.116 DEBUG:teuthology.parallel:result is None 2020-11-16T22:16:09.117 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:16:09.191 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:16:09.192 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:16:09.192 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:16:09.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 Processor -- start 2020-11-16T22:16:09.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- start start 2020-11-16T22:16:09.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b5b30 0x7f1a740b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a740b6b30 0x7f1a74143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a741444d0 con 0x7f1a7409d260 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a74144770 con 0x7f1a740b6b30 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74144a10 con 0x7f1a740b5b30 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a740b6b30 0x7f1a74143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b5b30 0x7f1a740b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1a740b6b30 0x7f1a74143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58376/0 (socket says 172.21.15.185:58376) 2020-11-16T22:16:09.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 -- 172.21.15.185:0/1849079912 learned_addr learned my addr 172.21.15.185:0/1849079912 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 -- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b5b30 msgr2=0x7f1a740b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 --2- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b5b30 0x7f1a740b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 -- 172.21.15.185:0/1849079912 >> v1:172.21.15.96:6789/0 conn(0x7f1a7409d260 legacy=0x7f1a741437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 -- 172.21.15.185:0/1849079912 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a74144cb0 con 0x7f1a740b6b30 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a73fff700 1 --2- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b6b30 0x7f1a74143f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1a6c006fd0 tx=0x7f1a6c006eb0).ready entity=mon.1 client_cookie=f41d7145a17d1b20 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:09.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a727fc700 1 -- 172.21.15.185:0/1849079912 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c009810 con 0x7f1a740b6b30 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a727fc700 1 -- 172.21.15.185:0/1849079912 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c009970 con 0x7f1a740b6b30 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a727fc700 1 -- 172.21.15.185:0/1849079912 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c009cf0 con 0x7f1a740b6b30 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b6b30 msgr2=0x7f1a74143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b6b30 0x7f1a74143f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1a6c006fd0 tx=0x7f1a6c006eb0).stop 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 -- 172.21.15.185:0/1849079912 shutdown_connections 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b6b30 0x7f1a74143f00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.259+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/1849079912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b5b30 0x7f1a740b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- 172.21.15.185:0/1849079912 >> 172.21.15.185:0/1849079912 conn(0x7f1a74068330 msgr2=0x7f1a740ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:09.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- 172.21.15.185:0/1849079912 shutdown_connections 2020-11-16T22:16:09.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- 172.21.15.185:0/1849079912 wait complete. 2020-11-16T22:16:09.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 Processor -- start 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- start start 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a740b5b30 0x7f1a740b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b2400 0x7f1a740b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741535a0 con 0x7f1a740b5b30 2020-11-16T22:16:09.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74153840 con 0x7f1a740b2400 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74153ae0 con 0x7f1a740b6b30 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51824/0 (socket says 172.21.15.185:51824) 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a740b5b30 0x7f1a740b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 -- 172.21.15.185:0/295261293 learned_addr learned my addr 172.21.15.185:0/295261293 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a73fff700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b2400 0x7f1a740b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 -- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b2400 msgr2=0x7f1a740b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b2400 0x7f1a740b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 -- 172.21.15.185:0/295261293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a740b5b30 msgr2=0x7f1a740b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a740b5b30 0x7f1a740b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 -- 172.21.15.185:0/295261293 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a74153d80 con 0x7f1a740b6b30 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a72ffd700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f1a5c00c710 tx=0x7f1a5c007ac0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c013410 con 0x7f1a740b6b30 2020-11-16T22:16:09.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c013570 con 0x7f1a740b6b30 2020-11-16T22:16:09.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.263+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a74153290 con 0x7f1a740b6b30 2020-11-16T22:16:09.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0138f0 con 0x7f1a740b6b30 2020-11-16T22:16:09.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a740b12e0 con 0x7f1a740b6b30 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7405d4e0 con 0x7f1a740b6b30 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1a5c010030 con 0x7f1a740b6b30 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a6b7fe700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a737fe700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f1a5c06d900 con 0x7f1a740b6b30 2020-11-16T22:16:09.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.267+0000 7f1a737fe700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1a640099f0 tx=0x7f1a640091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:09.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.275+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1a5c043c20 con 0x7f1a740b6b30 2020-11-16T22:16:09.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:09.439+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1a74060530 con 0x7f1a740b6b30 2020-11-16T22:16:10.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.055+0000 7f1a6b7fe700 1 -- 172.21.15.185:0/295261293 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a5c047240 con 0x7f1a740b6b30 2020-11-16T22:16:10.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 msgr2=0x7f1a50057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f1a640099f0 tx=0x7f1a640091d0).stop 2020-11-16T22:16:10.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 msgr2=0x7f1a740b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f1a5c00c710 tx=0x7f1a5c007ac0).stop 2020-11-16T22:16:10.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 shutdown_connections 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1a50055060 0x7f1a50057520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1a740b2400 0x7f1a740b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a740b5b30 0x7f1a740b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 --2- 172.21.15.185:0/295261293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1a740b6b30 0x7f1a740b3720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 >> 172.21.15.185:0/295261293 conn(0x7f1a74068330 msgr2=0x7f1a740b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 shutdown_connections 2020-11-16T22:16:10.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.059+0000 7f1a79d1f700 1 -- 172.21.15.185:0/295261293 wait complete. 2020-11-16T22:16:10.048 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:16:10.063 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:16:10.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.195+0000 7fc77ffff700 1 Processor -- start 2020-11-16T22:16:10.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.195+0000 7fc77ffff700 1 -- start start 2020-11-16T22:16:10.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc7800b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc780060f60 con 0x7fc780060130 2020-11-16T22:16:10.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7801420f0 con 0x7fc780060ab0 2020-11-16T22:16:10.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc780142390 con 0x7fc7800b7510 2020-11-16T22:16:10.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc780060ab0 0x7fc7800673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51826/0 (socket says 172.21.15.185:51826) 2020-11-16T22:16:10.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77f7fe700 1 -- 172.21.15.185:0/2761195033 learned_addr learned my addr 172.21.15.185:0/2761195033 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:10.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77effd700 1 --2- 172.21.15.185:0/2761195033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc7800b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77effd700 1 -- 172.21.15.185:0/2761195033 >> v1:172.21.15.185:6789/0 conn(0x7fc780060ab0 legacy=0x7fc7800673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77dffb700 1 -- 172.21.15.185:0/2761195033 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087839070 0 0) 0x7fc7801420f0 con 0x7fc780060ab0 2020-11-16T22:16:10.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77effd700 1 -- 172.21.15.185:0/2761195033 >> v1:172.21.15.96:6789/0 conn(0x7fc780060130 legacy=0x7fc78013ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77effd700 1 -- 172.21.15.185:0/2761195033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780142630 con 0x7fc7800b7510 2020-11-16T22:16:10.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77effd700 1 --2- 172.21.15.185:0/2761195033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc7800b78f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fc77400b9d0 tx=0x7fc774006df0).ready entity=mon.2 client_cookie=93fa2da6cbb00a9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77dffb700 1 -- 172.21.15.185:0/2761195033 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774009870 con 0x7fc7800b7510 2020-11-16T22:16:10.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77dffb700 1 -- 172.21.15.185:0/2761195033 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7740099d0 con 0x7fc7800b7510 2020-11-16T22:16:10.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77dffb700 1 -- 172.21.15.185:0/2761195033 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774009d50 con 0x7fc7800b7510 2020-11-16T22:16:10.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- 172.21.15.185:0/2761195033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 msgr2=0x7fc7800b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 --2- 172.21.15.185:0/2761195033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc7800b78f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fc77400b9d0 tx=0x7fc774006df0).stop 2020-11-16T22:16:10.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- 172.21.15.185:0/2761195033 shutdown_connections 2020-11-16T22:16:10.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 --2- 172.21.15.185:0/2761195033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc7800b78f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- 172.21.15.185:0/2761195033 >> 172.21.15.185:0/2761195033 conn(0x7fc78005ba80 msgr2=0x7fc78005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- 172.21.15.185:0/2761195033 shutdown_connections 2020-11-16T22:16:10.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.199+0000 7fc77ffff700 1 -- 172.21.15.185:0/2761195033 wait complete. 2020-11-16T22:16:10.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 Processor -- start 2020-11-16T22:16:10.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- start start 2020-11-16T22:16:10.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc780060130 0x7fc7800ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 0x7fc780145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780144760 con 0x7fc780060ab0 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800ad690 con 0x7fc780060130 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78013f180 con 0x7fc7800b7510 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc780060130 0x7fc7800ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51838/0 (socket says 172.21.15.185:51838) 2020-11-16T22:16:10.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 -- 172.21.15.185:0/615199184 learned_addr learned my addr 172.21.15.185:0/615199184 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:10.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77e7fc700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 0x7fc780145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 -- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc780060130 msgr2=0x7fc7800ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc780060130 0x7fc7800ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 -- 172.21.15.185:0/615199184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 msgr2=0x7fc780145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 0x7fc780145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 -- 172.21.15.185:0/615199184 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780144ab0 con 0x7fc7800b7510 2020-11-16T22:16:10.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77e7fc700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 0x7fc780145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77f7fe700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fc77800cae0 tx=0x7fc778007ce0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778005b80 con 0x7fc7800b7510 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778005ce0 con 0x7fc7800b7510 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780145780 con 0x7fc7800b7510 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778007360 con 0x7fc7800b7510 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.203+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780144d50 con 0x7fc7800b7510 2020-11-16T22:16:10.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78005d200 con 0x7fc7800b7510 2020-11-16T22:16:10.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc778010030 con 0x7fc7800b7510 2020-11-16T22:16:10.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc766ffd700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 0x7fc75c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc77806da90 con 0x7fc7800b7510 2020-11-16T22:16:10.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc77effd700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 0x7fc75c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.207+0000 7fc77effd700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 0x7fc75c057520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc77400b920 tx=0x7fc774015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.215+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc778043fc0 con 0x7fc7800b7510 2020-11-16T22:16:10.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.419+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc7800b3970 con 0x7fc7800b7510 2020-11-16T22:16:10.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.419+0000 7fc766ffd700 1 -- 172.21.15.185:0/615199184 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+4274 (secure 0 0 0) 0x7fc7780475e0 con 0x7fc7800b7510 2020-11-16T22:16:10.405 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:16:10.406 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.407 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.408 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.409 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.410 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.411 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.412 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.413 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.414 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:16:10.415 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.416 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.417 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.418 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.419 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:16:10.420 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow r", 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.421 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:16:10.422 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:16:10.423 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:16:10.424 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:16:10.425 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:10.426 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:16:10.427 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:16:10.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 msgr2=0x7fc75c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 0x7fc75c057520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fc77400b920 tx=0x7fc774015000).stop 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 msgr2=0x7fc78013c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fc77800cae0 tx=0x7fc778007ce0).stop 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 shutdown_connections 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc75c055060 0x7fc75c057520 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc780060130 0x7fc7800ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7800b7510 0x7fc78013c560 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 --2- 172.21.15.185:0/615199184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc780060ab0 0x7fc780145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 >> 172.21.15.185:0/615199184 conn(0x7fc78005ba80 msgr2=0x7fc7800b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 shutdown_connections 2020-11-16T22:16:10.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.423+0000 7fc77ffff700 1 -- 172.21.15.185:0/615199184 wait complete. 2020-11-16T22:16:10.430 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:10.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 Processor -- start 2020-11-16T22:16:10.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 -- start start 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040af130 con 0x7f9d04141b30 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040af600 con 0x7f9d040b8820 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040af360 con 0x7f9d040b30e0 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51848/0 (socket says 172.21.15.185:51848) 2020-11-16T22:16:10.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0259c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f9d040b8820 0x7f9d04141420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51840/0 (socket says 172.21.15.185:51840) 2020-11-16T22:16:10.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 -- 172.21.15.185:0/1752282549 learned_addr learned my addr 172.21.15.185:0/1752282549 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:10.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 -- 172.21.15.185:0/1752282549 >> v1:172.21.15.185:6789/0 conn(0x7f9d040b8820 legacy=0x7f9d04141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 -- 172.21.15.185:0/1752282549 >> v1:172.21.15.96:6789/0 conn(0x7f9d04141b30 legacy=0x7f9d041454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 -- 172.21.15.185:0/1752282549 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040b3dc0 con 0x7f9d040b30e0 2020-11-16T22:16:10.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d02d9d700 1 --2- 172.21.15.185:0/1752282549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9cfc00ba20 tx=0x7f9cfc006eb0).ready entity=mon.2 client_cookie=799231517380bb01 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0159a700 1 -- 172.21.15.185:0/1752282549 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0098d0 con 0x7f9d040b30e0 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0159a700 1 -- 172.21.15.185:0/1752282549 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc009a30 con 0x7f9d040b30e0 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.563+0000 7f9d0159a700 1 -- 172.21.15.185:0/1752282549 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc009db0 con 0x7f9d040b30e0 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/1752282549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 msgr2=0x7f9d040af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 --2- 172.21.15.185:0/1752282549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9cfc00ba20 tx=0x7f9cfc006eb0).stop 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/1752282549 shutdown_connections 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 --2- 172.21.15.185:0/1752282549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d040b30e0 0x7f9d040af7e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/1752282549 >> 172.21.15.185:0/1752282549 conn(0x7f9d040645d0 msgr2=0x7f9d040649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/1752282549 shutdown_connections 2020-11-16T22:16:10.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/1752282549 wait complete. 2020-11-16T22:16:10.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 Processor -- start 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- start start 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 0x7f9d0413bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 0x7f9d0413fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 0x7f9d04156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0413c3e0 con 0x7f9d040b30e0 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0413ea70 con 0x7f9d040b8820 2020-11-16T22:16:10.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0413ed10 con 0x7f9d0413f450 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 0x7f9d04156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d01d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 0x7f9d0413fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d01d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 0x7f9d0413fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58408/0 (socket says 172.21.15.185:58408) 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d01d9b700 1 -- 172.21.15.185:0/143695411 learned_addr learned my addr 172.21.15.185:0/143695411 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0259c700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 0x7f9d0413bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 -- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 msgr2=0x7f9d0413fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 0x7f9d0413fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 -- 172.21.15.185:0/143695411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 msgr2=0x7f9d0413bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 0x7f9d0413bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 -- 172.21.15.185:0/143695411 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0413efb0 con 0x7f9d0413f450 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0259c700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 0x7f9d0413bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d02d9d700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 0x7f9d04156b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9cfc00ba20 tx=0x7f9cfc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc021070 con 0x7f9d0413f450 2020-11-16T22:16:10.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04140310 con 0x7f9d0413f450 2020-11-16T22:16:10.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc006b20 con 0x7f9d0413f450 2020-11-16T22:16:10.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01c810 con 0x7f9d0413f450 2020-11-16T22:16:10.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.567+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04143f30 con 0x7f9d0413f450 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0413f190 con 0x7f9d0413f450 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9cfc00f070 con 0x7f9d0413f450 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9cfa7fc700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 0x7f9ce0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9cfc06f860 con 0x7f9d0413f450 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9d0259c700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 0x7f9ce0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.571+0000 7f9d0259c700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 0x7f9ce0057520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9cf4007b80 tx=0x7f9cf4007090).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.575+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9d0413f190 con 0x7f9d0413f450 2020-11-16T22:16:10.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.731+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d0413f190 con 0x7f9d0413f450 2020-11-16T22:16:10.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9cfa7fc700 1 -- 172.21.15.185:0/143695411 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13675 (secure 0 0 0) 0x7f9d0413f190 con 0x7f9d0413f450 2020-11-16T22:16:10.719 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:10.719 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":36,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:10.047979+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 msgr2=0x7f9ce0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 0x7f9ce0057520 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9cf4007b80 tx=0x7f9cf4007090).stop 2020-11-16T22:16:10.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 msgr2=0x7f9d04156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 0x7f9d04156b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9cfc00ba20 tx=0x7f9cfc015000).stop 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 shutdown_connections 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9ce0055060 0x7f9ce0057520 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d0413f450 0x7f9d04156b90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d040b30e0 0x7f9d0413bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.735+0000 7f9d0359e700 1 --2- 172.21.15.185:0/143695411 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d040b8820 0x7f9d0413fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.739+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 >> 172.21.15.185:0/143695411 conn(0x7f9d040645d0 msgr2=0x7f9d0413e080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.739+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 shutdown_connections 2020-11-16T22:16:10.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.739+0000 7f9d0359e700 1 -- 172.21.15.185:0/143695411 wait complete. 2020-11-16T22:16:10.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:16:10.739 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:16:10.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 Processor -- start 2020-11-16T22:16:10.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 -- start start 2020-11-16T22:16:10.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7f34068180 0x7f7f34065b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f340ac600 0x7f7f3413f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f34067130 con 0x7f7f340657a0 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f34065300 con 0x7f7f34068180 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f34068640 con 0x7f7f340ac600 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3902a700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7f340657a0 0x7f7f340abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51852/0 (socket says 172.21.15.185:51852) 2020-11-16T22:16:10.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3902a700 1 -- 172.21.15.185:0/3026893132 learned_addr learned my addr 172.21.15.185:0/3026893132 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:10.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 --2- 172.21.15.185:0/3026893132 >> v2:172.21.15.185:3300/0 conn(0x7f7f340ac600 0x7f7f3413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f38829700 1 --2- 172.21.15.185:0/3026893132 >> v2:172.21.15.96:3300/0 conn(0x7f7f34068180 0x7f7f34065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 -- 172.21.15.185:0/3026893132 >> v2:172.21.15.96:3300/0 conn(0x7f7f34068180 msgr2=0x7f7f34065b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:10.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 --2- 172.21.15.185:0/3026893132 >> v2:172.21.15.96:3300/0 conn(0x7f7f34068180 0x7f7f34065b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 -- 172.21.15.185:0/3026893132 >> v1:172.21.15.185:6789/0 conn(0x7f7f340657a0 legacy=0x7f7f340abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f23fff700 1 -- 172.21.15.185:0/3026893132 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165821126 0 0) 0x7f7f34067130 con 0x7f7f340657a0 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 -- 172.21.15.185:0/3026893132 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34067360 con 0x7f7f340ac600 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3982b700 1 --2- 172.21.15.185:0/3026893132 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340ac600 0x7f7f3413f310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7f34066e60 tx=0x7f7f2c0091d0).ready entity=mon.1 client_cookie=bf059ca016c83907 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f23fff700 1 -- 172.21.15.185:0/3026893132 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c00ba70 con 0x7f7f340ac600 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f23fff700 1 -- 172.21.15.185:0/3026893132 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c00bbd0 con 0x7f7f340ac600 2020-11-16T22:16:10.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f23fff700 1 -- 172.21.15.185:0/3026893132 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c011400 con 0x7f7f340ac600 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3026893132 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340ac600 msgr2=0x7f7f3413f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.879+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3026893132 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340ac600 0x7f7f3413f310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7f34066e60 tx=0x7f7f2c0091d0).stop 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3026893132 shutdown_connections 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3026893132 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340ac600 0x7f7f3413f310 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3026893132 >> v2:172.21.15.96:3300/0 conn(0x7f7f34068180 0x7f7f34065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3026893132 >> 172.21.15.185:0/3026893132 conn(0x7f7f34060130 msgr2=0x7f7f34060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:10.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3026893132 shutdown_connections 2020-11-16T22:16:10.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3026893132 wait complete. 2020-11-16T22:16:10.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 Processor -- start 2020-11-16T22:16:10.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- start start 2020-11-16T22:16:10.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 0x7f7f341405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 0x7f7f340ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34142240 con 0x7f7f340ac600 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341424e0 con 0x7f7f340657a0 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3413e560 con 0x7f7f34068180 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f38829700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 0x7f7f341405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f38829700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51864/0 (socket says 172.21.15.185:51864) 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f38829700 1 -- 172.21.15.185:0/3744315114 learned_addr learned my addr 172.21.15.185:0/3744315114 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3982b700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 0x7f7f340ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 -- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 msgr2=0x7f7f34139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 -- 172.21.15.185:0/3744315114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 msgr2=0x7f7f340ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:10.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 0x7f7f340ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:10.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 -- 172.21.15.185:0/3744315114 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34140ac0 con 0x7f7f340657a0 2020-11-16T22:16:10.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3982b700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 0x7f7f340ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:10.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f38829700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:10.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3902a700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 0x7f7f341405d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f7f30007fd0 tx=0x7f7f30007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300072e0 con 0x7f7f340657a0 2020-11-16T22:16:10.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f30007440 con 0x7f7f340657a0 2020-11-16T22:16:10.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3413bd80 con 0x7f7f340657a0 2020-11-16T22:16:10.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30005820 con 0x7f7f340657a0 2020-11-16T22:16:10.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.883+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340b03a0 con 0x7f7f340657a0 2020-11-16T22:16:10.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f341426e0 con 0x7f7f340657a0 2020-11-16T22:16:10.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7f30010070 con 0x7f7f340657a0 2020-11-16T22:16:10.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f217fa700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 0x7f7f0c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:10.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7f3006d570 con 0x7f7f340657a0 2020-11-16T22:16:10.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f38829700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 0x7f7f0c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:10.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.887+0000 7f7f38829700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 0x7f7f0c057520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f7f24006a90 tx=0x7f7f24006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:10.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:10.891+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7f30043b80 con 0x7f7f340657a0 2020-11-16T22:16:11.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.039+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7f3405d200 con 0x7f7f340657a0 2020-11-16T22:16:11.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.039+0000 7f7f217fa700 1 -- 172.21.15.185:0/3744315114 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v36) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7f300471a0 con 0x7f7f340657a0 2020-11-16T22:16:11.026 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:16:11.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 msgr2=0x7f7f0c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 0x7f7f0c057520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f7f24006a90 tx=0x7f7f24006df0).stop 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 msgr2=0x7f7f341405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 0x7f7f341405d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f7f30007fd0 tx=0x7f7f30007a00).stop 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 shutdown_connections 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f0c055060 0x7f7f0c057520 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f340657a0 0x7f7f341405d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f340ac600 0x7f7f340ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 --2- 172.21.15.185:0/3744315114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f34068180 0x7f7f34139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 >> 172.21.15.185:0/3744315114 conn(0x7f7f34060130 msgr2=0x7f7f34146430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 shutdown_connections 2020-11-16T22:16:11.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.043+0000 7f7f3a02c700 1 -- 172.21.15.185:0/3744315114 wait complete. 2020-11-16T22:16:11.045 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:16:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 Processor -- start 2020-11-16T22:16:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 -- start start 2020-11-16T22:16:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05ac0b3570 0x7f05ac143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0b7c20 con 0x7f05ac13de10 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0b3a80 con 0x7f05ac0b3570 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b31ff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b3d20 con 0x7f05ac13e1f0 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b19fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05ac0b3570 0x7f05ac143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05ac0b3570 0x7f05ac143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58432/0 (socket says 172.21.15.185:58432) 2020-11-16T22:16:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b19fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51874/0 (socket says 172.21.15.185:51874) 2020-11-16T22:16:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 -- 172.21.15.185:0/2125500818 learned_addr learned my addr 172.21.15.185:0/2125500818 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b21fd700 1 --1- 172.21.15.185:0/2125500818 >> v1:172.21.15.185:6789/0 conn(0x7f05ac13de10 0x7f05ac0b7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51866/0 (socket says 172.21.15.185:51866) 2020-11-16T22:16:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 -- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 msgr2=0x7f05ac0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 -- 172.21.15.185:0/2125500818 >> v1:172.21.15.185:6789/0 conn(0x7f05ac13de10 legacy=0x7f05ac0b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b11fb700 1 -- 172.21.15.185:0/2125500818 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859826161 0 0) 0x7f05ac0b7c20 con 0x7f05ac13de10 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 -- 172.21.15.185:0/2125500818 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0b3fc0 con 0x7f05ac0b3570 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b19fc700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.187+0000 7f05b29fe700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac0b3570 0x7f05ac143f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f05a4009d50 tx=0x7f05a4009290).ready entity=mon.1 client_cookie=227672c812fa2c76 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b11fb700 1 -- 172.21.15.185:0/2125500818 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400bbc0 con 0x7f05ac0b3570 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b11fb700 1 -- 172.21.15.185:0/2125500818 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400bd20 con 0x7f05ac0b3570 2020-11-16T22:16:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b11fb700 1 -- 172.21.15.185:0/2125500818 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4011610 con 0x7f05ac0b3570 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac0b3570 msgr2=0x7f05ac143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac0b3570 0x7f05ac143f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f05a4009d50 tx=0x7f05a4009290).stop 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- 172.21.15.185:0/2125500818 shutdown_connections 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac0b3570 0x7f05ac143f50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- 172.21.15.185:0/2125500818 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac13e1f0 0x7f05ac0666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- 172.21.15.185:0/2125500818 >> 172.21.15.185:0/2125500818 conn(0x7f05ac064b90 msgr2=0x7f05ac063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- 172.21.15.185:0/2125500818 shutdown_connections 2020-11-16T22:16:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- 172.21.15.185:0/2125500818 wait complete. 2020-11-16T22:16:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 Processor -- start 2020-11-16T22:16:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- start start 2020-11-16T22:16:11.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05ac0b3570 0x7f05ac066850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 0x7f05ac0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b0f10 con 0x7f05ac0b3570 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac146c30 con 0x7f05ac13de10 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b31ff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b1140 con 0x7f05ac148110 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b29fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 0x7f05ac0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b29fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51880/0 (socket says 172.21.15.185:51880) 2020-11-16T22:16:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b29fe700 1 -- 172.21.15.185:0/164533440 learned_addr learned my addr 172.21.15.185:0/164533440 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b21fd700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05ac0b3570 0x7f05ac066850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 -- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 msgr2=0x7f05ac0b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 -- 172.21.15.185:0/164533440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05ac0b3570 msgr2=0x7f05ac066850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05ac0b3570 0x7f05ac066850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b19fc700 1 -- 172.21.15.185:0/164533440 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0b6c00 con 0x7f05ac13de10 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.191+0000 7f05b29fe700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b19fc700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 0x7f05ac0b0170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f05a000ecf0 tx=0x7f05a0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000b500 con 0x7f05ac13de10 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a000ca10 con 0x7f05ac13de10 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac066d40 con 0x7f05ac13de10 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000cd90 con 0x7f05ac13de10 2020-11-16T22:16:11.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac147970 con 0x7f05ac13de10 2020-11-16T22:16:11.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac05d200 con 0x7f05ac13de10 2020-11-16T22:16:11.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f05a0012600 con 0x7f05ac13de10 2020-11-16T22:16:11.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 0x7f0584057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f05a006deb0 con 0x7f05ac13de10 2020-11-16T22:16:11.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b21fd700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 0x7f0584057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.195+0000 7f05b21fd700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 0x7f0584057520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f05a80011a0 tx=0x7f05a8007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.203+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f05a00443e0 con 0x7f05ac13de10 2020-11-16T22:16:11.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.359+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05ac0b1300 con 0x7f05ac13de10 2020-11-16T22:16:11.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.359+0000 7f059e7fc700 1 -- 172.21.15.185:0/164533440 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05a0047a00 con 0x7f05ac13de10 2020-11-16T22:16:11.345 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 msgr2=0x7f0584057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 0x7f0584057520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f05a80011a0 tx=0x7f05a8007a20).stop 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 msgr2=0x7f05ac0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 0x7f05ac0b0170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f05a000ecf0 tx=0x7f05a0006df0).stop 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 shutdown_connections 2020-11-16T22:16:11.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0584055060 0x7f0584057520 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05ac148110 0x7f05ac0b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05ac0b3570 0x7f05ac066850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 --2- 172.21.15.185:0/164533440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05ac13de10 0x7f05ac0b0170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 >> 172.21.15.185:0/164533440 conn(0x7f05ac064b90 msgr2=0x7f05ac067500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 shutdown_connections 2020-11-16T22:16:11.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.363+0000 7f05b31ff700 1 -- 172.21.15.185:0/164533440 wait complete. 2020-11-16T22:16:11.367 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:16:11.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 Processor -- start 2020-11-16T22:16:11.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 -- start start 2020-11-16T22:16:11.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd7000b17e0 0x7fd700067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd700098cc0 con 0x7fd7000b2160 2020-11-16T22:16:11.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7000b3950 con 0x7fd700066240 2020-11-16T22:16:11.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd708479700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70014c4f0 con 0x7fd7000b17e0 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706a16700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd700066240 0x7fd700065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51882/0 (socket says 172.21.15.185:51882) 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706a16700 1 -- 172.21.15.185:0/1653517776 learned_addr learned my addr 172.21.15.185:0/1653517776 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706215700 1 --2- 172.21.15.185:0/1653517776 >> v2:172.21.15.185:3300/0 conn(0x7fd7000b17e0 0x7fd700067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd705213700 1 -- 172.21.15.185:0/1653517776 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345254870 0 0) 0x7fd7000b3950 con 0x7fd700066240 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706215700 1 -- 172.21.15.185:0/1653517776 >> v1:172.21.15.185:6789/0 conn(0x7fd700066240 legacy=0x7fd700065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706215700 1 -- 172.21.15.185:0/1653517776 >> v1:172.21.15.96:6789/0 conn(0x7fd7000b2160 legacy=0x7fd70014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706215700 1 -- 172.21.15.185:0/1653517776 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7000b3bf0 con 0x7fd7000b17e0 2020-11-16T22:16:11.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd706215700 1 --2- 172.21.15.185:0/1653517776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd700067fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fd6f400b9a0 tx=0x7fd6f4006df0).ready entity=mon.1 client_cookie=2948976914772cff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd705213700 1 -- 172.21.15.185:0/1653517776 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40097b0 con 0x7fd7000b17e0 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd705213700 1 -- 172.21.15.185:0/1653517776 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f4009910 con 0x7fd7000b17e0 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.499+0000 7fd705213700 1 -- 172.21.15.185:0/1653517776 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4009c90 con 0x7fd7000b17e0 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- 172.21.15.185:0/1653517776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 msgr2=0x7fd700067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 --2- 172.21.15.185:0/1653517776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd700067fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fd6f400b9a0 tx=0x7fd6f4006df0).stop 2020-11-16T22:16:11.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- 172.21.15.185:0/1653517776 shutdown_connections 2020-11-16T22:16:11.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 --2- 172.21.15.185:0/1653517776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd700067fb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- 172.21.15.185:0/1653517776 >> 172.21.15.185:0/1653517776 conn(0x7fd7000604a0 msgr2=0x7fd7000608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- 172.21.15.185:0/1653517776 shutdown_connections 2020-11-16T22:16:11.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- 172.21.15.185:0/1653517776 wait complete. 2020-11-16T22:16:11.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 Processor -- start 2020-11-16T22:16:11.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- start start 2020-11-16T22:16:11.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 0x7fd700099d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 0x7fd700158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001482c0 con 0x7fd700066240 2020-11-16T22:16:11.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70015c0f0 con 0x7fd7000b17e0 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.503+0000 7fd708479700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70015bbe0 con 0x7fd7000b2160 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd705a14700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 0x7fd700158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706215700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 0x7fd700099d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706215700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 0x7fd700099d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.185:34510/0 (socket says 172.21.15.185:34510) 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706215700 1 -- 172.21.15.185:0/812788494 learned_addr learned my addr 172.21.15.185:0/812788494 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:11.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd705a14700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58452/0 (socket says 172.21.15.185:58452) 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 -- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 msgr2=0x7fd7001498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 -- 172.21.15.185:0/812788494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 msgr2=0x7fd700099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 0x7fd700099d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 -- 172.21.15.185:0/812788494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70015be50 con 0x7fd7000b2160 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd705a14700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:11.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706a16700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 0x7fd700158500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fd6f800c760 tx=0x7fd6f80078e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80072e0 con 0x7fd7000b2160 2020-11-16T22:16:11.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8007440 con 0x7fd7000b2160 2020-11-16T22:16:11.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700157dc0 con 0x7fd7000b2160 2020-11-16T22:16:11.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8005800 con 0x7fd7000b2160 2020-11-16T22:16:11.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd700149db0 con 0x7fd7000b2160 2020-11-16T22:16:11.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd700158260 con 0x7fd7000b2160 2020-11-16T22:16:11.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd6f8010070 con 0x7fd7000b2160 2020-11-16T22:16:11.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 0x7fd6d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd6f806cea0 con 0x7fd7000b2160 2020-11-16T22:16:11.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.507+0000 7fd706215700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 0x7fd6d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.511+0000 7fd706215700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 0x7fd6d8057520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fd6f400bf10 tx=0x7fd6f4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.515+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd700158260 con 0x7fd7000b2160 2020-11-16T22:16:11.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.667+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd700158260 con 0x7fd7000b2160 2020-11-16T22:16:11.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.667+0000 7fd6f27fc700 1 -- 172.21.15.185:0/812788494 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd700158260 con 0x7fd7000b2160 2020-11-16T22:16:11.654 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:11.654 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:16:11.655 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:16:11.656 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:16:11.656 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 msgr2=0x7fd6d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 0x7fd6d8057520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fd6f400bf10 tx=0x7fd6f4015000).stop 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 msgr2=0x7fd700158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 0x7fd700158500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fd6f800c760 tx=0x7fd6f80078e0).stop 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 shutdown_connections 2020-11-16T22:16:11.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd6d8055060 0x7fd6d8057520 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7000b2160 0x7fd700158500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd700066240 0x7fd700099d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 --2- 172.21.15.185:0/812788494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7000b17e0 0x7fd7001498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 >> 172.21.15.185:0/812788494 conn(0x7fd7000604a0 msgr2=0x7fd700062c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 shutdown_connections 2020-11-16T22:16:11.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.671+0000 7fd708479700 1 -- 172.21.15.185:0/812788494 wait complete. 2020-11-16T22:16:11.668 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:16:11.669 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:16:11.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 Processor -- start 2020-11-16T22:16:11.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 -- start start 2020-11-16T22:16:11.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f64240ba800 0x7f64240b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64240b7290 con 0x7f6424143470 2020-11-16T22:16:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64240b4960 con 0x7f64240b5d70 2020-11-16T22:16:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64240b4bf0 con 0x7f64240ba800 2020-11-16T22:16:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f641bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f64240ba800 0x7f64240b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642259c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f64240b5d70 0x7f64240b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51894/0 (socket says 172.21.15.185:51894) 2020-11-16T22:16:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f642259c700 1 -- 172.21.15.185:0/2561562181 learned_addr learned my addr 172.21.15.185:0/2561562181 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f641bfff700 1 -- 172.21.15.185:0/2561562181 >> v1:172.21.15.185:6789/0 conn(0x7f64240b5d70 legacy=0x7f64240b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f641bfff700 1 -- 172.21.15.185:0/2561562181 >> v1:172.21.15.96:6789/0 conn(0x7f6424143470 legacy=0x7f64241438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f641bfff700 1 -- 172.21.15.185:0/2561562181 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240b74c0 con 0x7f64240ba800 2020-11-16T22:16:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f641bfff700 1 --2- 172.21.15.185:0/2561562181 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240ba800 0x7f64240b4470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f640c00b9a0 tx=0x7f640c006df0).ready entity=mon.1 client_cookie=65e46dbfd832689d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f6421d9b700 1 -- 172.21.15.185:0/2561562181 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c0097b0 con 0x7f64240ba800 2020-11-16T22:16:11.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f6421d9b700 1 -- 172.21.15.185:0/2561562181 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c009910 con 0x7f64240ba800 2020-11-16T22:16:11.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.807+0000 7f6421d9b700 1 -- 172.21.15.185:0/2561562181 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c009c90 con 0x7f64240ba800 2020-11-16T22:16:11.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- 172.21.15.185:0/2561562181 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240ba800 msgr2=0x7f64240b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 --2- 172.21.15.185:0/2561562181 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240ba800 0x7f64240b4470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f640c00b9a0 tx=0x7f640c006df0).stop 2020-11-16T22:16:11.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- 172.21.15.185:0/2561562181 shutdown_connections 2020-11-16T22:16:11.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 --2- 172.21.15.185:0/2561562181 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240ba800 0x7f64240b4470 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- 172.21.15.185:0/2561562181 >> 172.21.15.185:0/2561562181 conn(0x7f642405ba80 msgr2=0x7f642405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:11.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- 172.21.15.185:0/2561562181 shutdown_connections 2020-11-16T22:16:11.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- 172.21.15.185:0/2561562181 wait complete. 2020-11-16T22:16:11.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 Processor -- start 2020-11-16T22:16:11.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- start start 2020-11-16T22:16:11.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 0x7f6424148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 0x7f64240b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 0x7f64240b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424154350 con 0x7f64240ba800 2020-11-16T22:16:11.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424154560 con 0x7f64240b5d70 2020-11-16T22:16:11.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64240ad0a0 con 0x7f6424143470 2020-11-16T22:16:11.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.811+0000 7f642259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 0x7f6424148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f642259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 0x7f6424148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58462/0 (socket says 172.21.15.185:58462) 2020-11-16T22:16:11.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f642259c700 1 -- 172.21.15.185:0/1638609893 learned_addr learned my addr 172.21.15.185:0/1638609893 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:11.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 0x7f64240b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641bfff700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 0x7f64240b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 -- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 msgr2=0x7f6424148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 0x7f6424148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 -- 172.21.15.185:0/1638609893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 msgr2=0x7f64240b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:11.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 0x7f64240b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:11.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 -- 172.21.15.185:0/1638609893 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240ad340 con 0x7f6424143470 2020-11-16T22:16:11.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641bfff700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 0x7f64240b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:11.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f6422d9d700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 0x7f64240b0a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f641c00ca60 tx=0x7f641c007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00a730 con 0x7f6424143470 2020-11-16T22:16:11.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c00a890 con 0x7f6424143470 2020-11-16T22:16:11.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00ac10 con 0x7f6424143470 2020-11-16T22:16:11.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64240af000 con 0x7f6424143470 2020-11-16T22:16:11.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64240b10b0 con 0x7f6424143470 2020-11-16T22:16:11.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642405d200 con 0x7f6424143470 2020-11-16T22:16:11.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f641c019030 con 0x7f6424143470 2020-11-16T22:16:11.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.815+0000 7f641a7fc700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 0x7f6400057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:11.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.819+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f641c06d290 con 0x7f6424143470 2020-11-16T22:16:11.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.819+0000 7f642259c700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 0x7f6400057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:11.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.819+0000 7f642259c700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 0x7f6400057520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f64140072d0 tx=0x7f6414007bf0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:11.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.823+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f641c043820 con 0x7f6424143470 2020-11-16T22:16:11.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:11.979+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6424069a50 con 0x7f6424143470 2020-11-16T22:16:12.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.963+0000 7f641a7fc700 1 -- 172.21.15.185:0/1638609893 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v37) v1 ==== 133+0+0 (secure 0 0 0) 0x7f641c046e40 con 0x7f6424143470 2020-11-16T22:16:12.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 msgr2=0x7f6400057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:12.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 0x7f6400057520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f64140072d0 tx=0x7f6414007bf0).stop 2020-11-16T22:16:12.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 msgr2=0x7f64240b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:12.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 0x7f64240b0a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f641c00ca60 tx=0x7f641c007e20).stop 2020-11-16T22:16:17.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 shutdown_connections 2020-11-16T22:16:17.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6400055060 0x7f6400057520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f64240b5d70 0x7f6424148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6424143470 0x7f64240b0a20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 --2- 172.21.15.185:0/1638609893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64240ba800 0x7f64240b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 >> 172.21.15.185:0/1638609893 conn(0x7f642405ba80 msgr2=0x7f64240b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 shutdown_connections 2020-11-16T22:16:17.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:12.967+0000 7f642359e700 1 -- 172.21.15.185:0/1638609893 wait complete. 2020-11-16T22:16:17.133 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:17.133 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:16:17.134 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:16:17.139 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:17.141 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 Processor -- start 2020-11-16T22:16:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 -- start start 2020-11-16T22:16:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43b40ac820 0x7f43b413f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b4060970 con 0x7f43b4060130 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b4060e40 con 0x7f43b40b3830 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b3fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b40b4610 con 0x7f43b40ac820 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.291+0000 7f43b27fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43b40ac820 0x7f43b413f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b2ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f43b40b3830 0x7f43b4143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51910/0 (socket says 172.21.15.185:51910) 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b27fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43b40ac820 0x7f43b413f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58474/0 (socket says 172.21.15.185:58474) 2020-11-16T22:16:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b2ffd700 1 -- 172.21.15.185:0/37121460 learned_addr learned my addr 172.21.15.185:0/37121460 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:17.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b27fc700 1 -- 172.21.15.185:0/37121460 >> v1:172.21.15.185:6789/0 conn(0x7f43b40b3830 legacy=0x7f43b4143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b27fc700 1 -- 172.21.15.185:0/37121460 >> v1:172.21.15.96:6789/0 conn(0x7f43b4060130 legacy=0x7f43b413fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b1ffb700 1 -- 172.21.15.185:0/37121460 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986614300 0 0) 0x7f43b4060e40 con 0x7f43b40b3830 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b27fc700 1 -- 172.21.15.185:0/37121460 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4060ba0 con 0x7f43b40ac820 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b27fc700 1 --2- 172.21.15.185:0/37121460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b413f840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f43a800b9a0 tx=0x7f43a8006df0).ready entity=mon.1 client_cookie=47d6f25042224b69 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b1ffb700 1 -- 172.21.15.185:0/37121460 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80097b0 con 0x7f43b40ac820 2020-11-16T22:16:17.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b1ffb700 1 -- 172.21.15.185:0/37121460 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8009910 con 0x7f43b40ac820 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b1ffb700 1 -- 172.21.15.185:0/37121460 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8009c90 con 0x7f43b40ac820 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- 172.21.15.185:0/37121460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 msgr2=0x7f43b413f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 --2- 172.21.15.185:0/37121460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b413f840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f43a800b9a0 tx=0x7f43a8006df0).stop 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- 172.21.15.185:0/37121460 shutdown_connections 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 --2- 172.21.15.185:0/37121460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b413f840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- 172.21.15.185:0/37121460 >> 172.21.15.185:0/37121460 conn(0x7f43b4063df0 msgr2=0x7f43b40641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:17.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- 172.21.15.185:0/37121460 shutdown_connections 2020-11-16T22:16:17.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- 172.21.15.185:0/37121460 wait complete. 2020-11-16T22:16:17.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 Processor -- start 2020-11-16T22:16:17.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.295+0000 7f43b3fff700 1 -- start start 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43b4060130 0x7f43b41533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 0x7f43b4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4118370 con 0x7f43b40b3830 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41538a0 con 0x7f43b40ac820 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4114700 con 0x7f43b4060130 2020-11-16T22:16:17.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58478/0 (socket says 172.21.15.185:58478) 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 -- 172.21.15.185:0/2635910313 learned_addr learned my addr 172.21.15.185:0/2635910313 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b2ffd700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43b4060130 0x7f43b41533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b37fe700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 0x7f43b4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 -- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43b4060130 msgr2=0x7f43b41533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43b4060130 0x7f43b41533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 -- 172.21.15.185:0/2635910313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 msgr2=0x7f43b4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 0x7f43b4157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 -- 172.21.15.185:0/2635910313 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4156ad0 con 0x7f43b40ac820 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b37fe700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 0x7f43b4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b27fc700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f43b40605e0 tx=0x7f43a8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a801e460 con 0x7f43b40ac820 2020-11-16T22:16:17.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a801e5c0 con 0x7f43b40ac820 2020-11-16T22:16:17.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b4156d70 con 0x7f43b40ac820 2020-11-16T22:16:17.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a801e940 con 0x7f43b40ac820 2020-11-16T22:16:17.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b40b02c0 con 0x7f43b40ac820 2020-11-16T22:16:17.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f43a8025030 con 0x7f43b40ac820 2020-11-16T22:16:17.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b405d200 con 0x7f43b40ac820 2020-11-16T22:16:17.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 0x7f4390057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:17.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f43b2ffd700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 0x7f4390057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:17.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.299+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f43a806f3e0 con 0x7f43b40ac820 2020-11-16T22:16:17.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.303+0000 7f43b2ffd700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 0x7f4390057520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f43a4000f40 tx=0x7f43a4007cc0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:17.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.307+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f43a80458c0 con 0x7f43b40ac820 2020-11-16T22:16:17.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.463+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f43b4061e20 con 0x7f43b40ac820 2020-11-16T22:16:17.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.971+0000 7f439affd700 1 -- 172.21.15.185:0/2635910313 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v41) v1 ==== 125+0+0 (secure 0 0 0) 0x7f43a8023070 con 0x7f43b40ac820 2020-11-16T22:16:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 msgr2=0x7f4390057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 0x7f4390057520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f43a4000f40 tx=0x7f43a4007cc0).stop 2020-11-16T22:16:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 msgr2=0x7f43b40b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f43b40605e0 tx=0x7f43a8015000).stop 2020-11-16T22:16:17.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 shutdown_connections 2020-11-16T22:16:17.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4390055060 0x7f4390057520 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b40b3830 0x7f43b4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43b4060130 0x7f43b41533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 --2- 172.21.15.185:0/2635910313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43b40ac820 0x7f43b40b56a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 >> 172.21.15.185:0/2635910313 conn(0x7f43b4063df0 msgr2=0x7f43b4141080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 shutdown_connections 2020-11-16T22:16:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:17.975+0000 7f43b3fff700 1 -- 172.21.15.185:0/2635910313 wait complete. 2020-11-16T22:16:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:16:17.976 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:16:18.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.167+0000 7fa07c1ed700 1 Processor -- start 2020-11-16T22:16:18.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.167+0000 7fa07c1ed700 1 -- start start 2020-11-16T22:16:18.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 0x7fa074148350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0740b45c0 0x7fa074143880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa074068620 con 0x7fa0740b41e0 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa074068420 con 0x7fa0740b45c0 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa074147970 con 0x7fa0740ae090 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079f89700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa0740b41e0 0x7fa074067d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51924/0 (socket says 172.21.15.185:51924) 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 0x7fa074148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079f89700 1 -- 172.21.15.185:0/867085869 learned_addr learned my addr 172.21.15.185:0/867085869 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07a78a700 1 --2- 172.21.15.185:0/867085869 >> v2:172.21.15.96:3300/0 conn(0x7fa0740b45c0 0x7fa074143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa078f87700 1 -- 172.21.15.185:0/867085869 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008291274 0 0) 0x7fa074068620 con 0x7fa0740b41e0 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa078f87700 1 -- 172.21.15.185:0/867085869 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003240 con 0x7fa0740b41e0 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 -- 172.21.15.185:0/867085869 >> v2:172.21.15.96:3300/0 conn(0x7fa0740b45c0 msgr2=0x7fa074143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 --2- 172.21.15.185:0/867085869 >> v2:172.21.15.96:3300/0 conn(0x7fa0740b45c0 0x7fa074143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 -- 172.21.15.185:0/867085869 >> v1:172.21.15.185:6789/0 conn(0x7fa0740b41e0 legacy=0x7fa074067d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 -- 172.21.15.185:0/867085869 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074147c10 con 0x7fa0740ae090 2020-11-16T22:16:18.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa079788700 1 --2- 172.21.15.185:0/867085869 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 0x7fa074148350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa06400b9a0 tx=0x7fa064006df0).ready entity=mon.2 client_cookie=6bc62f0a115d6daf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa078f87700 1 -- 172.21.15.185:0/867085869 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064009680 con 0x7fa0740ae090 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa078f87700 1 -- 172.21.15.185:0/867085869 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0640097e0 con 0x7fa0740ae090 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa078f87700 1 -- 172.21.15.185:0/867085869 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064009b60 con 0x7fa0740ae090 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- 172.21.15.185:0/867085869 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 msgr2=0x7fa074148350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/867085869 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 0x7fa074148350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa06400b9a0 tx=0x7fa064006df0).stop 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- 172.21.15.185:0/867085869 shutdown_connections 2020-11-16T22:16:18.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/867085869 >> v2:172.21.15.96:3300/0 conn(0x7fa0740b45c0 0x7fa074143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/867085869 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740ae090 0x7fa074148350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- 172.21.15.185:0/867085869 >> 172.21.15.185:0/867085869 conn(0x7fa074061240 msgr2=0x7fa074061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- 172.21.15.185:0/867085869 shutdown_connections 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.171+0000 7fa07c1ed700 1 -- 172.21.15.185:0/867085869 wait complete. 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 Processor -- start 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- start start 2020-11-16T22:16:18.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0740b41e0 0x7fa074153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa074145bb0 0x7fa0740aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740b1710 con 0x7fa0740b41e0 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074153660 con 0x7fa074145bb0 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07409d290 con 0x7fa0740b45c0 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51938/0 (socket says 172.21.15.185:51938) 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 -- 172.21.15.185:0/3947141224 learned_addr learned my addr 172.21.15.185:0/3947141224 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07a78a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa074145bb0 0x7fa0740aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079f89700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0740b41e0 0x7fa074153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 -- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa074145bb0 msgr2=0x7fa0740aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa074145bb0 0x7fa0740aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 -- 172.21.15.185:0/3947141224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0740b41e0 msgr2=0x7fa074153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0740b41e0 0x7fa074153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 -- 172.21.15.185:0/3947141224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074153890 con 0x7fa0740b45c0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa079788700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fa064006e30 tx=0x7fa064015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064011cc0 con 0x7fa0740b45c0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa064011e20 con 0x7fa0740b45c0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07409d530 con 0x7fa0740b45c0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06401d680 con 0x7fa0740b45c0 2020-11-16T22:16:18.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.175+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074154440 con 0x7fa0740b45c0 2020-11-16T22:16:18.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07405d4e0 con 0x7fa0740b45c0 2020-11-16T22:16:18.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa064024030 con 0x7fa0740b45c0 2020-11-16T22:16:18.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa061ffb700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 0x7fa04c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa06406f260 con 0x7fa0740b45c0 2020-11-16T22:16:18.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa079f89700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 0x7fa04c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.179+0000 7fa079f89700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 0x7fa04c057520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fa0700055a0 tx=0x7fa070009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.183+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa064045530 con 0x7fa0740b45c0 2020-11-16T22:16:18.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.379+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa074062250 con 0x7fa0740b45c0 2020-11-16T22:16:18.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.391+0000 7fa061ffb700 1 -- 172.21.15.185:0/3947141224 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v30) v1 ==== 187+0+0 (secure 0 0 0) 0x7fa064022030 con 0x7fa0740b45c0 2020-11-16T22:16:18.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 msgr2=0x7fa04c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 0x7fa04c057520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fa0700055a0 tx=0x7fa070009040).stop 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 msgr2=0x7fa0741456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fa064006e30 tx=0x7fa064015000).stop 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 shutdown_connections 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa04c055060 0x7fa04c057520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa074145bb0 0x7fa0740aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0740b41e0 0x7fa074153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 --2- 172.21.15.185:0/3947141224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0740b45c0 0x7fa0741456c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 >> 172.21.15.185:0/3947141224 conn(0x7fa074061240 msgr2=0x7fa074146480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 shutdown_connections 2020-11-16T22:16:18.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.395+0000 7fa07c1ed700 1 -- 172.21.15.185:0/3947141224 wait complete. 2020-11-16T22:16:18.383 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 7 and data pool 8 2020-11-16T22:16:18.397 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:18.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 Processor -- start 2020-11-16T22:16:18.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 -- start start 2020-11-16T22:16:18.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b400b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b40062700 con 0x7f4b400b0300 2020-11-16T22:16:18.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400629a0 con 0x7f4b40061640 2020-11-16T22:16:18.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b4837b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40062c40 con 0x7f4b40061cf0 2020-11-16T22:16:18.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b46918700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b400b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b45916700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4b40061640 0x7f4b4013dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51938/0 (socket says 172.21.15.185:51938) 2020-11-16T22:16:18.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b45916700 1 -- 172.21.15.185:0/165266827 learned_addr learned my addr 172.21.15.185:0/165266827 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:18.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b45115700 1 -- 172.21.15.185:0/165266827 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635149973 0 0) 0x7f4b400629a0 con 0x7f4b40061640 2020-11-16T22:16:18.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b46918700 1 -- 172.21.15.185:0/165266827 >> v1:172.21.15.185:6789/0 conn(0x7f4b40061640 legacy=0x7f4b4013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b46918700 1 -- 172.21.15.185:0/165266827 >> v1:172.21.15.96:6789/0 conn(0x7f4b400b0300 legacy=0x7f4b400b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b46918700 1 -- 172.21.15.185:0/165266827 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b40062ee0 con 0x7f4b40061cf0 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b46918700 1 --2- 172.21.15.185:0/165266827 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b400b06e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4b38009cc0 tx=0x7f4b380091d0).ready entity=mon.2 client_cookie=9a8d0543d677bbbb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.519+0000 7f4b45115700 1 -- 172.21.15.185:0/165266827 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800bbc0 con 0x7f4b40061cf0 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b45115700 1 -- 172.21.15.185:0/165266827 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3800bd20 con 0x7f4b40061cf0 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b45115700 1 -- 172.21.15.185:0/165266827 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38011610 con 0x7f4b40061cf0 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- 172.21.15.185:0/165266827 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 msgr2=0x7f4b400b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 --2- 172.21.15.185:0/165266827 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b400b06e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4b38009cc0 tx=0x7f4b380091d0).stop 2020-11-16T22:16:18.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- 172.21.15.185:0/165266827 shutdown_connections 2020-11-16T22:16:18.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 --2- 172.21.15.185:0/165266827 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b400b06e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- 172.21.15.185:0/165266827 >> 172.21.15.185:0/165266827 conn(0x7f4b40060130 msgr2=0x7f4b40060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- 172.21.15.185:0/165266827 shutdown_connections 2020-11-16T22:16:18.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- 172.21.15.185:0/165266827 wait complete. 2020-11-16T22:16:18.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 Processor -- start 2020-11-16T22:16:18.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- start start 2020-11-16T22:16:18.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b40061640 0x7f4b40141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b40141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40141a70 con 0x7f4b40061640 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40143820 con 0x7f4b400b0300 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b4837b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40142680 con 0x7f4b40061cf0 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b46918700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b46918700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58510/0 (socket says 172.21.15.185:58510) 2020-11-16T22:16:18.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.523+0000 7f4b46918700 1 -- 172.21.15.185:0/2280111737 learned_addr learned my addr 172.21.15.185:0/2280111737 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b45916700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b40141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46117700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b40061640 0x7f4b40141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 -- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 msgr2=0x7f4b40141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b40141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 -- 172.21.15.185:0/2280111737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b40061640 msgr2=0x7f4b40141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b40061640 0x7f4b40141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 -- 172.21.15.185:0/2280111737 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b401444e0 con 0x7f4b400b0300 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46918700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4b38009ea0 tx=0x7f4b38016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38006ea0 con 0x7f4b400b0300 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38011e30 con 0x7f4b400b0300 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38010cd0 con 0x7f4b400b0300 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b40142c30 con 0x7f4b400b0300 2020-11-16T22:16:18.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40142300 con 0x7f4b400b0300 2020-11-16T22:16:18.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4b38017070 con 0x7f4b400b0300 2020-11-16T22:16:18.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b40142dc0 con 0x7f4b400b0300 2020-11-16T22:16:18.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 0x7f4b18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b38070260 con 0x7f4b400b0300 2020-11-16T22:16:18.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46117700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 0x7f4b18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.527+0000 7f4b46117700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 0x7f4b18057520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f4b34003300 tx=0x7f4b34007b00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.535+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4b38046500 con 0x7f4b400b0300 2020-11-16T22:16:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b4005d4e0 con 0x7f4b400b0300 2020-11-16T22:16:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b327fc700 1 -- 172.21.15.185:0/2280111737 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f4b38015020 con 0x7f4b400b0300 2020-11-16T22:16:18.672 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:18.672 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":42,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:18.386630+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 msgr2=0x7f4b18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 0x7f4b18057520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f4b34003300 tx=0x7f4b34007b00).stop 2020-11-16T22:16:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 msgr2=0x7f4b4013f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4b38009ea0 tx=0x7f4b38016000).stop 2020-11-16T22:16:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 shutdown_connections 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b18055060 0x7f4b18057520 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b400b0300 0x7f4b4013f250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b40061640 0x7f4b40141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 --2- 172.21.15.185:0/2280111737 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b40061cf0 0x7f4b40141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 >> 172.21.15.185:0/2280111737 conn(0x7f4b40060130 msgr2=0x7f4b400b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 shutdown_connections 2020-11-16T22:16:18.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.687+0000 7f4b4837b700 1 -- 172.21.15.185:0/2280111737 wait complete. 2020-11-16T22:16:18.687 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:18.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.827+0000 7f3c4759e700 1 Processor -- start 2020-11-16T22:16:18.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.827+0000 7f3c4759e700 1 -- start start 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c480b7e10 con 0x7f3c480b4d40 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c480b7670 con 0x7f3c481418d0 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480b78d0 con 0x7f3c480675b0 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51958/0 (socket says 172.21.15.185:51958) 2020-11-16T22:16:18.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 -- 172.21.15.185:0/275620179 learned_addr learned my addr 172.21.15.185:0/275620179 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:18.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c45d9b700 1 --1- 172.21.15.185:0/275620179 >> v1:172.21.15.185:6789/0 conn(0x7f3c481418d0 0x7f3c480b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:51954/0 (socket says 172.21.15.185:51954) 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 -- 172.21.15.185:0/275620179 >> v1:172.21.15.185:6789/0 conn(0x7f3c481418d0 legacy=0x7f3c480b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 -- 172.21.15.185:0/275620179 >> v1:172.21.15.96:6789/0 conn(0x7f3c480b4d40 legacy=0x7f3c4813de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 -- 172.21.15.185:0/275620179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480b7b70 con 0x7f3c480675b0 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4659c700 1 --2- 172.21.15.185:0/275620179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f3c3800ba20 tx=0x7f3c38006eb0).ready entity=mon.2 client_cookie=fdab1b94e2970e68 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4559a700 1 -- 172.21.15.185:0/275620179 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380098d0 con 0x7f3c480675b0 2020-11-16T22:16:18.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4559a700 1 -- 172.21.15.185:0/275620179 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38009a30 con 0x7f3c480675b0 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4559a700 1 -- 172.21.15.185:0/275620179 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38009db0 con 0x7f3c480675b0 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- 172.21.15.185:0/275620179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 msgr2=0x7f3c48068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 --2- 172.21.15.185:0/275620179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f3c3800ba20 tx=0x7f3c38006eb0).stop 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- 172.21.15.185:0/275620179 shutdown_connections 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 --2- 172.21.15.185:0/275620179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48068460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- 172.21.15.185:0/275620179 >> 172.21.15.185:0/275620179 conn(0x7f3c4805ba80 msgr2=0x7f3c4805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- 172.21.15.185:0/275620179 shutdown_connections 2020-11-16T22:16:18.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.831+0000 7f3c4759e700 1 -- 172.21.15.185:0/275620179 wait complete. 2020-11-16T22:16:18.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 Processor -- start 2020-11-16T22:16:18.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 -- start start 2020-11-16T22:16:18.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c480b4d40 0x7f3c481397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c481418d0 0x7f3c48135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48137f10 con 0x7f3c480b4d40 2020-11-16T22:16:18.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c48144100 con 0x7f3c481418d0 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481390f0 con 0x7f3c480675b0 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51964/0 (socket says 172.21.15.185:51964) 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c4659c700 1 -- 172.21.15.185:0/591294815 learned_addr learned my addr 172.21.15.185:0/591294815 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c45d9b700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c480b4d40 0x7f3c481397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.835+0000 7f3c46d9d700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c481418d0 0x7f3c48135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 -- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c481418d0 msgr2=0x7f3c48135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c481418d0 0x7f3c48135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 -- 172.21.15.185:0/591294815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c480b4d40 msgr2=0x7f3c481397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c480b4d40 0x7f3c481397a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 -- 172.21.15.185:0/591294815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48144360 con 0x7f3c480675b0 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4659c700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3c38006ef0 tx=0x7f3c38015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38011e50 con 0x7f3c480675b0 2020-11-16T22:16:18.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3801e460 con 0x7f3c480675b0 2020-11-16T22:16:18.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c48144600 con 0x7f3c480675b0 2020-11-16T22:16:18.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801e800 con 0x7f3c480675b0 2020-11-16T22:16:18.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c48139c90 con 0x7f3c480675b0 2020-11-16T22:16:18.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4805d200 con 0x7f3c480675b0 2020-11-16T22:16:18.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3c38025030 con 0x7f3c480675b0 2020-11-16T22:16:18.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 0x7f3c1c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:18.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c3806f560 con 0x7f3c480675b0 2020-11-16T22:16:18.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c45d9b700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 0x7f3c1c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:18.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.839+0000 7f3c45d9b700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 0x7f3c1c057520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3c48138100 tx=0x7f3c3c007b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:18.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.847+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3c380458b0 con 0x7f3c480675b0 2020-11-16T22:16:18.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:18.999+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c48137270 con 0x7f3c480675b0 2020-11-16T22:16:18.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c367fc700 1 -- 172.21.15.185:0/591294815 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f3c380311e0 con 0x7f3c480675b0 2020-11-16T22:16:18.988 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:18.988 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":42,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:18.386630+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 msgr2=0x7f3c1c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 0x7f3c1c057520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f3c48138100 tx=0x7f3c3c007b80).stop 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 msgr2=0x7f3c48144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3c38006ef0 tx=0x7f3c38015000).stop 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 shutdown_connections 2020-11-16T22:16:18.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3c1c055060 0x7f3c1c057520 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c481418d0 0x7f3c48135380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c480675b0 0x7f3c48144aa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 --2- 172.21.15.185:0/591294815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c480b4d40 0x7f3c481397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 >> 172.21.15.185:0/591294815 conn(0x7f3c4805ba80 msgr2=0x7f3c48140d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 shutdown_connections 2020-11-16T22:16:18.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.003+0000 7f3c4759e700 1 -- 172.21.15.185:0/591294815 wait complete. 2020-11-16T22:16:19.002 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:16:19.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.139+0000 7fb6293b7700 1 Processor -- start 2020-11-16T22:16:19.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.139+0000 7fb6293b7700 1 -- start start 2020-11-16T22:16:19.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb6240b5d70 0x7fb6240ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6240b7290 con 0x7fb6241475d0 2020-11-16T22:16:19.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6240b4bf0 con 0x7fb6240b5d70 2020-11-16T22:16:19.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240b74c0 con 0x7fb6240b4470 2020-11-16T22:16:19.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51972/0 (socket says 172.21.15.185:51972) 2020-11-16T22:16:19.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 -- 172.21.15.185:0/1140616655 learned_addr learned my addr 172.21.15.185:0/1140616655 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb623fff700 1 --2- 172.21.15.185:0/1140616655 >> v2:172.21.15.96:3300/0 conn(0x7fb6240b5d70 0x7fb6240ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 -- 172.21.15.185:0/1140616655 >> v2:172.21.15.96:3300/0 conn(0x7fb6240b5d70 msgr2=0x7fb6240ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 --2- 172.21.15.185:0/1140616655 >> v2:172.21.15.96:3300/0 conn(0x7fb6240b5d70 0x7fb6240ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 -- 172.21.15.185:0/1140616655 >> v1:172.21.15.185:6789/0 conn(0x7fb6241475d0 legacy=0x7fb624147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 -- 172.21.15.185:0/1140616655 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6240b7760 con 0x7fb6240b4470 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6237fe700 1 --2- 172.21.15.185:0/1140616655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fb60c006fd0 tx=0x7fb60c006eb0).ready entity=mon.2 client_cookie=6b9127f15f67d6e4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb622ffd700 1 -- 172.21.15.185:0/1140616655 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0098d0 con 0x7fb6240b4470 2020-11-16T22:16:19.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb622ffd700 1 -- 172.21.15.185:0/1140616655 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c009a30 con 0x7fb6240b4470 2020-11-16T22:16:19.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb622ffd700 1 -- 172.21.15.185:0/1140616655 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c009db0 con 0x7fb6240b4470 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- 172.21.15.185:0/1140616655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 msgr2=0x7fb6240b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1140616655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fb60c006fd0 tx=0x7fb60c006eb0).stop 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- 172.21.15.185:0/1140616655 shutdown_connections 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1140616655 >> v2:172.21.15.96:3300/0 conn(0x7fb6240b5d70 0x7fb6240ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1140616655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240b7c00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- 172.21.15.185:0/1140616655 >> 172.21.15.185:0/1140616655 conn(0x7fb62405ba80 msgr2=0x7fb62405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:19.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- 172.21.15.185:0/1140616655 shutdown_connections 2020-11-16T22:16:19.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.143+0000 7fb6293b7700 1 -- 172.21.15.185:0/1140616655 wait complete. 2020-11-16T22:16:19.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 Processor -- start 2020-11-16T22:16:19.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- start start 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 0x7fb624156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 0x7fb6240af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240b16f0 con 0x7fb6241475d0 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624157360 con 0x7fb6240b5d70 2020-11-16T22:16:19.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624157600 con 0x7fb6240b4470 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 0x7fb624156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51978/0 (socket says 172.21.15.185:51978) 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 -- 172.21.15.185:0/1076509182 learned_addr learned my addr 172.21.15.185:0/1076509182 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb628bb6700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 0x7fb6240af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 -- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 msgr2=0x7fb624156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 0x7fb624156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 -- 172.21.15.185:0/1076509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 msgr2=0x7fb6240af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 0x7fb6240af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 -- 172.21.15.185:0/1076509182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6241578a0 con 0x7fb6240b4470 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb628bb6700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 0x7fb6240af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6237fe700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 0x7fb624156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb623fff700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fb61c00db60 tx=0x7fb61c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009790 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0098f0 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624157070 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009c90 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240afb20 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb61c013480 con 0x7fb6240b4470 2020-11-16T22:16:19.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.147+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62405d200 con 0x7fb6240b4470 2020-11-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.151+0000 7fb61bfff700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 0x7fb5f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.151+0000 7fb6237fe700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 0x7fb5f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.151+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb61c06e420 con 0x7fb6240b4470 2020-11-16T22:16:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.151+0000 7fb6237fe700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 0x7fb5f4057520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb6240ad7c0 tx=0x7fb60c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.155+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb61c044620 con 0x7fb6240b4470 2020-11-16T22:16:19.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.359+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb6240afd10 con 0x7fb6240b4470 2020-11-16T22:16:19.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.391+0000 7fb61bfff700 1 -- 172.21.15.185:0/1076509182 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb61c047c40 con 0x7fb6240b4470 2020-11-16T22:16:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 msgr2=0x7fb5f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 0x7fb5f4057520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb6240ad7c0 tx=0x7fb60c015000).stop 2020-11-16T22:16:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 msgr2=0x7fb6240ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fb61c00db60 tx=0x7fb61c008040).stop 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 shutdown_connections 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5f4055060 0x7fb5f4057520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6241475d0 0x7fb6240af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b4470 0x7fb6240ae3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 --2- 172.21.15.185:0/1076509182 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb6240b5d70 0x7fb624156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 >> 172.21.15.185:0/1076509182 conn(0x7fb62405ba80 msgr2=0x7fb624069720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:19.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 shutdown_connections 2020-11-16T22:16:19.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.395+0000 7fb6293b7700 1 -- 172.21.15.185:0/1076509182 wait complete. 2020-11-16T22:16:19.398 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:19.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 Processor -- start 2020-11-16T22:16:19.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 -- start start 2020-11-16T22:16:19.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad9c060130 0x7fad9c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c060f40 con 0x7fad9c0b7510 2020-11-16T22:16:19.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c1420f0 con 0x7fad9c060ab0 2020-11-16T22:16:19.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada315d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c142390 con 0x7fad9c060130 2020-11-16T22:16:19.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad9c060130 0x7fad9c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad9c060130 0x7fad9c0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58542/0 (socket says 172.21.15.185:58542) 2020-11-16T22:16:19.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 -- 172.21.15.185:0/2425921904 learned_addr learned my addr 172.21.15.185:0/2425921904 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 -- 172.21.15.185:0/2425921904 >> v1:172.21.15.185:6789/0 conn(0x7fad9c060ab0 legacy=0x7fad9c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 -- 172.21.15.185:0/2425921904 >> v1:172.21.15.96:6789/0 conn(0x7fad9c0b7510 legacy=0x7fad9c13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 -- 172.21.15.185:0/2425921904 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c142630 con 0x7fad9c060130 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.527+0000 7fada195a700 1 --2- 172.21.15.185:0/2425921904 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c060130 0x7fad9c0b78f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fad8c006fd0 tx=0x7fad8c006eb0).ready entity=mon.1 client_cookie=22769236a318cde7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada1159700 1 -- 172.21.15.185:0/2425921904 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0098d0 con 0x7fad9c060130 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada1159700 1 -- 172.21.15.185:0/2425921904 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c009a30 con 0x7fad9c060130 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada1159700 1 -- 172.21.15.185:0/2425921904 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c009db0 con 0x7fad9c060130 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- 172.21.15.185:0/2425921904 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c060130 msgr2=0x7fad9c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 --2- 172.21.15.185:0/2425921904 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c060130 0x7fad9c0b78f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fad8c006fd0 tx=0x7fad8c006eb0).stop 2020-11-16T22:16:19.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- 172.21.15.185:0/2425921904 shutdown_connections 2020-11-16T22:16:19.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 --2- 172.21.15.185:0/2425921904 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c060130 0x7fad9c0b78f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- 172.21.15.185:0/2425921904 >> 172.21.15.185:0/2425921904 conn(0x7fad9c05ba80 msgr2=0x7fad9c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:19.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- 172.21.15.185:0/2425921904 shutdown_connections 2020-11-16T22:16:19.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- 172.21.15.185:0/2425921904 wait complete. 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 Processor -- start 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- start start 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 0x7fad9c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 0x7fad9c13d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c143350 con 0x7fad9c060130 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c13f5a0 con 0x7fad9c0b7510 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada315d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0ac3e0 con 0x7fad9c060ab0 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada195a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada295c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 0x7fad9c13d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada195a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:51994/0 (socket says 172.21.15.185:51994) 2020-11-16T22:16:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada195a700 1 -- 172.21.15.185:0/2820577946 learned_addr learned my addr 172.21.15.185:0/2820577946 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.531+0000 7fada215b700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 0x7fad9c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 -- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 msgr2=0x7fad9c13d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 0x7fad9c13d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 -- 172.21.15.185:0/2820577946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 msgr2=0x7fad9c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 0x7fad9c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 -- 172.21.15.185:0/2820577946 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c13f060 con 0x7fad9c060ab0 2020-11-16T22:16:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada215b700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 0x7fad9c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada295c700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 0x7fad9c13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada195a700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fad8c009ed0 tx=0x7fad8c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01e460 con 0x7fad9c060ab0 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c01e5c0 con 0x7fad9c060ab0 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c13f300 con 0x7fad9c060ab0 2020-11-16T22:16:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01e960 con 0x7fad9c060ab0 2020-11-16T22:16:19.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c13eca0 con 0x7fad9c060ab0 2020-11-16T22:16:19.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c05d200 con 0x7fad9c060ab0 2020-11-16T22:16:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fad8c025030 con 0x7fad9c060ab0 2020-11-16T22:16:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 0x7fad74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada215b700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 0x7fad74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad8c06f5a0 con 0x7fad9c060ab0 2020-11-16T22:16:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.535+0000 7fada215b700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 0x7fad74057520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fad98007f90 tx=0x7fad98009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.543+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fad8c0457e0 con 0x7fad9c060ab0 2020-11-16T22:16:19.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.743+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad9c0b39b0 con 0x7fad9c060ab0 2020-11-16T22:16:19.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.743+0000 7fad8a7fc700 1 -- 172.21.15.185:0/2820577946 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2487 (secure 0 0 0) 0x7fad8c0239d0 con 0x7fad9c060ab0 2020-11-16T22:16:19.729 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:19.730 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5183,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:19.390162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:19.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 msgr2=0x7fad74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 0x7fad74057520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fad98007f90 tx=0x7fad98009040).stop 2020-11-16T22:16:19.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 msgr2=0x7fad9c144180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fad8c009ed0 tx=0x7fad8c015000).stop 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 shutdown_connections 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fad74055060 0x7fad74057520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad9c0b7510 0x7fad9c13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad9c060130 0x7fad9c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 --2- 172.21.15.185:0/2820577946 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad9c060ab0 0x7fad9c144180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 >> 172.21.15.185:0/2820577946 conn(0x7fad9c05ba80 msgr2=0x7fad9c13af30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:19.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 shutdown_connections 2020-11-16T22:16:19.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.747+0000 7fada315d700 1 -- 172.21.15.185:0/2820577946 wait complete. 2020-11-16T22:16:19.733 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 31 2020-11-16T22:16:19.744 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:19.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 Processor -- start 2020-11-16T22:16:19.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 -- start start 2020-11-16T22:16:19.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1b94066740 0x7f1b94064a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b941476b0 con 0x7f1b94066d80 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b94147950 con 0x7f1b940646a0 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b99b67700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b941485d0 con 0x7f1b94066740 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b93fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1b94066740 0x7f1b94064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b93fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1b94066740 0x7f1b94064a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58560/0 (socket says 172.21.15.185:58560) 2020-11-16T22:16:19.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.875+0000 7f1b93fff700 1 -- 172.21.15.185:0/2317945028 learned_addr learned my addr 172.21.15.185:0/2317945028 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b927fc700 1 -- 172.21.15.185:0/2317945028 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003917214 0 0) 0x7f1b941476b0 con 0x7f1b94066d80 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b93fff700 1 -- 172.21.15.185:0/2317945028 >> v1:172.21.15.185:6789/0 conn(0x7f1b940646a0 legacy=0x7f1b940b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b93fff700 1 -- 172.21.15.185:0/2317945028 >> v1:172.21.15.96:6789/0 conn(0x7f1b94066d80 legacy=0x7f1b941440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b93fff700 1 -- 172.21.15.185:0/2317945028 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94147bf0 con 0x7f1b94066740 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b93fff700 1 --2- 172.21.15.185:0/2317945028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b94066740 0x7f1b94064a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f1b8c009cd0 tx=0x7f1b8c009290).ready entity=mon.1 client_cookie=4e410ce16866748d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b927fc700 1 -- 172.21.15.185:0/2317945028 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00bb40 con 0x7f1b94066740 2020-11-16T22:16:19.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b927fc700 1 -- 172.21.15.185:0/2317945028 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c00bca0 con 0x7f1b94066740 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b927fc700 1 -- 172.21.15.185:0/2317945028 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c011400 con 0x7f1b94066740 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- 172.21.15.185:0/2317945028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b94066740 msgr2=0x7f1b94064a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 --2- 172.21.15.185:0/2317945028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b94066740 0x7f1b94064a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f1b8c009cd0 tx=0x7f1b8c009290).stop 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- 172.21.15.185:0/2317945028 shutdown_connections 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 --2- 172.21.15.185:0/2317945028 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b94066740 0x7f1b94064a80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- 172.21.15.185:0/2317945028 >> 172.21.15.185:0/2317945028 conn(0x7f1b94061240 msgr2=0x7f1b94061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:19.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- 172.21.15.185:0/2317945028 shutdown_connections 2020-11-16T22:16:19.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- 172.21.15.185:0/2317945028 wait complete. 2020-11-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 Processor -- start 2020-11-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.879+0000 7f1b99b67700 1 -- start start 2020-11-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b940646a0 0x7f1b941539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b94066d80 0x7f1b94156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94142cd0 con 0x7f1b94066d80 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94153260 con 0x7f1b940646a0 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94153500 con 0x7f1b94066740 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b940646a0 0x7f1b941539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52008/0 (socket says 172.21.15.185:52008) 2020-11-16T22:16:19.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 -- 172.21.15.185:0/3007887517 learned_addr learned my addr 172.21.15.185:0/3007887517 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b93fff700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b94066d80 0x7f1b94156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 -- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b940646a0 msgr2=0x7f1b941539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b940646a0 0x7f1b941539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 -- 172.21.15.185:0/3007887517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b94066d80 msgr2=0x7f1b94156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b94066d80 0x7f1b94156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 -- 172.21.15.185:0/3007887517 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94153e90 con 0x7f1b94066740 2020-11-16T22:16:19.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b92ffd700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1b8400c730 tx=0x7f1b84007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400e490 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8400e610 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b94152f80 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400e9d0 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9409c9a0 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1b84019030 con 0x7f1b94066740 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 0x7f1b6c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:19.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9405d4e0 con 0x7f1b94066740 2020-11-16T22:16:19.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.883+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b8406d0b0 con 0x7f1b94066740 2020-11-16T22:16:19.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.887+0000 7f1b937fe700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 0x7f1b6c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:19.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.887+0000 7f1b937fe700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 0x7f1b6c057520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1b7c000f20 tx=0x7f1b7c007b70).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:19.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:19.891+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1b840433c0 con 0x7f1b94066740 2020-11-16T22:16:20.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.047+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b94153640 con 0x7f1b94066740 2020-11-16T22:16:20.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b737fe700 1 -- 172.21.15.185:0/3007887517 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f1b84021020 con 0x7f1b94066740 2020-11-16T22:16:20.035 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:20.035 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":43,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:19.390901+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:20.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 msgr2=0x7f1b6c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 0x7f1b6c057520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1b7c000f20 tx=0x7f1b7c007b70).stop 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 msgr2=0x7f1b940b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1b8400c730 tx=0x7f1b84007a90).stop 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 shutdown_connections 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1b6c055060 0x7f1b6c057520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b940646a0 0x7f1b941539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b94066d80 0x7f1b94156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 --2- 172.21.15.185:0/3007887517 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b94066740 0x7f1b940b35c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 >> 172.21.15.185:0/3007887517 conn(0x7f1b94061240 msgr2=0x7f1b94140060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:20.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 shutdown_connections 2020-11-16T22:16:20.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.051+0000 7f1b99b67700 1 -- 172.21.15.185:0/3007887517 wait complete. 2020-11-16T22:16:20.049 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:16:20.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.191+0000 7f1d63fff700 1 Processor -- start 2020-11-16T22:16:20.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.191+0000 7f1d63fff700 1 -- start start 2020-11-16T22:16:20.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d63fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d63fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d63fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d64143db0 con 0x7f1d640611c0 2020-11-16T22:16:20.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d63fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d640b2e70 con 0x7f1d640604a0 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d63fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64144bc0 con 0x7f1d641406f0 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d62ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52018/0 (socket says 172.21.15.185:52018) 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d62ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58570/0 (socket says 172.21.15.185:58570) 2020-11-16T22:16:20.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 -- 172.21.15.185:0/3563150330 learned_addr learned my addr 172.21.15.185:0/3563150330 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 -- 172.21.15.185:0/3563150330 >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 msgr2=0x7f1d640b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 --2- 172.21.15.185:0/3563150330 >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 -- 172.21.15.185:0/3563150330 >> v1:172.21.15.96:6789/0 conn(0x7f1d640611c0 legacy=0x7f1d641444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 -- 172.21.15.185:0/3563150330 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64140b30 con 0x7f1d641406f0 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d62ffd700 1 --2- 172.21.15.185:0/3563150330 >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:20.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d637fe700 1 --2- 172.21.15.185:0/3563150330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1d640b2fb0 tx=0x7f1d5c006eb0).ready entity=mon.2 client_cookie=aa3944d782cae897 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d61ffb700 1 -- 172.21.15.185:0/3563150330 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009810 con 0x7f1d641406f0 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d61ffb700 1 -- 172.21.15.185:0/3563150330 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c009970 con 0x7f1d641406f0 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.195+0000 7f1d61ffb700 1 -- 172.21.15.185:0/3563150330 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009cf0 con 0x7f1d641406f0 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- 172.21.15.185:0/3563150330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 msgr2=0x7f1d6413a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 --2- 172.21.15.185:0/3563150330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1d640b2fb0 tx=0x7f1d5c006eb0).stop 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- 172.21.15.185:0/3563150330 shutdown_connections 2020-11-16T22:16:20.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 --2- 172.21.15.185:0/3563150330 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d641406f0 0x7f1d6413a210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 --2- 172.21.15.185:0/3563150330 >> v2:172.21.15.185:3300/0 conn(0x7f1d640604a0 0x7f1d640b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- 172.21.15.185:0/3563150330 >> 172.21.15.185:0/3563150330 conn(0x7f1d64065ff0 msgr2=0x7f1d64066400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:20.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- 172.21.15.185:0/3563150330 shutdown_connections 2020-11-16T22:16:20.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- 172.21.15.185:0/3563150330 wait complete. 2020-11-16T22:16:20.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 Processor -- start 2020-11-16T22:16:20.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.199+0000 7f1d63fff700 1 -- start start 2020-11-16T22:16:20.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d640611c0 0x7f1d640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d641406f0 0x7f1d64154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d640b1300 con 0x7f1d641406f0 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64113bb0 con 0x7f1d640604a0 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64113e50 con 0x7f1d640611c0 2020-11-16T22:16:20.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58576/0 (socket says 172.21.15.185:58576) 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 -- 172.21.15.185:0/1278519501 learned_addr learned my addr 172.21.15.185:0/1278519501 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d627fc700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d640611c0 0x7f1d640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d637fe700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d641406f0 0x7f1d64154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 -- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d640611c0 msgr2=0x7f1d640b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d640611c0 0x7f1d640b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 -- 172.21.15.185:0/1278519501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d641406f0 msgr2=0x7f1d64154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d641406f0 0x7f1d64154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 -- 172.21.15.185:0/1278519501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d641140f0 con 0x7f1d640604a0 2020-11-16T22:16:20.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d62ffd700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f1d5400c150 tx=0x7f1d540091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:20.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400cd20 con 0x7f1d640604a0 2020-11-16T22:16:20.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5400ce80 con 0x7f1d640604a0 2020-11-16T22:16:20.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d641546f0 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54012600 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64151080 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.203+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6405d200 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.207+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1d54022070 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.207+0000 7f1d4affd700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 0x7f1d40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.207+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d5406e690 con 0x7f1d640604a0 2020-11-16T22:16:20.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.207+0000 7f1d627fc700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 0x7f1d40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.207+0000 7f1d627fc700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 0x7f1d40057520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f1d640b5ce0 tx=0x7f1d58009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:20.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.211+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1d54044890 con 0x7f1d640604a0 2020-11-16T22:16:20.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.415+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f1d64154880 con 0x7f1d640604a0 2020-11-16T22:16:20.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.423+0000 7f1d4affd700 1 -- 172.21.15.185:0/1278519501 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1d5406ce60 con 0x7f1d640604a0 2020-11-16T22:16:20.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 msgr2=0x7f1d40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 0x7f1d40057520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f1d640b5ce0 tx=0x7f1d58009040).stop 2020-11-16T22:16:20.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 msgr2=0x7f1d64110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f1d5400c150 tx=0x7f1d540091d0).stop 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 shutdown_connections 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d40055060 0x7f1d40057520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d640604a0 0x7f1d64110d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d641406f0 0x7f1d64154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 --2- 172.21.15.185:0/1278519501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d640611c0 0x7f1d640b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:20.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 >> 172.21.15.185:0/1278519501 conn(0x7f1d64065ff0 msgr2=0x7f1d640663d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 shutdown_connections 2020-11-16T22:16:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.427+0000 7f1d63fff700 1 -- 172.21.15.185:0/1278519501 wait complete. 2020-11-16T22:16:20.413 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:16:20.426 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:20.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 Processor -- start 2020-11-16T22:16:20.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 -- start start 2020-11-16T22:16:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa89c0b3090 0x7fa89c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:20.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c061fe0 con 0x7fa89c0ad110 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89c0b2b60 con 0x7fa89c0b3090 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa8a0e2d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0b0fe0 con 0x7fa89c0b1d30 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52028/0 (socket says 172.21.15.185:52028) 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89affd700 1 -- 172.21.15.185:0/3413442248 learned_addr learned my addr 172.21.15.185:0/3413442248 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:20.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa89c0b3090 0x7fa89c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:24.679 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:16:23.978+0000 7fbbc74dd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:16:24.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 -- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 msgr2=0x7fa89c14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:24.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89a7fc700 1 -- 172.21.15.185:0/3413442248 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653594805 0 0) 0x7fa89c061fe0 con 0x7fa89c0ad110 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 -- 172.21.15.185:0/3413442248 >> v1:172.21.15.96:6789/0 conn(0x7fa89c0ad110 legacy=0x7fa89c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 -- 172.21.15.185:0/3413442248 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0b1280 con 0x7fa89c0b3090 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89affd700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89b7fe700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0b3090 0x7fa89c0b1720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fa88c00b9a0 tx=0x7fa88c006df0).ready entity=mon.1 client_cookie=ac4d71416f59ff5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89a7fc700 1 -- 172.21.15.185:0/3413442248 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0097b0 con 0x7fa89c0b3090 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89a7fc700 1 -- 172.21.15.185:0/3413442248 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c009910 con 0x7fa89c0b3090 2020-11-16T22:16:24.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.563+0000 7fa89a7fc700 1 -- 172.21.15.185:0/3413442248 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c009c90 con 0x7fa89c0b3090 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0b3090 msgr2=0x7fa89c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0b3090 0x7fa89c0b1720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fa88c00b9a0 tx=0x7fa88c006df0).stop 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3413442248 shutdown_connections 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0b3090 0x7fa89c0b1720 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3413442248 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b1d30 0x7fa89c14b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3413442248 >> 172.21.15.185:0/3413442248 conn(0x7fa89c060130 msgr2=0x7fa89c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3413442248 shutdown_connections 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3413442248 wait complete. 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 Processor -- start 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.567+0000 7fa8a0e2d700 1 -- start start 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 0x7fa89c1460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa89c0b1d30 0x7fa89c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:24.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 0x7fa89c0af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c141940 con 0x7fa89c0b1d30 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c140940 con 0x7fa89c0ad110 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c142cd0 con 0x7fa89c0b3090 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 0x7fa89c1460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 0x7fa89c0af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 0x7fa89c1460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58590/0 (socket says 172.21.15.185:58590) 2020-11-16T22:16:24.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89b7fe700 1 -- 172.21.15.185:0/3375322951 learned_addr learned my addr 172.21.15.185:0/3375322951 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89affd700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa89c0b1d30 0x7fa89c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 -- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 msgr2=0x7fa89c1460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 0x7fa89c1460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 -- 172.21.15.185:0/3375322951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa89c0b1d30 msgr2=0x7fa89c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa89c0b1d30 0x7fa89c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 -- 172.21.15.185:0/3375322951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c141b70 con 0x7fa89c0b3090 2020-11-16T22:16:24.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa89bfff700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 0x7fa89c0af670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fa89400c710 tx=0x7fa894007890).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894013410 con 0x7fa89c0b3090 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894013570 con 0x7fa89c0b3090 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c140b70 con 0x7fa89c0b3090 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894013930 con 0x7fa89c0b3090 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c141390 con 0x7fa89c0b3090 2020-11-16T22:16:24.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.571+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa894010030 con 0x7fa89c0b3090 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.575+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c0aefc0 con 0x7fa89c0b3090 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.575+0000 7fa8937fe700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 0x7fa86c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.575+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa89406d670 con 0x7fa89c0b3090 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.575+0000 7fa89b7fe700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 0x7fa86c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.575+0000 7fa89b7fe700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 0x7fa86c057550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa88c00c010 tx=0x7fa88c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.579+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa8940438d0 con 0x7fa89c0b3090 2020-11-16T22:16:24.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.783+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa89c05d200 con 0x7fa89c0b3090 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.783+0000 7fa8937fe700 1 -- 172.21.15.185:0/3375322951 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2487 (secure 0 0 0) 0x7fa894046ef0 con 0x7fa89c0b3090 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 msgr2=0x7fa86c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 0x7fa86c057550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fa88c00c010 tx=0x7fa88c015000).stop 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 msgr2=0x7fa89c0af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 0x7fa89c0af670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fa89400c710 tx=0x7fa894007890).stop 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 shutdown_connections 2020-11-16T22:16:24.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa86c055090 0x7fa86c057550 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa89c0ad110 0x7fa89c1460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa89c0b3090 0x7fa89c0af670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 --2- 172.21.15.185:0/3375322951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa89c0b1d30 0x7fa89c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 >> 172.21.15.185:0/3375322951 conn(0x7fa89c060130 msgr2=0x7fa89c14d900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 shutdown_connections 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:20.787+0000 7fa8a0e2d700 1 -- 172.21.15.185:0/3375322951 wait complete. 2020-11-16T22:16:24.688 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 31 2020-11-16T22:16:24.689 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:24.689 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5183,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:19.390162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:24.991 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:16:18.387140+0000', 'modified': '2020-11-16T22:16:19.390162+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5542': {'gid': 5542, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6834/653581747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 653581747}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 653581747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:16:24.992 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:16:24.993 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:16:25.133 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":5542,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":6.7531987019999997,"mdsmap_epoch":31,"osdmap_epoch":44,"osdmap_epoch_barrier":42,"uptime":15.931336356999999} 2020-11-16T22:16:25.134 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5542, "mdsmap_epoch": 31, "osdmap_epoch": 44, "osdmap_epoch_barrier": 42, "rank_uptime": 6.753198702, "state": "up:active", "uptime": 15.931336357, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:16:25.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:16:25.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:16:25.135 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:16:25.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:16:25.136 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:16:25.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:16:25.136 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:16:25.136 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:16:25.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.137 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:16:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:16:25.157 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:16:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:16:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:16:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.160 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:16:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:16:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:16:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:16:25.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.161 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:16:25.194 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:16:25.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.198 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:16:25.210 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:16:25.211 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:16:25.211 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:16:25.211 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:16:25.211 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:16:25.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.212 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:16:25.213 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:16:25.213 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:16:25.213 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:16:25.213 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:16:25.214 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:16:25.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.353 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:16:25.369 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:16:25.370 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:16:25.370 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:16:25.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:25.371 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:16:25.399 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:16:25.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 Processor -- start 2020-11-16T22:16:25.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 -- start start 2020-11-16T22:16:25.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 0x7f78a00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 0x7f78a0062e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a00b4060 con 0x7f78a00b86d0 2020-11-16T22:16:25.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a00632f0 con 0x7f78a00b30e0 2020-11-16T22:16:25.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a69ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00b4430 con 0x7f78a0062970 2020-11-16T22:16:25.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a51eb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 0x7f78a00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 0x7f78a0062e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a51eb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 0x7f78a00b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58598/0 (socket says 172.21.15.185:58598) 2020-11-16T22:16:25.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a51eb700 1 -- 172.21.15.185:0/1431920968 learned_addr learned my addr 172.21.15.185:0/1431920968 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:25.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 -- 172.21.15.185:0/1431920968 >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 msgr2=0x7f78a00b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:25.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 --2- 172.21.15.185:0/1431920968 >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 0x7f78a00b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 -- 172.21.15.185:0/1431920968 >> v1:172.21.15.96:6789/0 conn(0x7f78a00b86d0 legacy=0x7f78a00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 -- 172.21.15.185:0/1431920968 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00b46d0 con 0x7f78a0062970 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a61ed700 1 --2- 172.21.15.185:0/1431920968 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 0x7f78a0062e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f7898009c90 tx=0x7f78980091d0).ready entity=mon.2 client_cookie=53c8ce54f9a2801c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.543+0000 7f78a49ea700 1 -- 172.21.15.185:0/1431920968 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800ba70 con 0x7f78a0062970 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a49ea700 1 -- 172.21.15.185:0/1431920968 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800bbd0 con 0x7f78a0062970 2020-11-16T22:16:25.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a49ea700 1 -- 172.21.15.185:0/1431920968 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898011400 con 0x7f78a0062970 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- 172.21.15.185:0/1431920968 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 msgr2=0x7f78a0062e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1431920968 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 0x7f78a0062e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f7898009c90 tx=0x7f78980091d0).stop 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- 172.21.15.185:0/1431920968 shutdown_connections 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1431920968 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a0062970 0x7f78a0062e00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1431920968 >> v2:172.21.15.185:3300/0 conn(0x7f78a00b30e0 0x7f78a00b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- 172.21.15.185:0/1431920968 >> 172.21.15.185:0/1431920968 conn(0x7f78a00645d0 msgr2=0x7f78a00649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- 172.21.15.185:0/1431920968 shutdown_connections 2020-11-16T22:16:25.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- 172.21.15.185:0/1431920968 wait complete. 2020-11-16T22:16:25.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 Processor -- start 2020-11-16T22:16:25.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- start start 2020-11-16T22:16:25.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f78a0062970 0x7f78a01570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78a00b86d0 0x7f78a0139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a01463e0 con 0x7f78a00b86d0 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a0147e00 con 0x7f78a0062970 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a69ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a013ac60 con 0x7f78a00b30e0 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52048/0 (socket says 172.21.15.185:52048) 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 -- 172.21.15.185:0/1776850828 learned_addr learned my addr 172.21.15.185:0/1776850828 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a59ec700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f78a0062970 0x7f78a01570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a61ed700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78a00b86d0 0x7f78a0139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 -- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f78a0062970 msgr2=0x7f78a01570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f78a0062970 0x7f78a01570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 -- 172.21.15.185:0/1776850828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78a00b86d0 msgr2=0x7f78a0139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78a00b86d0 0x7f78a0139f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.547+0000 7f78a51eb700 1 -- 172.21.15.185:0/1776850828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0148060 con 0x7f78a00b30e0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a51eb700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f789c00f890 tx=0x7f789c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c014670 con 0x7f78a00b30e0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c0147d0 con 0x7f78a00b30e0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a0148300 con 0x7f78a00b30e0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c014b50 con 0x7f78a00b30e0 2020-11-16T22:16:25.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a0147980 con 0x7f78a00b30e0 2020-11-16T22:16:25.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a00b48b0 con 0x7f78a00b30e0 2020-11-16T22:16:25.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f789c024070 con 0x7f78a00b30e0 2020-11-16T22:16:25.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 0x7f7878057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f789c06e240 con 0x7f78a00b30e0 2020-11-16T22:16:25.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a59ec700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 0x7f7878057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.551+0000 7f78a59ec700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 0x7f7878057520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f7894003300 tx=0x7f7894007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.559+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f789c0444a0 con 0x7f78a00b30e0 2020-11-16T22:16:25.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.707+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78a0146520 con 0x7f78a00b30e0 2020-11-16T22:16:25.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f7891ffb700 1 -- 172.21.15.185:0/1776850828 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f789c018020 con 0x7f78a00b30e0 2020-11-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:25.695 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":44,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:20.401788+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 msgr2=0x7f7878057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 0x7f7878057520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f7894003300 tx=0x7f7894007b90).stop 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 msgr2=0x7f78a00abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f789c00f890 tx=0x7f789c008040).stop 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 shutdown_connections 2020-11-16T22:16:25.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7878055060 0x7f7878057520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f78a0062970 0x7f78a01570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78a00b86d0 0x7f78a0139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 --2- 172.21.15.185:0/1776850828 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78a00b30e0 0x7f78a00abef0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 >> 172.21.15.185:0/1776850828 conn(0x7f78a00645d0 msgr2=0x7f78a013b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 shutdown_connections 2020-11-16T22:16:25.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.711+0000 7f78a69ee700 1 -- 172.21.15.185:0/1776850828 wait complete. 2020-11-16T22:16:25.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:16:12.044444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:16:17.522251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:16:25.710 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:16:25.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.855+0000 7f0110deb700 1 Processor -- start 2020-11-16T22:16:25.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.855+0000 7f0110deb700 1 -- start start 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 0x7f010c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c060970 con 0x7f010c0b3830 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c0b4610 con 0x7f010c060130 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c060ba0 con 0x7f010c0ac820 2020-11-16T22:16:25.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 0x7f010c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52056/0 (socket says 172.21.15.185:52056) 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 -- 172.21.15.185:0/945006394 learned_addr learned my addr 172.21.15.185:0/945006394 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010bfff700 1 --2- 172.21.15.185:0/945006394 >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 0x7f010c13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58614/0 (socket says 172.21.15.185:58614) 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010b7fe700 1 --1- 172.21.15.185:0/945006394 >> v1:172.21.15.185:6789/0 conn(0x7f010c0b3830 0x7f010c143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52048/0 (socket says 172.21.15.185:52048) 2020-11-16T22:16:25.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 -- 172.21.15.185:0/945006394 >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 msgr2=0x7f010c13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:25.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 --2- 172.21.15.185:0/945006394 >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 0x7f010c13fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 -- 172.21.15.185:0/945006394 >> v1:172.21.15.185:6789/0 conn(0x7f010c0b3830 legacy=0x7f010c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 -- 172.21.15.185:0/945006394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c060e40 con 0x7f010c0ac820 2020-11-16T22:16:25.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010affd700 1 --2- 172.21.15.185:0/945006394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0100006fd0 tx=0x7f0100006eb0).ready entity=mon.2 client_cookie=4d8261af1fae1fb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010a7fc700 1 -- 172.21.15.185:0/945006394 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100009810 con 0x7f010c0ac820 2020-11-16T22:16:25.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010a7fc700 1 -- 172.21.15.185:0/945006394 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100009970 con 0x7f010c0ac820 2020-11-16T22:16:25.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f010a7fc700 1 -- 172.21.15.185:0/945006394 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100009cf0 con 0x7f010c0ac820 2020-11-16T22:16:25.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- 172.21.15.185:0/945006394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 msgr2=0x7f010c13f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 --2- 172.21.15.185:0/945006394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0100006fd0 tx=0x7f0100006eb0).stop 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- 172.21.15.185:0/945006394 shutdown_connections 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 --2- 172.21.15.185:0/945006394 >> v2:172.21.15.185:3300/0 conn(0x7f010c060130 0x7f010c13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 --2- 172.21.15.185:0/945006394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c0ac820 0x7f010c13f840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- 172.21.15.185:0/945006394 >> 172.21.15.185:0/945006394 conn(0x7f010c063df0 msgr2=0x7f010c0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- 172.21.15.185:0/945006394 shutdown_connections 2020-11-16T22:16:25.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.859+0000 7f0110deb700 1 -- 172.21.15.185:0/945006394 wait complete. 2020-11-16T22:16:25.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 Processor -- start 2020-11-16T22:16:25.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- start start 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 0x7f010c0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f010c0ac820 0x7f010c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c14fe70 con 0x7f010c0ac820 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c150110 con 0x7f010c0b3830 2020-11-16T22:16:25.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c143da0 con 0x7f010c060130 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 0x7f010c0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 0x7f010c0b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52062/0 (socket says 172.21.15.185:52062) 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58620/0 (socket says 172.21.15.185:58620) 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010b7fe700 1 -- 172.21.15.185:0/2727120024 learned_addr learned my addr 172.21.15.185:0/2727120024 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010affd700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f010c0ac820 0x7f010c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 -- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 msgr2=0x7f010c0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 0x7f010c0b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 -- 172.21.15.185:0/2727120024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f010c0ac820 msgr2=0x7f010c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f010c0ac820 0x7f010c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 -- 172.21.15.185:0/2727120024 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c1426c0 con 0x7f010c0b3830 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f010bfff700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f010400e950 tx=0x7f0104006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010400c6b0 con 0x7f010c0b3830 2020-11-16T22:16:25.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010400c810 con 0x7f010c0b3830 2020-11-16T22:16:25.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1435c0 con 0x7f010c0b3830 2020-11-16T22:16:25.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010400cbd0 con 0x7f010c0b3830 2020-11-16T22:16:25.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.863+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c0afd00 con 0x7f010c0b3830 2020-11-16T22:16:25.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c05d200 con 0x7f010c0b3830 2020-11-16T22:16:25.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f010401a030 con 0x7f010c0b3830 2020-11-16T22:16:25.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f00eb7fe700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 0x7f00e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:25.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f010406d8e0 con 0x7f010c0b3830 2020-11-16T22:16:25.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f010b7fe700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 0x7f00e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:25.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.867+0000 7f010b7fe700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 0x7f00e4057520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f00fc000f40 tx=0x7f00fc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:25.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:25.875+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0104043b40 con 0x7f010c0b3830 2020-11-16T22:16:26.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f010c061e20 con 0x7f010c0b3830 2020-11-16T22:16:26.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f00eb7fe700 1 -- 172.21.15.185:0/2727120024 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0104047160 con 0x7f010c0b3830 2020-11-16T22:16:26.007 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:16:26.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 msgr2=0x7f00e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 0x7f00e4057520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f00fc000f40 tx=0x7f00fc009040).stop 2020-11-16T22:16:26.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 msgr2=0x7f010c0af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f010400e950 tx=0x7f0104006df0).stop 2020-11-16T22:16:26.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 shutdown_connections 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f00e4055060 0x7f00e4057520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f010c0b3830 0x7f010c0af810 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f010c060130 0x7f010c0b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 --2- 172.21.15.185:0/2727120024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f010c0ac820 0x7f010c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 >> 172.21.15.185:0/2727120024 conn(0x7f010c063df0 msgr2=0x7f010c13c420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 shutdown_connections 2020-11-16T22:16:26.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.023+0000 7f0110deb700 1 -- 172.21.15.185:0/2727120024 wait complete. 2020-11-16T22:16:26.021 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:16:26.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 Processor -- start 2020-11-16T22:16:26.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 -- start start 2020-11-16T22:16:26.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f20d4140cc0 0x7f20d4141130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40b2690 con 0x7f20d40b38a0 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40ad350 con 0x7f20d4140cc0 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20db35e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b2930 con 0x7f20d405bc50 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52070/0 (socket says 172.21.15.185:52070) 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20da35c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f20d40b38a0 0x7f20d4144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52062/0 (socket says 172.21.15.185:52062) 2020-11-16T22:16:26.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 -- 172.21.15.185:0/1730729769 learned_addr learned my addr 172.21.15.185:0/1730729769 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:26.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20dab5d700 1 --2- 172.21.15.185:0/1730729769 >> v2:172.21.15.96:3300/0 conn(0x7f20d4140cc0 0x7f20d4141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 -- 172.21.15.185:0/1730729769 >> v2:172.21.15.96:3300/0 conn(0x7f20d4140cc0 msgr2=0x7f20d4141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:26.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 --2- 172.21.15.185:0/1730729769 >> v2:172.21.15.96:3300/0 conn(0x7f20d4140cc0 0x7f20d4141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 -- 172.21.15.185:0/1730729769 >> v1:172.21.15.185:6789/0 conn(0x7f20d40b38a0 legacy=0x7f20d4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 -- 172.21.15.185:0/1730729769 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d41443d0 con 0x7f20d405bc50 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d9b5b700 1 --2- 172.21.15.185:0/1730729769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f20c8009d50 tx=0x7f20c8009290).ready entity=mon.2 client_cookie=d8b9c56f35be8f87 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d935a700 1 -- 172.21.15.185:0/1730729769 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c800bc80 con 0x7f20d405bc50 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d935a700 1 -- 172.21.15.185:0/1730729769 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c800bde0 con 0x7f20d405bc50 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.163+0000 7f20d935a700 1 -- 172.21.15.185:0/1730729769 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8011610 con 0x7f20d405bc50 2020-11-16T22:16:26.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- 172.21.15.185:0/1730729769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 msgr2=0x7f20d40b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- 172.21.15.185:0/1730729769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f20c8009d50 tx=0x7f20c8009290).stop 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- 172.21.15.185:0/1730729769 shutdown_connections 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- 172.21.15.185:0/1730729769 >> v2:172.21.15.96:3300/0 conn(0x7f20d4140cc0 0x7f20d4141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- 172.21.15.185:0/1730729769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d405bc50 0x7f20d40b2060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- 172.21.15.185:0/1730729769 >> 172.21.15.185:0/1730729769 conn(0x7f20d4063df0 msgr2=0x7f20d40641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- 172.21.15.185:0/1730729769 shutdown_connections 2020-11-16T22:16:26.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- 172.21.15.185:0/1730729769 wait complete. 2020-11-16T22:16:26.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 Processor -- start 2020-11-16T22:16:26.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- start start 2020-11-16T22:16:26.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20d40b38a0 0x7f20d40ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 0x7f20d40ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b09d0 con 0x7f20d40b38a0 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b0c70 con 0x7f20d405bc50 2020-11-16T22:16:26.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20db35e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40b0f10 con 0x7f20d4140cc0 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58630/0 (socket says 172.21.15.185:58630) 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 -- 172.21.15.185:0/2745322635 learned_addr learned my addr 172.21.15.185:0/2745322635 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20dab5d700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 0x7f20d40ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20d9b5b700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20d40b38a0 0x7f20d40ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 -- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 msgr2=0x7f20d40ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 0x7f20d40ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 -- 172.21.15.185:0/2745322635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20d40b38a0 msgr2=0x7f20d40ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20d40b38a0 0x7f20d40ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 -- 172.21.15.185:0/2745322635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40b11b0 con 0x7f20d405bc50 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20dab5d700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 0x7f20d40ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.167+0000 7f20da35c700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f20c400c710 tx=0x7f20c4007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40072e0 con 0x7f20d405bc50 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4005530 con 0x7f20d405bc50 2020-11-16T22:16:26.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d4135dd0 con 0x7f20d405bc50 2020-11-16T22:16:26.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40058b0 con 0x7f20d405bc50 2020-11-16T22:16:26.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d412d620 con 0x7f20d405bc50 2020-11-16T22:16:26.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d405d200 con 0x7f20d405bc50 2020-11-16T22:16:26.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f20c401a060 con 0x7f20d405bc50 2020-11-16T22:16:26.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 0x7f20ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20c406d480 con 0x7f20d405bc50 2020-11-16T22:16:26.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d9b5b700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 0x7f20ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:26.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.171+0000 7f20d9b5b700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 0x7f20ac057520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f20d412d100 tx=0x7f20c8016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:26.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.175+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f20c40436e0 con 0x7f20d405bc50 2020-11-16T22:16:26.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.327+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20d40b1340 con 0x7f20d405bc50 2020-11-16T22:16:26.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.327+0000 7f20d27fc700 1 -- 172.21.15.185:0/2745322635 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20c4018020 con 0x7f20d405bc50 2020-11-16T22:16:26.313 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 msgr2=0x7f20ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 0x7f20ac057520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f20d412d100 tx=0x7f20c8016000).stop 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 msgr2=0x7f20d41358e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f20c400c710 tx=0x7f20c4007c40).stop 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 shutdown_connections 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f20ac055060 0x7f20ac057520 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20d405bc50 0x7f20d41358e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20d4140cc0 0x7f20d40ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 --2- 172.21.15.185:0/2745322635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20d40b38a0 0x7f20d40ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 >> 172.21.15.185:0/2745322635 conn(0x7f20d4063df0 msgr2=0x7f20d40aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 shutdown_connections 2020-11-16T22:16:26.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.331+0000 7f20db35e700 1 -- 172.21.15.185:0/2745322635 wait complete. 2020-11-16T22:16:26.328 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:16:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 Processor -- start 2020-11-16T22:16:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 -- start start 2020-11-16T22:16:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f36880b5d70 0x7f36880ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f36880b4470 0x7f36880b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36880b7290 con 0x7f36881475d0 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36880b4bf0 con 0x7f36880b5d70 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368dd2a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36880b74c0 con 0x7f36880b4470 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368d529700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f36881475d0 0x7f3688147a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52080/0 (socket says 172.21.15.185:52080) 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368d529700 1 -- 172.21.15.185:0/1196540362 learned_addr learned my addr 172.21.15.185:0/1196540362 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 --2- 172.21.15.185:0/1196540362 >> v2:172.21.15.185:3300/0 conn(0x7f36880b4470 0x7f36880b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f368cd28700 1 --2- 172.21.15.185:0/1196540362 >> v2:172.21.15.96:3300/0 conn(0x7f36880b5d70 0x7f36880ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f36877fe700 1 -- 172.21.15.185:0/1196540362 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627780364 0 0) 0x7f36880b7290 con 0x7f36881475d0 2020-11-16T22:16:30.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f36877fe700 1 -- 172.21.15.185:0/1196540362 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3668003240 con 0x7f36881475d0 2020-11-16T22:16:30.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 -- 172.21.15.185:0/1196540362 >> v2:172.21.15.96:3300/0 conn(0x7f36880b5d70 msgr2=0x7f36880ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:30.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 --2- 172.21.15.185:0/1196540362 >> v2:172.21.15.96:3300/0 conn(0x7f36880b5d70 0x7f36880ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 -- 172.21.15.185:0/1196540362 >> v1:172.21.15.185:6789/0 conn(0x7f36881475d0 legacy=0x7f3688147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 -- 172.21.15.185:0/1196540362 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36880b7760 con 0x7f36880b4470 2020-11-16T22:16:30.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f3687fff700 1 --2- 172.21.15.185:0/1196540362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f36880b7c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f3688080bf0 tx=0x7f3678006df0).ready entity=mon.1 client_cookie=11365b4d4a6f6036 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f36877fe700 1 -- 172.21.15.185:0/1196540362 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678009640 con 0x7f36880b4470 2020-11-16T22:16:30.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f36877fe700 1 -- 172.21.15.185:0/1196540362 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780097a0 con 0x7f36880b4470 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.471+0000 7f36877fe700 1 -- 172.21.15.185:0/1196540362 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678009b20 con 0x7f36880b4470 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- 172.21.15.185:0/1196540362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 msgr2=0x7f36880b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 --2- 172.21.15.185:0/1196540362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f36880b7c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f3688080bf0 tx=0x7f3678006df0).stop 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- 172.21.15.185:0/1196540362 shutdown_connections 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 --2- 172.21.15.185:0/1196540362 >> v2:172.21.15.96:3300/0 conn(0x7f36880b5d70 0x7f36880ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 --2- 172.21.15.185:0/1196540362 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f36880b7c00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- 172.21.15.185:0/1196540362 >> 172.21.15.185:0/1196540362 conn(0x7f368805ba80 msgr2=0x7f368805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- 172.21.15.185:0/1196540362 shutdown_connections 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- 172.21.15.185:0/1196540362 wait complete. 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 Processor -- start 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.475+0000 7f368dd2a700 1 -- start start 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f3688146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 0x7f36881535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36881475d0 0x7f3688153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36880afa40 con 0x7f36881475d0 2020-11-16T22:16:30.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688153f80 con 0x7f36880b4470 2020-11-16T22:16:30.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36880aed50 con 0x7f36880b5d70 2020-11-16T22:16:30.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368cd28700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f3688146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 0x7f36881535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368cd28700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f3688146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58644/0 (socket says 172.21.15.185:58644) 2020-11-16T22:16:30.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368cd28700 1 -- 172.21.15.185:0/2512335070 learned_addr learned my addr 172.21.15.185:0/2512335070 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:30.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368d529700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36881475d0 0x7f3688153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 -- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 msgr2=0x7f3688146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f3688146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 -- 172.21.15.185:0/2512335070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36881475d0 msgr2=0x7f3688153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36881475d0 0x7f3688153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 -- 172.21.15.185:0/2512335070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36880aeff0 con 0x7f36880b5d70 2020-11-16T22:16:30.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3687fff700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 0x7f36881535a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3678012000 tx=0x7f3678015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678011c40 con 0x7f36880b5d70 2020-11-16T22:16:30.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678011da0 con 0x7f36880b5d70 2020-11-16T22:16:30.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36880af2f0 con 0x7f36880b5d70 2020-11-16T22:16:30.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367801d680 con 0x7f36880b5d70 2020-11-16T22:16:30.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36880ad000 con 0x7f36880b5d70 2020-11-16T22:16:30.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.479+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368805d200 con 0x7f36880b5d70 2020-11-16T22:16:30.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.483+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3678023030 con 0x7f36880b5d70 2020-11-16T22:16:30.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.483+0000 7f3684ff9700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 0x7f3664057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.483+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f367806ec40 con 0x7f36880b5d70 2020-11-16T22:16:30.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.483+0000 7f368cd28700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 0x7f3664057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.483+0000 7f368cd28700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 0x7f3664057520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3670006a90 tx=0x7f3670006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.487+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3678044d90 con 0x7f36880b5d70 2020-11-16T22:16:30.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36880b80f0 con 0x7f36880b5d70 2020-11-16T22:16:30.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f3684ff9700 1 -- 172.21.15.185:0/2512335070 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f36780483b0 con 0x7f36880b5d70 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 msgr2=0x7f3664057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 0x7f3664057520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3670006a90 tx=0x7f3670006df0).stop 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 msgr2=0x7f36881535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.643+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 0x7f36881535a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3678012000 tx=0x7f3678015000).stop 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 shutdown_connections 2020-11-16T22:16:30.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3664055060 0x7f3664057520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f36880b4470 0x7f3688146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36881475d0 0x7f3688153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 --2- 172.21.15.185:0/2512335070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f36880b5d70 0x7f36881535a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 >> 172.21.15.185:0/2512335070 conn(0x7f368805ba80 msgr2=0x7f36880b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 shutdown_connections 2020-11-16T22:16:30.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:26.647+0000 7f368dd2a700 1 -- 172.21.15.185:0/2512335070 wait complete. 2020-11-16T22:16:30.632 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:30.632 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:16:30.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.787+0000 7f053a741700 1 Processor -- start 2020-11-16T22:16:30.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.787+0000 7f053a741700 1 -- start start 2020-11-16T22:16:30.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05340b3830 0x7f0534143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0534060130 0x7f0534143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0534060970 con 0x7f05340ac820 2020-11-16T22:16:30.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340b4610 con 0x7f05340b3830 2020-11-16T22:16:30.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534060ba0 con 0x7f0534060130 2020-11-16T22:16:30.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05340b3830 0x7f0534143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0538f3e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f05340ac820 0x7f053413f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52092/0 (socket says 172.21.15.185:52092) 2020-11-16T22:16:30.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0538f3e700 1 -- 172.21.15.185:0/3524622874 learned_addr learned my addr 172.21.15.185:0/3524622874 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:30.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0539f40700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0534060130 0x7f0534143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 -- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0534060130 msgr2=0x7f0534143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:30.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0534060130 0x7f0534143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 -- 172.21.15.185:0/3524622874 >> v1:172.21.15.185:6789/0 conn(0x7f05340ac820 legacy=0x7f053413f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0523fff700 1 -- 172.21.15.185:0/3524622874 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370051648 0 0) 0x7f0534060970 con 0x7f05340ac820 2020-11-16T22:16:30.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 -- 172.21.15.185:0/3524622874 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534060e40 con 0x7f05340b3830 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053973f700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05340b3830 0x7f0534143a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f053000dc30 tx=0x7f0530006d80).ready entity=mon.1 client_cookie=11432bec5ce50b39 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0523fff700 1 -- 172.21.15.185:0/3524622874 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000bcb0 con 0x7f05340b3830 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0523fff700 1 -- 172.21.15.185:0/3524622874 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053000be10 con 0x7f05340b3830 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f0523fff700 1 -- 172.21.15.185:0/3524622874 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530015600 con 0x7f05340b3830 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05340b3830 msgr2=0x7f0534143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05340b3830 0x7f0534143a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f053000dc30 tx=0x7f0530006d80).stop 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- 172.21.15.185:0/3524622874 shutdown_connections 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0534060130 0x7f0534143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 --2- 172.21.15.185:0/3524622874 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05340b3830 0x7f0534143a40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- 172.21.15.185:0/3524622874 >> 172.21.15.185:0/3524622874 conn(0x7f0534063df0 msgr2=0x7f05340641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- 172.21.15.185:0/3524622874 shutdown_connections 2020-11-16T22:16:30.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.791+0000 7f053a741700 1 -- 172.21.15.185:0/3524622874 wait complete. 2020-11-16T22:16:30.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.795+0000 7f053a741700 1 Processor -- start 2020-11-16T22:16:30.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.795+0000 7f053a741700 1 -- start start 2020-11-16T22:16:30.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 0x7f05340afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534110980 con 0x7f05340ac820 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534110c20 con 0x7f0534060130 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340af970 con 0x7f05340b3830 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053973f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053973f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58658/0 (socket says 172.21.15.185:58658) 2020-11-16T22:16:30.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053973f700 1 -- 172.21.15.185:0/4274940236 learned_addr learned my addr 172.21.15.185:0/4274940236 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:30.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52106/0 (socket says 172.21.15.185:52106) 2020-11-16T22:16:30.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0538f3e700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 0x7f05340afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 -- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 msgr2=0x7f0534142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 -- 172.21.15.185:0/4274940236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 msgr2=0x7f05340afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 0x7f05340afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 -- 172.21.15.185:0/4274940236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534143300 con 0x7f05340b3830 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0538f3e700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 0x7f05340afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053973f700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f0539f40700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f052c00ef50 tx=0x7f052c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0095f0 con 0x7f05340b3830 2020-11-16T22:16:30.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00cb30 con 0x7f05340b3830 2020-11-16T22:16:30.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534110ec0 con 0x7f05340b3830 2020-11-16T22:16:30.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0123f0 con 0x7f05340b3830 2020-11-16T22:16:30.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341525a0 con 0x7f05340b3830 2020-11-16T22:16:30.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0534152820 con 0x7f05340b3830 2020-11-16T22:16:30.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.799+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f052c022070 con 0x7f05340b3830 2020-11-16T22:16:30.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.803+0000 7f05217fa700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 0x7f050c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:30.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.803+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f052c06e4a0 con 0x7f05340b3830 2020-11-16T22:16:30.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.803+0000 7f053973f700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 0x7f050c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:30.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.803+0000 7f053973f700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 0x7f050c057520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0530006dc0 tx=0x7f0530019000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:30.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.807+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f052c044700 con 0x7f05340b3830 2020-11-16T22:16:30.944 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.959+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f053405d200 con 0x7f05340b3830 2020-11-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.959+0000 7f05217fa700 1 -- 172.21.15.185:0/4274940236 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f052c016020 con 0x7f05340b3830 2020-11-16T22:16:30.945 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:16:30.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 msgr2=0x7f050c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 0x7f050c057520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0530006dc0 tx=0x7f0530019000).stop 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 msgr2=0x7f053413a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f052c00ef50 tx=0x7f052c0091d0).stop 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 shutdown_connections 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f050c055060 0x7f050c057520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0534060130 0x7f0534142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05340b3830 0x7f053413a8c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 --2- 172.21.15.185:0/4274940236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05340ac820 0x7f05340afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 >> 172.21.15.185:0/4274940236 conn(0x7f0534063df0 msgr2=0x7f053413e960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 shutdown_connections 2020-11-16T22:16:30.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:30.963+0000 7f053a741700 1 -- 172.21.15.185:0/4274940236 wait complete. 2020-11-16T22:16:30.964 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:16:30.965 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:31.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 Processor -- start 2020-11-16T22:16:31.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 -- start start 2020-11-16T22:16:31.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe5740b17e0 0x7fe574067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b2160 0x7fe57414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 0x7fe574065630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe574098cc0 con 0x7fe5740b17e0 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740b3950 con 0x7fe574066240 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740b3bf0 con 0x7fe5740b2160 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe572d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 0x7fe574065630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe572d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 0x7fe574065630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58670/0 (socket says 172.21.15.185:58670) 2020-11-16T22:16:31.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe571d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b2160 0x7fe57414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe572d9d700 1 -- 172.21.15.185:0/1957165170 learned_addr learned my addr 172.21.15.185:0/1957165170 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 --2- 172.21.15.185:0/1957165170 >> v2:172.21.15.96:3300/0 conn(0x7fe5740b17e0 0x7fe574067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 -- 172.21.15.185:0/1957165170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b2160 msgr2=0x7fe57414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 --2- 172.21.15.185:0/1957165170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b2160 0x7fe57414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 -- 172.21.15.185:0/1957165170 >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 msgr2=0x7fe574065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:31.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 --2- 172.21.15.185:0/1957165170 >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 0x7fe574065630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 -- 172.21.15.185:0/1957165170 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574067b10 con 0x7fe5740b17e0 2020-11-16T22:16:31.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57259c700 1 --2- 172.21.15.185:0/1957165170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5740b17e0 0x7fe574067fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe5740b65a0 tx=0x7fe56c0091d0).ready entity=mon.0 client_cookie=5df48851ca72c4ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:31.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57159a700 1 -- 172.21.15.185:0/1957165170 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00b9c0 con 0x7fe5740b17e0 2020-11-16T22:16:31.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57159a700 1 -- 172.21.15.185:0/1957165170 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c00bb20 con 0x7fe5740b17e0 2020-11-16T22:16:31.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe57159a700 1 -- 172.21.15.185:0/1957165170 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00bea0 con 0x7fe5740b17e0 2020-11-16T22:16:31.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 -- 172.21.15.185:0/1957165170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5740b17e0 msgr2=0x7fe574067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.099+0000 7fe578ff4700 1 --2- 172.21.15.185:0/1957165170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5740b17e0 0x7fe574067fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe5740b65a0 tx=0x7fe56c0091d0).stop 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/1957165170 shutdown_connections 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- 172.21.15.185:0/1957165170 >> v2:172.21.15.185:3300/0 conn(0x7fe574066240 0x7fe574065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- 172.21.15.185:0/1957165170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5740b17e0 0x7fe574067fb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- 172.21.15.185:0/1957165170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b2160 0x7fe57414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/1957165170 >> 172.21.15.185:0/1957165170 conn(0x7fe5740604a0 msgr2=0x7fe5740608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/1957165170 shutdown_connections 2020-11-16T22:16:31.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/1957165170 wait complete. 2020-11-16T22:16:31.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 Processor -- start 2020-11-16T22:16:31.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- start start 2020-11-16T22:16:31.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 0x7fe574147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5740b2160 0x7fe5741587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740ac2b0 con 0x7fe574066240 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740ac550 con 0x7fe5740b2160 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740ac7f0 con 0x7fe5740b17e0 2020-11-16T22:16:31.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52118/0 (socket says 172.21.15.185:52118) 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 -- 172.21.15.185:0/2766544065 learned_addr learned my addr 172.21.15.185:0/2766544065 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe572d9d700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5740b2160 0x7fe5741587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe57259c700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 0x7fe574147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 -- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5740b2160 msgr2=0x7fe5741587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5740b2160 0x7fe5741587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 -- 172.21.15.185:0/2766544065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 msgr2=0x7fe574147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 0x7fe574147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 -- 172.21.15.185:0/2766544065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57415b2f0 con 0x7fe5740b17e0 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe57259c700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 0x7fe574147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe571d9b700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe56800eba0 tx=0x7fe568006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800c9b0 con 0x7fe5740b17e0 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56800cb10 con 0x7fe5740b17e0 2020-11-16T22:16:31.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5741480f0 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800ce90 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.103+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe574157e90 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe568012600 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5740683f0 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe5627fc700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 0x7fe548057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe56806e050 con 0x7fe5740b17e0 2020-11-16T22:16:31.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe57259c700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 0x7fe548057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:31.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.107+0000 7fe57259c700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 0x7fe548057520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe56c009fd0 tx=0x7fe56c016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:31.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.115+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe5680442b0 con 0x7fe5740b17e0 2020-11-16T22:16:31.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.319+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe574157e90 con 0x7fe5740b17e0 2020-11-16T22:16:31.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe5627fc700 1 -- 172.21.15.185:0/2766544065 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2487 (secure 0 0 0) 0x7fe574157e90 con 0x7fe5740b17e0 2020-11-16T22:16:31.306 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:31.307 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5183,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:19.390162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:31.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 msgr2=0x7fe548057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:31.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 0x7fe548057520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe56c009fd0 tx=0x7fe56c016000).stop 2020-11-16T22:16:31.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 msgr2=0x7fe574158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe56800eba0 tx=0x7fe568006df0).stop 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 shutdown_connections 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe548055060 0x7fe548057520 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5740b2160 0x7fe5741587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe574066240 0x7fe574147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 --2- 172.21.15.185:0/2766544065 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5740b17e0 0x7fe574158300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:31.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 >> 172.21.15.185:0/2766544065 conn(0x7fe5740604a0 msgr2=0x7fe574146e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:31.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 shutdown_connections 2020-11-16T22:16:31.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:31.323+0000 7fe578ff4700 1 -- 172.21.15.185:0/2766544065 wait complete. 2020-11-16T22:16:31.310 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 31 2020-11-16T22:16:31.624 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:16:18.387140+0000', 'modified': '2020-11-16T22:16:19.390162+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5542': {'gid': 5542, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6834/653581747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 653581747}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 653581747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:16:31.624 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:16:31.625 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:16:31.763 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":5542,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.383208736,"mdsmap_epoch":31,"osdmap_epoch":44,"osdmap_epoch_barrier":42,"uptime":22.561336093000001} 2020-11-16T22:16:31.764 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5542, "mdsmap_epoch": 31, "osdmap_epoch": 44, "osdmap_epoch_barrier": 42, "rank_uptime": 13.383208736, "state": "up:active", "uptime": 22.561336093, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:16:31.765 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:16:31.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:31.765 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:16:31.784 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:16:31.784 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:16:31.854 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:16:31.854 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:16:31.878 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:16:31.879 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:16:31.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:16:31.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:16:31.888 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:16:31.919 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:16:32.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 Processor -- start 2020-11-16T22:16:32.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 -- start start 2020-11-16T22:16:32.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbd40b1dc0 0x7fdbd40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:32.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd413e400 con 0x7fdbd40b2e80 2020-11-16T22:16:32.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd40ac070 con 0x7fdbd40b3920 2020-11-16T22:16:32.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd3fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd40ac310 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd27fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fdbd40b3920 0x7fdbd40b16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52118/0 (socket says 172.21.15.185:52118) 2020-11-16T22:16:32.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd27fc700 1 -- 172.21.15.185:0/1824111045 learned_addr learned my addr 172.21.15.185:0/1824111045 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:32.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd37fe700 1 --2- 172.21.15.185:0/1824111045 >> v2:172.21.15.96:3300/0 conn(0x7fdbd40b1dc0 0x7fdbd40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:32.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994449096 0 0) 0x7fdbd40ac070 con 0x7fdbd40b3920 2020-11-16T22:16:32.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.071+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb8003240 con 0x7fdbd40b3920 2020-11-16T22:16:32.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd37fe700 1 -- 172.21.15.185:0/1824111045 >> v1:172.21.15.185:6789/0 conn(0x7fdbd40b3920 legacy=0x7fdbd40b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4159058811 0 0) 0x7fdbb8003240 con 0x7fdbd40b3920 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd37fe700 1 -- 172.21.15.185:0/1824111045 >> v1:172.21.15.96:6789/0 conn(0x7fdbd40b2e80 legacy=0x7fdbd413dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd37fe700 1 -- 172.21.15.185:0/1824111045 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40ac5b0 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd37fe700 1 --2- 172.21.15.185:0/1824111045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b1dc0 0x7fdbd40b3260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fdbcc00b9a0 tx=0x7fdbcc006df0).ready entity=mon.0 client_cookie=4f3aa305e860899a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc001520 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc012460 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd1ffb700 1 -- 172.21.15.185:0/1824111045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0127e0 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- 172.21.15.185:0/1824111045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b1dc0 msgr2=0x7fdbd40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:32.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/1824111045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b1dc0 0x7fdbd40b3260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fdbcc00b9a0 tx=0x7fdbcc006df0).stop 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- 172.21.15.185:0/1824111045 shutdown_connections 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/1824111045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b1dc0 0x7fdbd40b3260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- 172.21.15.185:0/1824111045 >> 172.21.15.185:0/1824111045 conn(0x7fdbd4060130 msgr2=0x7fdbd4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- 172.21.15.185:0/1824111045 shutdown_connections 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- 172.21.15.185:0/1824111045 wait complete. 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 Processor -- start 2020-11-16T22:16:32.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.075+0000 7fdbd3fff700 1 -- start start 2020-11-16T22:16:32.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:32.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 0x7fdbd41434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:32.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 0x7fdbd41439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4144c70 con 0x7fdbd40b2e80 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41468c0 con 0x7fdbd40b3920 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4144290 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 0x7fdbd41439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52130/0 (socket says 172.21.15.185:52130) 2020-11-16T22:16:32.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 -- 172.21.15.185:0/809871191 learned_addr learned my addr 172.21.15.185:0/809871191 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd27fc700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 0x7fdbd41434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 -- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 msgr2=0x7fdbd41439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 0x7fdbd41439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 -- 172.21.15.185:0/809871191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 msgr2=0x7fdbd41434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 0x7fdbd41434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 -- 172.21.15.185:0/809871191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4144530 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd27fc700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 0x7fdbd41434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd37fe700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 0x7fdbd41439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd2ffd700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fdbc400cb90 tx=0x7fdbc4007f50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400a790 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc400a8f0 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd4144830 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400acb0 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd41465b0 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.079+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd405d200 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.083+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdbc4010070 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.083+0000 7fdbcaffd700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 0x7fdba4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.083+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbc406db70 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.083+0000 7fdbd27fc700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 0x7fdba4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.083+0000 7fdbd27fc700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 0x7fdba4057520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fdbd40b72f0 tx=0x7fdbbc001dd0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:32.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.091+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdbc4043d70 con 0x7fdbd40b1dc0 2020-11-16T22:16:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:32.287+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fdbd4061540 con 0x7fdbd40b1dc0 2020-11-16T22:16:33.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.183+0000 7fdbcaffd700 1 -- 172.21.15.185:0/809871191 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v32) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdbc4047390 con 0x7fdbd40b1dc0 2020-11-16T22:16:33.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.187+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 msgr2=0x7fdba4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.187+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 0x7fdba4057520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fdbd40b72f0 tx=0x7fdbbc001dd0).stop 2020-11-16T22:16:33.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.187+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 msgr2=0x7fdbd4141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.187+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fdbc400cb90 tx=0x7fdbc4007f50).stop 2020-11-16T22:16:33.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 shutdown_connections 2020-11-16T22:16:33.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdba4055060 0x7fdba4057520 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdbd40b3920 0x7fdbd41439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdbd40b1dc0 0x7fdbd4141e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 --2- 172.21.15.185:0/809871191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd40b2e80 0x7fdbd41434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 >> 172.21.15.185:0/809871191 conn(0x7fdbd4060130 msgr2=0x7fdbd40b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 shutdown_connections 2020-11-16T22:16:33.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.191+0000 7fdbd3fff700 1 -- 172.21.15.185:0/809871191 wait complete. 2020-11-16T22:16:33.193 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:33.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 Processor -- start 2020-11-16T22:16:33.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- start start 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f60440b17e0 0x7f6044067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6044098cc0 con 0x7f60440b2160 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60440b3950 con 0x7f6044066240 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604414c4f0 con 0x7f60440b17e0 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6043fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6044066240 0x7f6044065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52136/0 (socket says 172.21.15.185:52136) 2020-11-16T22:16:33.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6043fff700 1 -- 172.21.15.185:0/1201535865 learned_addr learned my addr 172.21.15.185:0/1201535865 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:33.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60437fe700 1 --2- 172.21.15.185:0/1201535865 >> v2:172.21.15.96:3300/0 conn(0x7f60440b17e0 0x7f6044067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60427fc700 1 -- 172.21.15.185:0/1201535865 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237449339 0 0) 0x7f60440b3950 con 0x7f6044066240 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60427fc700 1 -- 172.21.15.185:0/1201535865 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6024003240 con 0x7f6044066240 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60437fe700 1 -- 172.21.15.185:0/1201535865 >> v1:172.21.15.185:6789/0 conn(0x7f6044066240 legacy=0x7f6044065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60437fe700 1 -- 172.21.15.185:0/1201535865 >> v1:172.21.15.96:6789/0 conn(0x7f60440b2160 legacy=0x7f604414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60437fe700 1 -- 172.21.15.185:0/1201535865 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440b3bf0 con 0x7f60440b17e0 2020-11-16T22:16:33.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60437fe700 1 --2- 172.21.15.185:0/1201535865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60440b17e0 0x7f6044067fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f602c00b9a0 tx=0x7f602c006df0).ready entity=mon.0 client_cookie=c4aacf50c17978fd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60427fc700 1 -- 172.21.15.185:0/1201535865 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c0097b0 con 0x7f60440b17e0 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60427fc700 1 -- 172.21.15.185:0/1201535865 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c009910 con 0x7f60440b17e0 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f60427fc700 1 -- 172.21.15.185:0/1201535865 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c009c90 con 0x7f60440b17e0 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- 172.21.15.185:0/1201535865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60440b17e0 msgr2=0x7f6044067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 --2- 172.21.15.185:0/1201535865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60440b17e0 0x7f6044067fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f602c00b9a0 tx=0x7f602c006df0).stop 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- 172.21.15.185:0/1201535865 shutdown_connections 2020-11-16T22:16:33.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 --2- 172.21.15.185:0/1201535865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60440b17e0 0x7f6044067fb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- 172.21.15.185:0/1201535865 >> 172.21.15.185:0/1201535865 conn(0x7f60440604a0 msgr2=0x7f60440608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:33.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.323+0000 7f6049b52700 1 -- 172.21.15.185:0/1201535865 shutdown_connections 2020-11-16T22:16:33.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- 172.21.15.185:0/1201535865 wait complete. 2020-11-16T22:16:33.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 Processor -- start 2020-11-16T22:16:33.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- start start 2020-11-16T22:16:33.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044066240 0x7f604415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 0x7f6044096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044157b30 con 0x7f6044066240 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604415bff0 con 0x7f60440b2160 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60440ad7e0 con 0x7f60440b17e0 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:33.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58702/0 (socket says 172.21.15.185:58702) 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 -- 172.21.15.185:0/1913412886 learned_addr learned my addr 172.21.15.185:0/1913412886 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6042ffd700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 0x7f6044096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f60437fe700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044066240 0x7f604415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 -- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 msgr2=0x7f6044096e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 0x7f6044096e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 -- 172.21.15.185:0/1913412886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044066240 msgr2=0x7f604415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044066240 0x7f604415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 -- 172.21.15.185:0/1913412886 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440ada80 con 0x7f60440b2160 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6042ffd700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 0x7f6044096e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6043fff700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f603c00c6e0 tx=0x7f603c0079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0072e0 con 0x7f60440b2160 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c007440 con 0x7f60440b2160 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c005800 con 0x7f60440b2160 2020-11-16T22:16:33.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6044157d60 con 0x7f60440b2160 2020-11-16T22:16:33.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.327+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60440ae720 con 0x7f60440b2160 2020-11-16T22:16:33.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6044157ef0 con 0x7f60440b2160 2020-11-16T22:16:33.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f603c010070 con 0x7f60440b2160 2020-11-16T22:16:33.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f603b7fe700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:33.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f603c06d5a0 con 0x7f60440b2160 2020-11-16T22:16:33.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f60437fe700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:33.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.331+0000 7f60437fe700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 0x7f6020057520 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f602c006e30 tx=0x7f602c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:33.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.339+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f603c0436f0 con 0x7f60440b2160 2020-11-16T22:16:33.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.539+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604405d4e0 con 0x7f60440b2160 2020-11-16T22:16:33.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.539+0000 7f603b7fe700 1 -- 172.21.15.185:0/1913412886 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2503 (secure 0 0 0) 0x7f603c046d10 con 0x7f60440b2160 2020-11-16T22:16:33.526 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:33.526 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5183,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:33.189234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5542,"mds_1":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:creating","state_seq":2,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:33.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 msgr2=0x7f6020057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 0x7f6020057520 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f602c006e30 tx=0x7f602c015000).stop 2020-11-16T22:16:33.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 msgr2=0x7f6044158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:33.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f603c00c6e0 tx=0x7f603c0079e0).stop 2020-11-16T22:16:33.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 shutdown_connections 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6020055060 0x7f6020057520 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f60440b2160 0x7f6044158200 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044066240 0x7f604415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 --2- 172.21.15.185:0/1913412886 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f60440b17e0 0x7f6044096e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 >> 172.21.15.185:0/1913412886 conn(0x7f60440604a0 msgr2=0x7f6044060880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:33.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 shutdown_connections 2020-11-16T22:16:33.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:33.543+0000 7f6049b52700 1 -- 172.21.15.185:0/1913412886 wait complete. 2020-11-16T22:16:33.530 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 33 2020-11-16T22:16:33.843 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:16:18.387140+0000', 'modified': '2020-11-16T22:16:33.189234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5542, 'mds_1': 5475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 1, 'incarnation': 33, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.15.96:6835/4236989230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6833', 'nonce': 4236989230}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 4236989230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5542': {'gid': 5542, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6834/653581747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 653581747}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 653581747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:16:33.844 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5475:up:creating 2020-11-16T22:16:34.845 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:34.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.987+0000 7f2d59c0d700 1 Processor -- start 2020-11-16T22:16:34.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- start start 2020-11-16T22:16:34.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2d540682b0 0x7f2d540b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d54067510 con 0x7f2d540b7470 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540677b0 con 0x7f2d54067070 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540b17a0 con 0x7f2d540682b0 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2d540682b0 0x7f2d540b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2d540682b0 0x7f2d540b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58722/0 (socket says 172.21.15.185:58722) 2020-11-16T22:16:34.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 -- 172.21.15.185:0/4140349730 learned_addr learned my addr 172.21.15.185:0/4140349730 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d53fff700 1 --1- 172.21.15.185:0/4140349730 >> v1:172.21.15.185:6789/0 conn(0x7f2d54067070 0x7f2d540ac030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52158/0 (socket says 172.21.15.185:52158) 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 -- 172.21.15.185:0/4140349730 >> v1:172.21.15.185:6789/0 conn(0x7f2d54067070 legacy=0x7f2d540ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 -- 172.21.15.185:0/4140349730 >> v1:172.21.15.96:6789/0 conn(0x7f2d540b7470 legacy=0x7f2d540b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d537fe700 1 -- 172.21.15.185:0/4140349730 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121036449 0 0) 0x7f2d540677b0 con 0x7f2d54067070 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 -- 172.21.15.185:0/4140349730 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540b1a40 con 0x7f2d540682b0 2020-11-16T22:16:34.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d5940c700 1 --2- 172.21.15.185:0/4140349730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d540b7850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f2d4c009d50 tx=0x7f2d4c009290).ready entity=mon.1 client_cookie=febd37b17635c53e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:34.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d537fe700 1 -- 172.21.15.185:0/4140349730 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00bbc0 con 0x7f2d540682b0 2020-11-16T22:16:34.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d537fe700 1 -- 172.21.15.185:0/4140349730 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c00bd20 con 0x7f2d540682b0 2020-11-16T22:16:34.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d537fe700 1 -- 172.21.15.185:0/4140349730 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00f680 con 0x7f2d540682b0 2020-11-16T22:16:34.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- 172.21.15.185:0/4140349730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 msgr2=0x7f2d540b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:34.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/4140349730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d540b7850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f2d4c009d50 tx=0x7f2d4c009290).stop 2020-11-16T22:16:34.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- 172.21.15.185:0/4140349730 shutdown_connections 2020-11-16T22:16:34.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/4140349730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d540b7850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:34.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- 172.21.15.185:0/4140349730 >> 172.21.15.185:0/4140349730 conn(0x7f2d540637b0 msgr2=0x7f2d540b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:34.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- 172.21.15.185:0/4140349730 shutdown_connections 2020-11-16T22:16:34.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.991+0000 7f2d59c0d700 1 -- 172.21.15.185:0/4140349730 wait complete. 2020-11-16T22:16:34.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 Processor -- start 2020-11-16T22:16:34.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- start start 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d54067070 0x7f2d5414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 0x7f2d5413f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540aef40 con 0x7f2d54067070 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540ace50 con 0x7f2d540682b0 2020-11-16T22:16:34.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d541521c0 con 0x7f2d540b7470 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58726/0 (socket says 172.21.15.185:58726) 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 -- 172.21.15.185:0/947580991 learned_addr learned my addr 172.21.15.185:0/947580991 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d5940c700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 0x7f2d5413f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d58c0b700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d54067070 0x7f2d5414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 -- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 msgr2=0x7f2d5413f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:34.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 0x7f2d5413f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 -- 172.21.15.185:0/947580991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d54067070 msgr2=0x7f2d5414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d54067070 0x7f2d5414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 -- 172.21.15.185:0/947580991 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54152460 con 0x7f2d540682b0 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d5940c700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 0x7f2d5413f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d53fff700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f2d4000c710 tx=0x7f2d40007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400072e0 con 0x7f2d540682b0 2020-11-16T22:16:34.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d40005530 con 0x7f2d540682b0 2020-11-16T22:16:34.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5414ef60 con 0x7f2d540682b0 2020-11-16T22:16:34.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400058f0 con 0x7f2d540682b0 2020-11-16T22:16:34.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.995+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5413cd10 con 0x7f2d540682b0 2020-11-16T22:16:34.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5405d200 con 0x7f2d540682b0 2020-11-16T22:16:34.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2d4001a060 con 0x7f2d540682b0 2020-11-16T22:16:34.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d50ff9700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 0x7f2d2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:34.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d4006d500 con 0x7f2d540682b0 2020-11-16T22:16:34.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d58c0b700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 0x7f2d2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:34.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:34.999+0000 7f2d58c0b700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 0x7f2d2c057520 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f2d5414fb50 tx=0x7f2d48009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:34.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.003+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2d40043760 con 0x7f2d540682b0 2020-11-16T22:16:35.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.207+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d5413a7c0 con 0x7f2d540682b0 2020-11-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.207+0000 7f2d50ff9700 1 -- 172.21.15.185:0/947580991 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2516 (secure 0 0 0) 0x7f2d40018070 con 0x7f2d540682b0 2020-11-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:35.192 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:35.203561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:35.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 msgr2=0x7f2d2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 0x7f2d2c057520 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f2d5414fb50 tx=0x7f2d48009040).stop 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 msgr2=0x7f2d54152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f2d4000c710 tx=0x7f2d40007a60).stop 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 shutdown_connections 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2d2c055060 0x7f2d2c057520 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:35.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d540b7470 0x7f2d5413f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d54067070 0x7f2d5414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 --2- 172.21.15.185:0/947580991 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d540682b0 0x7f2d54152900 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 >> 172.21.15.185:0/947580991 conn(0x7f2d540637b0 msgr2=0x7f2d540b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 shutdown_connections 2020-11-16T22:16:35.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:35.211+0000 7f2d59c0d700 1 -- 172.21.15.185:0/947580991 wait complete. 2020-11-16T22:16:35.197 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 36 2020-11-16T22:16:35.509 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:16:18.387140+0000', 'modified': '2020-11-16T22:16:35.203561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5542, 'mds_1': 5475, 'mds_2': 5183}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5183': {'gid': 5183, 'name': 'a', 'rank': 2, 'incarnation': 35, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.185:6835/2473550168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 2473550168}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 2473550168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 1, 'incarnation': 33, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.96:6835/4236989230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6833', 'nonce': 4236989230}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 4236989230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_5542': {'gid': 5542, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6834/653581747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 653581747}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 653581747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:16:35.509 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:16:35.510 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:16:35.649 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":5183,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":1.4530650949999999,"mdsmap_epoch":36,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":26.445482575} 2020-11-16T22:16:35.650 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5183, "mdsmap_epoch": 36, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 1.453065095, "state": "up:active", "uptime": 26.445482575, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:16:35.650 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:16:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":5475,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.6094264150000002,"mdsmap_epoch":36,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":26.588083547} 2020-11-16T22:16:35.791 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5475, "mdsmap_epoch": 36, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 2.609426415, "state": "up:active", "uptime": 26.588083547, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:16:35.792 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:16:35.919 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":5542,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.538876843000001,"mdsmap_epoch":36,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":26.717030069} 2020-11-16T22:16:35.920 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5542, "mdsmap_epoch": 36, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 17.538876843, "state": "up:active", "uptime": 26.717030069, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:16:35.920 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:36.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 Processor -- start 2020-11-16T22:16:36.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 -- start start 2020-11-16T22:16:36.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd594067070 0x7fd594143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd594067510 con 0x7fd5940682b0 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940677b0 con 0x7fd594067070 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd59a3d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b17a0 con 0x7fd5940b7470 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd5993d3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd5993d3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52178/0 (socket says 172.21.15.185:52178) 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd5993d3700 1 -- 172.21.15.185:0/305208435 learned_addr learned my addr 172.21.15.185:0/305208435 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:36.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.063+0000 7fd598bd2700 1 --2- 172.21.15.185:0/305208435 >> v2:172.21.15.96:3300/0 conn(0x7fd594067070 0x7fd594143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd5993d3700 1 -- 172.21.15.185:0/305208435 >> v2:172.21.15.96:3300/0 conn(0x7fd594067070 msgr2=0x7fd594143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:36.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd5993d3700 1 --2- 172.21.15.185:0/305208435 >> v2:172.21.15.96:3300/0 conn(0x7fd594067070 0x7fd594143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd5993d3700 1 -- 172.21.15.185:0/305208435 >> v1:172.21.15.96:6789/0 conn(0x7fd5940682b0 legacy=0x7fd5940b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd5993d3700 1 -- 172.21.15.185:0/305208435 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940b1a40 con 0x7fd5940b7470 2020-11-16T22:16:36.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd5993d3700 1 --2- 172.21.15.185:0/305208435 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fd584006fd0 tx=0x7fd584006eb0).ready entity=mon.2 client_cookie=16b3a9007f6a7495 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd593fff700 1 -- 172.21.15.185:0/305208435 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584009810 con 0x7fd5940b7470 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd593fff700 1 -- 172.21.15.185:0/305208435 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584009970 con 0x7fd5940b7470 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd593fff700 1 -- 172.21.15.185:0/305208435 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584009cf0 con 0x7fd5940b7470 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- 172.21.15.185:0/305208435 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 msgr2=0x7fd5940b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/305208435 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fd584006fd0 tx=0x7fd584006eb0).stop 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- 172.21.15.185:0/305208435 shutdown_connections 2020-11-16T22:16:36.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/305208435 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940b7470 0x7fd5940b7850 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/305208435 >> v2:172.21.15.96:3300/0 conn(0x7fd594067070 0x7fd594143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- 172.21.15.185:0/305208435 >> 172.21.15.185:0/305208435 conn(0x7fd5940637b0 msgr2=0x7fd5940b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:36.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- 172.21.15.185:0/305208435 shutdown_connections 2020-11-16T22:16:36.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- 172.21.15.185:0/305208435 wait complete. 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 Processor -- start 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.067+0000 7fd59a3d5700 1 -- start start 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 0x7fd594154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 0x7fd594151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59413ceb0 con 0x7fd594067070 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59413d150 con 0x7fd5940b7470 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941515f0 con 0x7fd5940682b0 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd598bd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 0x7fd594151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58742/0 (socket says 172.21.15.185:58742) 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 -- 172.21.15.185:0/186380642 learned_addr learned my addr 172.21.15.185:0/186380642 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:36.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd5993d3700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 0x7fd594154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 -- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 msgr2=0x7fd594151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 0x7fd594151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 -- 172.21.15.185:0/186380642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 msgr2=0x7fd594154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 0x7fd594154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 -- 172.21.15.185:0/186380642 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5941512e0 con 0x7fd5940b7470 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd598bd2700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 0x7fd594151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:36.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd5993d3700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 0x7fd594154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd599bd4700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd58c00c710 tx=0x7fd58c007af0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c013410 con 0x7fd5940b7470 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c013570 con 0x7fd5940b7470 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c013910 con 0x7fd5940b7470 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59413cb30 con 0x7fd5940b7470 2020-11-16T22:16:36.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5941470c0 con 0x7fd5940b7470 2020-11-16T22:16:36.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.071+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5941517b0 con 0x7fd5940b7470 2020-11-16T22:16:36.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.075+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd58c010030 con 0x7fd5940b7470 2020-11-16T22:16:36.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.075+0000 7fd5917fa700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 0x7fd56c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:36.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.075+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd58c06d880 con 0x7fd5940b7470 2020-11-16T22:16:36.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.075+0000 7fd5993d3700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 0x7fd56c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:36.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.075+0000 7fd5993d3700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 0x7fd56c057520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd58400b1b0 tx=0x7fd584015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:36.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.079+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd58c043ae0 con 0x7fd5940b7470 2020-11-16T22:16:36.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.283+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd59413ccc0 con 0x7fd5940b7470 2020-11-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.283+0000 7fd5917fa700 1 -- 172.21.15.185:0/186380642 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2516 (secure 0 0 0) 0x7fd58c001e60 con 0x7fd5940b7470 2020-11-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:36.268 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:35.203561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 msgr2=0x7fd56c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 0x7fd56c057520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd58400b1b0 tx=0x7fd584015000).stop 2020-11-16T22:16:36.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 msgr2=0x7fd5941461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd58c00c710 tx=0x7fd58c007af0).stop 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 shutdown_connections 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd56c055060 0x7fd56c057520 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5940b7470 0x7fd5941461b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd594067070 0x7fd594154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 --2- 172.21.15.185:0/186380642 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5940682b0 0x7fd594151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 >> 172.21.15.185:0/186380642 conn(0x7fd5940637b0 msgr2=0x7fd5940b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 shutdown_connections 2020-11-16T22:16:36.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:36.287+0000 7fd59a3d5700 1 -- 172.21.15.185:0/186380642 wait complete. 2020-11-16T22:16:36.273 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 36 2020-11-16T22:16:36.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:36.283 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:16:36.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:36.347 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:16:38.960 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:16:38.978+0000 7fbbc74dd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:16:39.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:39.213 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:16:39.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:39.253 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:16:39.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:39.290 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:16:39.373 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:39.511 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:16:39.512 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:39.512 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:41.513 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:41.649 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:16:41.650 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:41.650 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:43.652 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:43.788 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:16:43.789 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:43.789 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:45.791 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:45.929 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:16:45.930 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:45.930 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:47.269 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:47.276 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:47.278 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:16:47.931 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:16:48.066 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:48.066 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:50.068 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:16:50.208 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:16:50.209 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:16:50.209 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:16:50.210 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:16:50.333 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:16:50.334 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:16:50.334 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:16:52.336 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:16:52.475 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:16:52.476 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:16:52.477 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:16:54.478 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:16:54.613 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:16:54.614 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:16:54.614 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:16:56.616 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:16:56.753 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:16:56.754 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:16:56.754 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:16:56.755 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:16:56.890 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:16:56.891 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:16:56.892 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:16:56.892 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:57.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 Processor -- start 2020-11-16T22:16:57.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 -- start start 2020-11-16T22:16:57.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f61580b3090 0x7f61580b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 0x7f615814b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61580b2b60 con 0x7f61580b3090 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6158061fe0 con 0x7f61580b1d30 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f615ce4f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61580b0fe0 con 0x7f61580ad110 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6156ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 0x7f615814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52194/0 (socket says 172.21.15.185:52194) 2020-11-16T22:16:57.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6156ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 0x7f615814b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58748/0 (socket says 172.21.15.185:58748) 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 -- 172.21.15.185:0/2623708064 learned_addr learned my addr 172.21.15.185:0/2623708064 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f61577fe700 1 --2- 172.21.15.185:0/2623708064 >> v2:172.21.15.96:3300/0 conn(0x7f61580b3090 0x7f61580b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 -- 172.21.15.185:0/2623708064 >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 msgr2=0x7f615814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 --2- 172.21.15.185:0/2623708064 >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 0x7f615814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 -- 172.21.15.185:0/2623708064 >> v2:172.21.15.96:3300/0 conn(0x7f61580b3090 msgr2=0x7f61580b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 --2- 172.21.15.185:0/2623708064 >> v2:172.21.15.96:3300/0 conn(0x7f61580b3090 0x7f61580b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.031+0000 7f6157fff700 1 -- 172.21.15.185:0/2623708064 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580b1280 con 0x7f61580ad110 2020-11-16T22:16:57.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f6157fff700 1 --2- 172.21.15.185:0/2623708064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f6158066720 tx=0x7f6150006eb0).ready entity=mon.2 client_cookie=9ac26480b5a17a94 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f61567fc700 1 -- 172.21.15.185:0/2623708064 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150009810 con 0x7f61580ad110 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f61567fc700 1 -- 172.21.15.185:0/2623708064 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6150009970 con 0x7f61580ad110 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f61567fc700 1 -- 172.21.15.185:0/2623708064 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150009cf0 con 0x7f61580ad110 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- 172.21.15.185:0/2623708064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 msgr2=0x7f61581477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 --2- 172.21.15.185:0/2623708064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f6158066720 tx=0x7f6150006eb0).stop 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- 172.21.15.185:0/2623708064 shutdown_connections 2020-11-16T22:16:57.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 --2- 172.21.15.185:0/2623708064 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61581477b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 --2- 172.21.15.185:0/2623708064 >> v2:172.21.15.96:3300/0 conn(0x7f61580b3090 0x7f61580b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 --2- 172.21.15.185:0/2623708064 >> v2:172.21.15.185:3300/0 conn(0x7f61580b1d30 0x7f615814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- 172.21.15.185:0/2623708064 >> 172.21.15.185:0/2623708064 conn(0x7f6158060130 msgr2=0x7f6158060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- 172.21.15.185:0/2623708064 shutdown_connections 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- 172.21.15.185:0/2623708064 wait complete. 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 Processor -- start 2020-11-16T22:16:57.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.035+0000 7f615ce4f700 1 -- start start 2020-11-16T22:16:57.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 0x7f615814c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 0x7f615815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615814b120 con 0x7f61580b3090 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615814b400 con 0x7f61580b1d30 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61580af7e0 con 0x7f61580ad110 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52196/0 (socket says 172.21.15.185:52196) 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 -- 172.21.15.185:0/3292556755 learned_addr learned my addr 172.21.15.185:0/3292556755 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f6156ffd700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 0x7f615814c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f6157fff700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 0x7f615815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 -- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 msgr2=0x7f615814c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 0x7f615814c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 -- 172.21.15.185:0/3292556755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 msgr2=0x7f615815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 0x7f615815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 -- 172.21.15.185:0/3292556755 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615814b630 con 0x7f61580ad110 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f6157fff700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 0x7f615815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:57.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f6156ffd700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 0x7f615814c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f61577fe700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f614800db90 tx=0x7f6148008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148009820 con 0x7f61580ad110 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148009980 con 0x7f61580ad110 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61580afa40 con 0x7f61580ad110 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148009d00 con 0x7f61580ad110 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61580b04c0 con 0x7f61580ad110 2020-11-16T22:16:57.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615805d200 con 0x7f61580ad110 2020-11-16T22:16:57.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6148013690 con 0x7f61580ad110 2020-11-16T22:16:57.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 0x7f6128057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f6156ffd700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 0x7f6128057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.039+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f614806e4e0 con 0x7f61580ad110 2020-11-16T22:16:57.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.043+0000 7f6156ffd700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 0x7f6128057520 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f61580650f0 tx=0x7f61400091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.043+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6148044630 con 0x7f61580ad110 2020-11-16T22:16:57.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.243+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61580b2ca0 con 0x7f61580ad110 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.243+0000 7f614f7fe700 1 -- 172.21.15.185:0/3292556755 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2520 (secure 0 0 0) 0x7f6148009e60 con 0x7f61580ad110 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:57.229 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:55.182015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:57.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 msgr2=0x7f6128057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 0x7f6128057520 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f61580650f0 tx=0x7f61400091d0).stop 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 msgr2=0x7f61580afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f614800db90 tx=0x7f6148008040).stop 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 shutdown_connections 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6128055060 0x7f6128057520 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61580b3090 0x7f615815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61580ad110 0x7f61580afee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 --2- 172.21.15.185:0/3292556755 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61580b1d30 0x7f615814c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 >> 172.21.15.185:0/3292556755 conn(0x7f6158060130 msgr2=0x7f6158148ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 shutdown_connections 2020-11-16T22:16:57.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.247+0000 7f615ce4f700 1 -- 172.21.15.185:0/3292556755 wait complete. 2020-11-16T22:16:57.234 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 38 2020-11-16T22:16:57.249 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:16:57.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 Processor -- start 2020-11-16T22:16:57.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 -- start start 2020-11-16T22:16:57.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 0x7fe298140be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe2980b39e0 0x7fe2980b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2981410d0 con 0x7fe298144a00 2020-11-16T22:16:57.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2980b3610 con 0x7fe2980b39e0 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d8c9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980b2410 con 0x7fe2980b2d40 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 0x7fe298140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d0c8700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe298144a00 0x7fe29813d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52198/0 (socket says 172.21.15.185:52198) 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29d0c8700 1 -- 172.21.15.185:0/4101723541 learned_addr learned my addr 172.21.15.185:0/4101723541 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe28ffff700 1 --2- 172.21.15.185:0/4101723541 >> v2:172.21.15.96:3300/0 conn(0x7fe2980b39e0 0x7fe2980b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe28f7fe700 1 -- 172.21.15.185:0/4101723541 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948874447 0 0) 0x7fe290001450 con 0x7fe298144a00 2020-11-16T22:16:57.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 -- 172.21.15.185:0/4101723541 >> v2:172.21.15.96:3300/0 conn(0x7fe2980b39e0 msgr2=0x7fe2980b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/4101723541 >> v2:172.21.15.96:3300/0 conn(0x7fe2980b39e0 0x7fe2980b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 -- 172.21.15.185:0/4101723541 >> v1:172.21.15.185:6789/0 conn(0x7fe298144a00 legacy=0x7fe29813d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 -- 172.21.15.185:0/4101723541 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2981442c0 con 0x7fe2980b2d40 2020-11-16T22:16:57.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.391+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/4101723541 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 0x7fe298140be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe28800b9a0 tx=0x7fe288006df0).ready entity=mon.2 client_cookie=94aa5784a4087dd6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe28f7fe700 1 -- 172.21.15.185:0/4101723541 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880097b0 con 0x7fe2980b2d40 2020-11-16T22:16:57.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe28f7fe700 1 -- 172.21.15.185:0/4101723541 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288009910 con 0x7fe2980b2d40 2020-11-16T22:16:57.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe28f7fe700 1 -- 172.21.15.185:0/4101723541 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288009c90 con 0x7fe2980b2d40 2020-11-16T22:16:57.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- 172.21.15.185:0/4101723541 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 msgr2=0x7fe298140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/4101723541 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 0x7fe298140be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe28800b9a0 tx=0x7fe288006df0).stop 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- 172.21.15.185:0/4101723541 shutdown_connections 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/4101723541 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b2d40 0x7fe298140be0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/4101723541 >> v2:172.21.15.96:3300/0 conn(0x7fe2980b39e0 0x7fe2980b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- 172.21.15.185:0/4101723541 >> 172.21.15.185:0/4101723541 conn(0x7fe298060130 msgr2=0x7fe298060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- 172.21.15.185:0/4101723541 shutdown_connections 2020-11-16T22:16:57.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- 172.21.15.185:0/4101723541 wait complete. 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 Processor -- start 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- start start 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980b2d40 0x7fe29812eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 0x7fe298136550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29812e050 con 0x7fe2980b2d40 2020-11-16T22:16:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2981323a0 con 0x7fe298144a00 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d8c9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980b0870 con 0x7fe2980b39e0 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 0x7fe298136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58768/0 (socket says 172.21.15.185:58768) 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 0x7fe298136550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52210/0 (socket says 172.21.15.185:52210) 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29d0c8700 1 -- 172.21.15.185:0/1471012814 learned_addr learned my addr 172.21.15.185:0/1471012814 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.395+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980b2d40 0x7fe29812eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 -- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 msgr2=0x7fe298136550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 0x7fe298136550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 -- 172.21.15.185:0/1471012814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980b2d40 msgr2=0x7fe29812eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980b2d40 0x7fe29812eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 -- 172.21.15.185:0/1471012814 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29812f060 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe29000cae0 tx=0x7fe290007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 -- 172.21.15.185:0/1471012814 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290005b80 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 -- 172.21.15.185:0/1471012814 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290005ce0 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 -- 172.21.15.185:0/1471012814 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290007340 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2981319c0 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2980651d0 con 0x7fe298144a00 2020-11-16T22:16:57.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 -- 172.21.15.185:0/1471012814 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe290010030 con 0x7fe298144a00 2020-11-16T22:16:57.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 0x7fe270057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe28cff9700 1 -- 172.21.15.185:0/1471012814 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe29006dab0 con 0x7fe298144a00 2020-11-16T22:16:57.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 0x7fe270057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.399+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 0x7fe270057520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fe288006e30 tx=0x7fe288015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 Processor -- start 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 -- start start 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba4da0 0x1ba3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a304b0 con 0x1918bb0 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ae4930 con 0x19867b0 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19a46f0 con 0x1ba4da0 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba4da0 0x1ba3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58772/0 (socket says 172.21.15.185:58772) 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 -- 172.21.15.185:0/219783206 learned_addr learned my addr 172.21.15.185:0/219783206 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe28ffff700 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 -- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba4da0 msgr2=0x1ba3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba4da0 0x1ba3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 -- 172.21.15.185:0/219783206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 msgr2=0x1ba2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 -- 172.21.15.185:0/219783206 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x199f5a0 con 0x19867b0 2020-11-16T22:16:57.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fe288006fd0 tx=0x7fe28801f000).ready entity=mon.1 client_cookie=1f40982a968a5f24 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe275ffb700 1 -- 172.21.15.185:0/219783206 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288006920 con 0x19867b0 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe275ffb700 1 -- 172.21.15.185:0/219783206 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288011d50 con 0x19867b0 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.403+0000 7fe275ffb700 1 -- 172.21.15.185:0/219783206 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28802d690 con 0x19867b0 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 msgr2=0x1ba48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fe288006fd0 tx=0x7fe28801f000).stop 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/219783206 shutdown_connections 2020-11-16T22:16:57.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19867b0 0x1ba48b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba4da0 0x1ba3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/219783206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/219783206 >> 172.21.15.185:0/219783206 conn(0x1b54e40 msgr2=0x1ba1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/219783206 shutdown_connections 2020-11-16T22:16:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/219783206 wait complete. 2020-11-16T22:16:57.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- start start 2020-11-16T22:16:57.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 Processor -- start 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 0x1ba57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad16e0 con 0x1918bb0 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad1820 con 0x1ba4da0 2020-11-16T22:16:57.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ad2a90 con 0x19867b0 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 0x1ba57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 10 client.0 ll_register_callbacks cb 0x1ac7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58782/0 (socket says 172.21.15.185:58782) 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe29d0c8700 1 -- 172.21.15.185:0/3016485998 learned_addr learned my addr 172.21.15.185:0/3016485998 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 -- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 msgr2=0x1ba5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 -- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 msgr2=0x1ba8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 -- 172.21.15.185:0/3016485998 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ac2280 con 0x19867b0 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe28ffff700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 0x1ba57a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fe298136c50 tx=0x7fe294008040).ready entity=mon.2 client_cookie=bdea47d9487fe15e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294015700 con 0x19867b0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe294015860 con 0x19867b0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.407+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1b856c0 con 0x19867b0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe294015c30 con 0x19867b0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 10 client.5330 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x18f7fd0 con 0x19867b0 2020-11-16T22:16:57.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1b856c0 con 0x19867b0 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fe29401ea80 con 0x19867b0 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 10 client.5330 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 10 client.5330 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 20 client.5330 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1ba84f0 0x1bd30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 4 client.5330 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x19c2c50 con 0x1ba84f0 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1ba84f0 0x1bd30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 10 client.5330 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1ba84f0 0x1bd30d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d89154e07874d907 server_cookie=7abb10ce33ae4d54 in_seq=0 out_seq=0 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 1 -- 172.21.15.185:0/3016485998 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x19c2c50 con 0x1ba84f0 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 10 client.5330 handle_command_reply: tid=0 2020-11-16T22:16:57.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 client.5330 shutdown 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 msgr2=0x1ba57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 0x1ba57a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fe298136c50 tx=0x7fe294008040).stop 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 shutdown_connections 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1ba84f0 0x1bd30d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1ba5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1918bb0 0x1ba8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3016485998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19867b0 0x1ba57a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 >> 172.21.15.185:0/3016485998 conn(0x1b54e40 msgr2=0x1ba2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe266ffd700 0 client.5330 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:16:57.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 shutdown_connections 2020-11-16T22:16:57.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3016485998 wait complete. 2020-11-16T22:16:57.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.411+0000 7fe2ad5bc740 20 client.5330 trim_cache size 0 max 16384 2020-11-16T22:16:57.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.427+0000 7fe2ad5bc740 1 Processor -- start 2020-11-16T22:16:57.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.427+0000 7fe2ad5bc740 1 -- start start 2020-11-16T22:16:57.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 0x1b996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba4da0 0x1ba2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aa9e40 con 0x1ba4da0 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bdaea0 con 0x1918bb0 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b9f380 con 0x1b54e40 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58788/0 (socket says 172.21.15.185:58788) 2020-11-16T22:16:57.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 -- 172.21.15.185:0/1845854836 learned_addr learned my addr 172.21.15.185:0/1845854836 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 0x1b996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba4da0 0x1ba2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 -- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 msgr2=0x1b996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 0x1b996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 -- 172.21.15.185:0/1845854836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba4da0 msgr2=0x1ba2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba4da0 0x1ba2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 -- 172.21.15.185:0/1845854836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c008160 con 0x1918bb0 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 0x1b996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe28ffff700 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe294001e50 tx=0x7fe294022000).ready entity=mon.1 client_cookie=f3012f545993b753 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2767fc700 1 -- 172.21.15.185:0/1845854836 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294015c30 con 0x1918bb0 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2767fc700 1 -- 172.21.15.185:0/1845854836 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294015d90 con 0x1918bb0 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2767fc700 1 -- 172.21.15.185:0/1845854836 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294014410 con 0x1918bb0 2020-11-16T22:16:57.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 msgr2=0x1b99bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe294001e50 tx=0x7fe294022000).stop 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/1845854836 shutdown_connections 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba4da0 0x1ba2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b54e40 0x1b996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/1845854836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1918bb0 0x1b99bb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/1845854836 >> 172.21.15.185:0/1845854836 conn(0x7fe26c002320 msgr2=0x1ba6240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/1845854836 shutdown_connections 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/1845854836 wait complete. 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 -- start start 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.431+0000 7fe2ad5bc740 1 Processor -- start 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 0x1c3caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 0x1c3cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bd7f30 con 0x1b54e40 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ba2da0 con 0x1ba4da0 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ba3040 con 0x1918bb0 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29c8c7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 0x1c3caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 10 client.0 ll_register_callbacks cb 0x1b9e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:57.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29c8c7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 0x1c3caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52236/0 (socket says 172.21.15.185:52236) 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58796/0 (socket says 172.21.15.185:58796) 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29c8c7700 1 -- 172.21.15.185:0/3226735298 learned_addr learned my addr 172.21.15.185:0/3226735298 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe28ffff700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 0x1c3cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 -- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 msgr2=0x1c3caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 0x1c3caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 -- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 msgr2=0x1c3cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:57.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 0x1c3cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 -- 172.21.15.185:0/3226735298 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1936720 con 0x1ba4da0 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe28ffff700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 0x1c3cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29d0c8700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe290047900 tx=0x7fe290011000).ready entity=mon.1 client_cookie=4636a298c1005833 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290030070 con 0x1ba4da0 2020-11-16T22:16:57.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe290011360 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe288011d50 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2900437c0 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 10 client.5758 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x1941e00 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288011d50 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fe290043920 con 0x1ba4da0 2020-11-16T22:16:57.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 10 client.5758 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 10 client.5758 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 20 client.5758 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1bd9770 0x1c46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 4 client.5758 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x19c9f50 con 0x1bd9770 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1bd9770 0x1c46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe266ffd700 10 client.5758 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.435+0000 7fe29c8c7700 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1bd9770 0x1c46bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2cf989ebb1b61201 server_cookie=ba0116ee530f786d in_seq=0 out_seq=0 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.463+0000 7fe266ffd700 1 -- 172.21.15.185:0/3226735298 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x19c9f50 con 0x1bd9770 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.463+0000 7fe266ffd700 10 client.5758 handle_command_reply: tid=0 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.463+0000 7fe2ad5bc740 1 client.5758 shutdown 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 msgr2=0x1c466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fe290047900 tx=0x7fe290011000).stop 2020-11-16T22:16:57.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 shutdown_connections 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x1bd9770 0x1c46bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ba4da0 0x1c466c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe266ffd700 0 client.5758 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1918bb0 0x1c3caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 --2- 172.21.15.185:0/3226735298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b54e40 0x1c3cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 >> 172.21.15.185:0/3226735298 conn(0x7fe26c002320 msgr2=0x1be8380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 shutdown_connections 2020-11-16T22:16:57.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 1 -- 172.21.15.185:0/3226735298 wait complete. 2020-11-16T22:16:57.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe2ad5bc740 20 client.5758 trim_cache size 0 max 16384 2020-11-16T22:16:57.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 msgr2=0x7fe270057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 0x7fe270057520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fe288006e30 tx=0x7fe288015000).stop 2020-11-16T22:16:57.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 msgr2=0x7fe29812f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe29000cae0 tx=0x7fe290007ba0).stop 2020-11-16T22:16:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 shutdown_connections 2020-11-16T22:16:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe270055060 0x7fe270057520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe298144a00 0x7fe29812f450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980b2d40 0x7fe29812eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 --2- 172.21.15.185:0/1471012814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe2980b39e0 0x7fe298136550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 >> 172.21.15.185:0/1471012814 conn(0x7fe298060130 msgr2=0x7fe2980abef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 shutdown_connections 2020-11-16T22:16:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.467+0000 7fe29d8c9700 1 -- 172.21.15.185:0/1471012814 wait complete. 2020-11-16T22:16:57.832 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:16:57.832 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:16:57.832 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:16:57.832 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:16:57.833 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:57.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 Processor -- start 2020-11-16T22:16:57.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 -- start start 2020-11-16T22:16:57.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 0x7f5ff80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff8067510 con 0x7f5ff8067070 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80677b0 con 0x7f5ff80b7470 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffd706700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80b17a0 con 0x7f5ff80682b0 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ff7fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5ff80b7470 0x7f5ff80b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52238/0 (socket says 172.21.15.185:52238) 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ff7fff700 1 -- 172.21.15.185:0/3035922643 learned_addr learned my addr 172.21.15.185:0/3035922643 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ffcf05700 1 --2- 172.21.15.185:0/3035922643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 0x7f5ff80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024888896 0 0) 0x7f5ff80677b0 con 0x7f5ff80b7470 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.983+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003240 con 0x7f5ff80b7470 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346342705 0 0) 0x7f5ff8067510 con 0x7f5ff8067070 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff80677b0 con 0x7f5ff8067070 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 817791016 0 0) 0x7f5fd8003240 con 0x7f5ff80b7470 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffcf05700 1 -- 172.21.15.185:0/3035922643 >> v1:172.21.15.185:6789/0 conn(0x7f5ff80b7470 legacy=0x7f5ff80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffcf05700 1 -- 172.21.15.185:0/3035922643 >> v1:172.21.15.96:6789/0 conn(0x7f5ff8067070 legacy=0x7f5ff80ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffcf05700 1 -- 172.21.15.185:0/3035922643 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80b1a40 con 0x7f5ff80682b0 2020-11-16T22:16:57.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffcf05700 1 --2- 172.21.15.185:0/3035922643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 0x7f5ff80b7850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f5ff0009c40 tx=0x7f5ff00091d0).ready entity=mon.2 client_cookie=d39fb563e6000478 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80014e0 con 0x7f5ff80682b0 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000b730 con 0x7f5ff80682b0 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ff77fe700 1 -- 172.21.15.185:0/3035922643 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000bab0 con 0x7f5ff80682b0 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- 172.21.15.185:0/3035922643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 msgr2=0x7f5ff80b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 --2- 172.21.15.185:0/3035922643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 0x7f5ff80b7850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f5ff0009c40 tx=0x7f5ff00091d0).stop 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- 172.21.15.185:0/3035922643 shutdown_connections 2020-11-16T22:16:57.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 --2- 172.21.15.185:0/3035922643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80682b0 0x7f5ff80b7850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- 172.21.15.185:0/3035922643 >> 172.21.15.185:0/3035922643 conn(0x7f5ff80637b0 msgr2=0x7f5ff80b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:57.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- 172.21.15.185:0/3035922643 shutdown_connections 2020-11-16T22:16:57.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- 172.21.15.185:0/3035922643 wait complete. 2020-11-16T22:16:57.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 Processor -- start 2020-11-16T22:16:57.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- start start 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ff80682b0 0x7f5ff8139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81518e0 con 0x7f5ff80682b0 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8151b80 con 0x7f5ff8067070 2020-11-16T22:16:57.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.987+0000 7f5ffd706700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80af4a0 con 0x7f5ff80b7470 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffcf05700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffcf05700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52254/0 (socket says 172.21.15.185:52254) 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58806/0 (socket says 172.21.15.185:58806) 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffcf05700 1 -- 172.21.15.185:0/1390971655 learned_addr learned my addr 172.21.15.185:0/1390971655 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5feffff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ff80682b0 0x7f5ff8139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 -- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 msgr2=0x7f5ff8139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 -- 172.21.15.185:0/1390971655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ff80682b0 msgr2=0x7f5ff8139760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ff80682b0 0x7f5ff8139760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 -- 172.21.15.185:0/1390971655 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80af740 con 0x7f5ff8067070 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffcf05700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff7fff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5fe800ca90 tx=0x7f5fe80019f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800ce60 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8001a80 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff8065700 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80073a0 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81113b0 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff805d200 con 0x7f5ff8067070 2020-11-16T22:16:57.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5fe8010070 con 0x7f5ff8067070 2020-11-16T22:16:57.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 0x7f5fd4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:57.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5fe806d910 con 0x7f5ff8067070 2020-11-16T22:16:57.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5feffff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 0x7f5fd4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:57.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.991+0000 7f5feffff700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 0x7f5fd4057520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5ff8150d90 tx=0x7f5fe0007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:57.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:57.999+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5fe8043b70 con 0x7f5ff8067070 2020-11-16T22:16:58.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.199+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff813a7c0 con 0x7f5ff8067070 2020-11-16T22:16:58.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ff4ff9700 1 -- 172.21.15.185:0/1390971655 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2520 (secure 0 0 0) 0x7f5fe8047190 con 0x7f5ff8067070 2020-11-16T22:16:58.186 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:58.186 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:55.182015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:58.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 msgr2=0x7f5fd4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 0x7f5fd4057520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5ff8150d90 tx=0x7f5fe0007a20).stop 2020-11-16T22:16:58.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 msgr2=0x7f5ff80aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5fe800ca90 tx=0x7f5fe80019f0).stop 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 shutdown_connections 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5fd4055060 0x7f5fd4057520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5ff8067070 0x7f5ff80aecc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5ff80b7470 0x7f5ff8139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 --2- 172.21.15.185:0/1390971655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ff80682b0 0x7f5ff8139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 >> 172.21.15.185:0/1390971655 conn(0x7f5ff80637b0 msgr2=0x7f5ff8143f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 shutdown_connections 2020-11-16T22:16:58.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.203+0000 7f5ffd706700 1 -- 172.21.15.185:0/1390971655 wait complete. 2020-11-16T22:16:58.190 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 38 2020-11-16T22:16:58.201 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:16:58.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.339+0000 7f214359e700 1 Processor -- start 2020-11-16T22:16:58.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.339+0000 7f214359e700 1 -- start start 2020-11-16T22:16:58.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f21440b5d70 0x7f21440ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2144143470 0x7f2144143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21440b7290 con 0x7f21440b4470 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21440b4bf0 con 0x7f21440b5d70 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21440b74c0 con 0x7f2144143470 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2144143470 0x7f2144143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2144143470 0x7f2144143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58818/0 (socket says 172.21.15.185:58818) 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 -- 172.21.15.185:0/578498935 learned_addr learned my addr 172.21.15.185:0/578498935 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214259c700 1 --2- 172.21.15.185:0/578498935 >> v2:172.21.15.96:3300/0 conn(0x7f21440b5d70 0x7f21440ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 -- 172.21.15.185:0/578498935 >> v2:172.21.15.96:3300/0 conn(0x7f21440b5d70 msgr2=0x7f21440ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 --2- 172.21.15.185:0/578498935 >> v2:172.21.15.96:3300/0 conn(0x7f21440b5d70 0x7f21440ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 -- 172.21.15.185:0/578498935 >> v1:172.21.15.96:6789/0 conn(0x7f21440b4470 legacy=0x7f21440b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 -- 172.21.15.185:0/578498935 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440b7760 con 0x7f2144143470 2020-11-16T22:16:58.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2142d9d700 1 --2- 172.21.15.185:0/578498935 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2144143470 0x7f2144143920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f21440b30c0 tx=0x7f213c009290).ready entity=mon.1 client_cookie=422ff8232dd8ed9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2141d9b700 1 -- 172.21.15.185:0/578498935 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00bb80 con 0x7f2144143470 2020-11-16T22:16:58.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2141d9b700 1 -- 172.21.15.185:0/578498935 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00bce0 con 0x7f2144143470 2020-11-16T22:16:58.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f2141d9b700 1 -- 172.21.15.185:0/578498935 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c011610 con 0x7f2144143470 2020-11-16T22:16:58.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- 172.21.15.185:0/578498935 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2144143470 msgr2=0x7f2144143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 --2- 172.21.15.185:0/578498935 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2144143470 0x7f2144143920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f21440b30c0 tx=0x7f213c009290).stop 2020-11-16T22:16:58.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- 172.21.15.185:0/578498935 shutdown_connections 2020-11-16T22:16:58.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 --2- 172.21.15.185:0/578498935 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2144143470 0x7f2144143920 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 --2- 172.21.15.185:0/578498935 >> v2:172.21.15.96:3300/0 conn(0x7f21440b5d70 0x7f21440ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- 172.21.15.185:0/578498935 >> 172.21.15.185:0/578498935 conn(0x7f214405ba80 msgr2=0x7f214405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- 172.21.15.185:0/578498935 shutdown_connections 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- 172.21.15.185:0/578498935 wait complete. 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 Processor -- start 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.343+0000 7f214359e700 1 -- start start 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21440b5d70 0x7f21441466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2144143470 0x7f21440ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21440ae800 con 0x7f21440b5d70 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21441532e0 con 0x7f21440b4470 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2144153580 con 0x7f2144143470 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58820/0 (socket says 172.21.15.185:58820) 2020-11-16T22:16:58.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 -- 172.21.15.185:0/3562563575 learned_addr learned my addr 172.21.15.185:0/3562563575 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f2142d9d700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2144143470 0x7f21440ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f2139d9b700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21440b5d70 0x7f21441466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 -- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2144143470 msgr2=0x7f21440ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2144143470 0x7f21440ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 -- 172.21.15.185:0/3562563575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21440b5d70 msgr2=0x7f21441466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21440b5d70 0x7f21441466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 -- 172.21.15.185:0/3562563575 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144153820 con 0x7f21440b4470 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214259c700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f213400db60 tx=0x7f2134008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f213affd700 1 -- 172.21.15.185:0/3562563575 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400b130 con 0x7f21440b4470 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f213affd700 1 -- 172.21.15.185:0/3562563575 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213400be20 con 0x7f21440b4470 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2144146bd0 con 0x7f21440b4470 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f213affd700 1 -- 172.21.15.185:0/3562563575 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340099a0 con 0x7f21440b4470 2020-11-16T22:16:58.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21440ae4f0 con 0x7f21440b4470 2020-11-16T22:16:58.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.347+0000 7f213affd700 1 -- 172.21.15.185:0/3562563575 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2134009d10 con 0x7f21440b4470 2020-11-16T22:16:58.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.351+0000 7f213affd700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 0x7f2120057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.351+0000 7f2139d9b700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 0x7f2120057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.351+0000 7f213affd700 1 -- 172.21.15.185:0/3562563575 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f213406dc10 con 0x7f21440b4470 2020-11-16T22:16:58.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.351+0000 7f2139d9b700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 0x7f2120057520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f2144154240 tx=0x7f212c007ae0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 Processor -- start 2020-11-16T22:16:58.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- start start 2020-11-16T22:16:58.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x24f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23814b0 con 0x2269bb0 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2435930 con 0x24f5da0 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x22f56f0 con 0x22d77b0 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58828/0 (socket says 172.21.15.185:58828) 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 -- 172.21.15.185:0/3011424832 learned_addr learned my addr 172.21.15.185:0/3011424832 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f214259c700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2139d9b700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x24f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 -- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 msgr2=0x24f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 -- 172.21.15.185:0/3011424832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 msgr2=0x24f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x24f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 -- 172.21.15.185:0/3011424832 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22f05a0 con 0x24f5da0 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2139d9b700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x24f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2142d9d700 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f21440b7600 tx=0x7f213c016000).ready entity=mon.1 client_cookie=49615a1a0740ab62 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f211b7fe700 1 -- 172.21.15.185:0/3011424832 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c011610 con 0x24f5da0 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f211b7fe700 1 -- 172.21.15.185:0/3011424832 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c021070 con 0x24f5da0 2020-11-16T22:16:58.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f211b7fe700 1 -- 172.21.15.185:0/3011424832 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00be50 con 0x24f5da0 2020-11-16T22:16:58.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 msgr2=0x24f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f21440b7600 tx=0x7f213c016000).stop 2020-11-16T22:16:58.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- 172.21.15.185:0/3011424832 shutdown_connections 2020-11-16T22:16:58.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f4380 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 --2- 172.21.15.185:0/3011424832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x24f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- 172.21.15.185:0/3011424832 >> 172.21.15.185:0/3011424832 conn(0x24a5e40 msgr2=0x24f2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- 172.21.15.185:0/3011424832 shutdown_connections 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.355+0000 7f2157309740 1 -- 172.21.15.185:0/3011424832 wait complete. 2020-11-16T22:16:58.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- start start 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 Processor -- start 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x252bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24226e0 con 0x2269bb0 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2422820 con 0x24f5da0 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2423a90 con 0x22d77b0 2020-11-16T22:16:58.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 10 client.0 ll_register_callbacks cb 0x2418ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2139d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58838/0 (socket says 172.21.15.185:58838) 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 -- 172.21.15.185:0/295977500 learned_addr learned my addr 172.21.15.185:0/295977500 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f214259c700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x252bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 -- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 msgr2=0x24f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 -- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 msgr2=0x252bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x252bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 -- 172.21.15.185:0/295977500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2413280 con 0x24f5da0 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2142d9d700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f213c01dda0 tx=0x7f213c01f070).ready entity=mon.1 client_cookie=f783778b5230c0d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c02b070 con 0x24f5da0 2020-11-16T22:16:58.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f213c00be50 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24d66c0 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f213c011610 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 10 client.5776 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x2248fd0 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24d66c0 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f213c039020 con 0x24f5da0 2020-11-16T22:16:58.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 10 client.5776 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 10 client.5776 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 20 client.5776 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2524090 0x253d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 4 client.5776 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2313c50 con 0x2524090 2020-11-16T22:16:58.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.359+0000 7f214259c700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2524090 0x253d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f214259c700 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2524090 0x253d4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=d27b05facba293f2 server_cookie=f1dfe21dfb7080e9 in_seq=0 out_seq=0 2020-11-16T22:16:58.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2118ff9700 10 client.5776 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2118ff9700 1 -- 172.21.15.185:0/295977500 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2313c50 con 0x2524090 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2118ff9700 10 client.5776 handle_command_reply: tid=0 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 client.5776 shutdown 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 msgr2=0x24f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f213c01dda0 tx=0x7f213c01f070).stop 2020-11-16T22:16:58.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 shutdown_connections 2020-11-16T22:16:58.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2524090 0x253d4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24f5da0 0x24f8e10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2269bb0 0x252bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 --2- 172.21.15.185:0/295977500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22d77b0 0x24f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2118ff9700 0 client.5776 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:16:58.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 >> 172.21.15.185:0/295977500 conn(0x24a5e40 msgr2=0x24f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 shutdown_connections 2020-11-16T22:16:58.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 1 -- 172.21.15.185:0/295977500 wait complete. 2020-11-16T22:16:58.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.363+0000 7f2157309740 20 client.5776 trim_cache size 0 max 16384 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 Processor -- start 2020-11-16T22:16:58.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- start start 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24a5e40 0x24f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 0x24f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23fae40 con 0x24a5e40 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d7200 con 0x22d77b0 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24ef1c0 con 0x2269bb0 2020-11-16T22:16:58.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52286/0 (socket says 172.21.15.185:52286) 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 -- 172.21.15.185:0/1061662501 learned_addr learned my addr 172.21.15.185:0/1061662501 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2142d9d700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 0x24f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f214259c700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24a5e40 0x24f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 -- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 msgr2=0x24f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 0x24f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 -- 172.21.15.185:0/1061662501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24a5e40 msgr2=0x24f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24a5e40 0x24f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 -- 172.21.15.185:0/1061662501 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24ef9d0 con 0x2269bb0 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2142d9d700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 0x24f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2139d9b700 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f212c012a70 tx=0x7f212c014040).ready entity=mon.2 client_cookie=c4d99919404166a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f211bfff700 1 -- 172.21.15.185:0/1061662501 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0119e0 con 0x2269bb0 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f211bfff700 1 -- 172.21.15.185:0/1061662501 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c018040 con 0x2269bb0 2020-11-16T22:16:58.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f211bfff700 1 -- 172.21.15.185:0/1061662501 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c011d50 con 0x2269bb0 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 msgr2=0x24f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f212c012a70 tx=0x7f212c014040).stop 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- 172.21.15.185:0/1061662501 shutdown_connections 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22d77b0 0x24f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24a5e40 0x24f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 --2- 172.21.15.185:0/1061662501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2269bb0 0x24f1200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- 172.21.15.185:0/1061662501 >> 172.21.15.185:0/1061662501 conn(0x7f210c002320 msgr2=0x24f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- 172.21.15.185:0/1061662501 shutdown_connections 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.383+0000 7f2157309740 1 -- 172.21.15.185:0/1061662501 wait complete. 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- start start 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 Processor -- start 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 0x25292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a5e40 0x25329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24f2f30 con 0x22d77b0 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24f3120 con 0x2269bb0 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24f33c0 con 0x24a5e40 2020-11-16T22:16:58.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 10 client.0 ll_register_callbacks cb 0x2528950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:58.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58848/0 (socket says 172.21.15.185:58848) 2020-11-16T22:16:58.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 -- 172.21.15.185:0/2904392562 learned_addr learned my addr 172.21.15.185:0/2904392562 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2142d9d700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a5e40 0x25329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2139d9b700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 0x25292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 -- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a5e40 msgr2=0x25329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a5e40 0x25329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 -- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 msgr2=0x25292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 0x25292f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 -- 172.21.15.185:0/2904392562 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c008160 con 0x2269bb0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2139d9b700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 0x25292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f214259c700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f2134047830 tx=0x7f213402d280).ready entity=mon.1 client_cookie=c54eded0e4d474a5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213402c070 con 0x2269bb0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f213c021070 con 0x2269bb0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2134043920 con 0x2269bb0 2020-11-16T22:16:58.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2134046a90 con 0x2269bb0 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 10 client.5782 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x2287720 con 0x2269bb0 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c021070 con 0x2269bb0 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f2134046bf0 con 0x2269bb0 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 10 client.5782 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 10 client.5782 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:16:58.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 20 client.5782 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x24f5da0 0x258b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 4 client.5782 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x231af50 con 0x24f5da0 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2139d9b700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x24f5da0 0x258b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2118ff9700 10 client.5782 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:16:58.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.387+0000 7f2139d9b700 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x24f5da0 0x258b470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=58829e2decb4a4af server_cookie=60da086491538624 in_seq=0 out_seq=0 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2118ff9700 1 -- 172.21.15.185:0/2904392562 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x231af50 con 0x24f5da0 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2118ff9700 10 client.5782 handle_command_reply: tid=0 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 client.5782 shutdown 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 msgr2=0x253cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f2134047830 tx=0x7f213402d280).stop 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 shutdown_connections 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x24f5da0 0x258b470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2269bb0 0x253cba0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2118ff9700 0 client.5782 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a5e40 0x25329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 --2- 172.21.15.185:0/2904392562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22d77b0 0x25292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 >> 172.21.15.185:0/2904392562 conn(0x7f210c002320 msgr2=0x2524470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 shutdown_connections 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 1 -- 172.21.15.185:0/2904392562 wait complete. 2020-11-16T22:16:58.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f2157309740 20 client.5782 trim_cache size 0 max 16384 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 msgr2=0x7f2120057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 0x7f2120057520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f2144154240 tx=0x7f212c007ae0).stop 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 msgr2=0x7f2144153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f213400db60 tx=0x7f2134008040).stop 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 shutdown_connections 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2120055060 0x7f2120057520 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f21440b4470 0x7f2144153cc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2144143470 0x7f21440ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 --2- 172.21.15.185:0/3562563575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21440b5d70 0x7f21441466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 >> 172.21.15.185:0/3562563575 conn(0x7f214405ba80 msgr2=0x7f2144147370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 shutdown_connections 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.407+0000 7f214359e700 1 -- 172.21.15.185:0/3562563575 wait complete. 2020-11-16T22:16:58.782 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:16:58.783 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 Processor -- start 2020-11-16T22:16:58.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 -- start start 2020-11-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52dc0ad110 0x7f52dc1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc061fe0 con 0x7f52dc0b1d30 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc0b2b60 con 0x7f52dc0ad110 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52e0a69700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc14c060 con 0x7f52dc0b3090 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52dbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52dc0ad110 0x7f52dc1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52dbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52dc0ad110 0x7f52dc1477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58858/0 (socket says 172.21.15.185:58858) 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52dbfff700 1 -- 172.21.15.185:0/1391983901 learned_addr learned my addr 172.21.15.185:0/1391983901 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.931+0000 7f52db7fe700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 msgr2=0x7f52dc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1391983901 >> v1:172.21.15.185:6789/0 conn(0x7f52dc0b1d30 legacy=0x7f52dc14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1391983901 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc0b0fe0 con 0x7f52dc0ad110 2020-11-16T22:16:58.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0ad110 0x7f52dc1477b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f52d4009d50 tx=0x7f52d4009290).ready entity=mon.1 client_cookie=bd94465c2ce07be4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52da7fc700 1 -- 172.21.15.185:0/1391983901 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400bbc0 con 0x7f52dc0ad110 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52da7fc700 1 -- 172.21.15.185:0/1391983901 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400bd20 con 0x7f52dc0ad110 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52da7fc700 1 -- 172.21.15.185:0/1391983901 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4011610 con 0x7f52dc0ad110 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0ad110 msgr2=0x7f52dc1477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0ad110 0x7f52dc1477b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f52d4009d50 tx=0x7f52d4009290).stop 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- 172.21.15.185:0/1391983901 shutdown_connections 2020-11-16T22:16:58.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0ad110 0x7f52dc1477b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1391983901 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- 172.21.15.185:0/1391983901 >> 172.21.15.185:0/1391983901 conn(0x7f52dc060130 msgr2=0x7f52dc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- 172.21.15.185:0/1391983901 shutdown_connections 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- 172.21.15.185:0/1391983901 wait complete. 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 Processor -- start 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- start start 2020-11-16T22:16:58.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52dc0ad110 0x7f52dc10f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0b1d30 0x7f52dc067320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc112240 con 0x7f52dc0ad110 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1110e0 con 0x7f52dc0b1d30 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52e0a69700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc111fa0 con 0x7f52dc0b3090 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52308/0 (socket says 172.21.15.185:52308) 2020-11-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1586106234 learned_addr learned my addr 172.21.15.185:0/1586106234 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52daffd700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0b1d30 0x7f52dc067320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52db7fe700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52dc0ad110 0x7f52dc10f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0b1d30 msgr2=0x7f52dc067320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0b1d30 0x7f52dc067320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1586106234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52dc0ad110 msgr2=0x7f52dc10f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52dc0ad110 0x7f52dc10f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.935+0000 7f52dbfff700 1 -- 172.21.15.185:0/1586106234 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc1579e0 con 0x7f52dc0b3090 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52dbfff700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f52d40092d0 tx=0x7f52d4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400bbc0 con 0x7f52dc0b3090 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400be90 con 0x7f52dc0b3090 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc157c80 con 0x7f52dc0b3090 2020-11-16T22:16:58.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4010d60 con 0x7f52dc0b3090 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc11e090 con 0x7f52dc0b3090 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc157e10 con 0x7f52dc0b3090 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f52d4023030 con 0x7f52dc0b3090 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.939+0000 7f52bb7fe700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 0x7f52b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.943+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52d406f4d0 con 0x7f52dc0b3090 2020-11-16T22:16:58.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.943+0000 7f52db7fe700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 0x7f52b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.943+0000 7f52db7fe700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 0x7f52b4057520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f52cc006fd0 tx=0x7f52cc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:58.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:58.943+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f52d4045750 con 0x7f52dc0b3090 2020-11-16T22:16:59.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.147+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52dc05d200 con 0x7f52dc0b3090 2020-11-16T22:16:59.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.147+0000 7f52bb7fe700 1 -- 172.21.15.185:0/1586106234 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2520 (secure 0 0 0) 0x7f52d4021090 con 0x7f52dc0b3090 2020-11-16T22:16:59.132 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:16:59.132 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:55.182015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 msgr2=0x7f52b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 0x7f52b4057520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f52cc006fd0 tx=0x7f52cc008040).stop 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 msgr2=0x7f52dc158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f52d40092d0 tx=0x7f52d4016000).stop 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 shutdown_connections 2020-11-16T22:16:59.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f52b4055060 0x7f52b4057520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52dc0b3090 0x7f52dc158120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52dc0ad110 0x7f52dc10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 --2- 172.21.15.185:0/1586106234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52dc0b1d30 0x7f52dc067320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 >> 172.21.15.185:0/1586106234 conn(0x7f52dc060130 msgr2=0x7f52dc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 shutdown_connections 2020-11-16T22:16:59.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.151+0000 7f52e0a69700 1 -- 172.21.15.185:0/1586106234 wait complete. 2020-11-16T22:16:59.136 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 38 2020-11-16T22:16:59.147 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:16:59.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 Processor -- start 2020-11-16T22:16:59.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- start start 2020-11-16T22:16:59.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 0x7feeec0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7feeec0b37e0 0x7feeec143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 0x7feeec13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0b31d0 con 0x7feeec0603f0 2020-11-16T22:16:59.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec066f50 con 0x7feeec0b37e0 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0671f0 con 0x7feeec144340 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7feeec0b37e0 0x7feeec143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef1173700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 0x7feeec13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7feeec0b37e0 0x7feeec143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58870/0 (socket says 172.21.15.185:58870) 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 -- 172.21.15.185:0/2313002101 learned_addr learned my addr 172.21.15.185:0/2313002101 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef0972700 1 --2- 172.21.15.185:0/2313002101 >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 0x7feeec0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 -- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 msgr2=0x7feeec13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 0x7feeec13fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 -- 172.21.15.185:0/2313002101 >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 msgr2=0x7feeec0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 --2- 172.21.15.185:0/2313002101 >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 0x7feeec0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 -- 172.21.15.185:0/2313002101 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec067490 con 0x7feeec0b37e0 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef0972700 1 --2- 172.21.15.185:0/2313002101 >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 0x7feeec0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef1173700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 0x7feeec13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:59.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeebfff700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec143e50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7feeec060f30 tx=0x7feed4006eb0).ready entity=mon.1 client_cookie=735fa8decd12ac0d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeeb7fe700 1 -- 172.21.15.185:0/2313002101 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40097d0 con 0x7feeec0b37e0 2020-11-16T22:16:59.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeeb7fe700 1 -- 172.21.15.185:0/2313002101 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4009930 con 0x7feeec0b37e0 2020-11-16T22:16:59.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feeeb7fe700 1 -- 172.21.15.185:0/2313002101 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4009cb0 con 0x7feeec0b37e0 2020-11-16T22:16:59.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 msgr2=0x7feeec143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec143e50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7feeec060f30 tx=0x7feed4006eb0).stop 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- 172.21.15.185:0/2313002101 shutdown_connections 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec144340 0x7feeec13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- 172.21.15.185:0/2313002101 >> v2:172.21.15.96:3300/0 conn(0x7feeec0603f0 0x7feeec0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 --2- 172.21.15.185:0/2313002101 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec143e50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- 172.21.15.185:0/2313002101 >> 172.21.15.185:0/2313002101 conn(0x7feeec063f60 msgr2=0x7feeec064340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- 172.21.15.185:0/2313002101 shutdown_connections 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.291+0000 7feef2bd6700 1 -- 172.21.15.185:0/2313002101 wait complete. 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 Processor -- start 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- start start 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec0603f0 0x7feeec0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec144340 0x7feeec139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0b12a0 con 0x7feeec144340 2020-11-16T22:16:59.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0ad0c0 con 0x7feeec0b37e0 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec153460 con 0x7feeec0603f0 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec0603f0 0x7feeec0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58878/0 (socket says 172.21.15.185:58878) 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 -- 172.21.15.185:0/3118651441 learned_addr learned my addr 172.21.15.185:0/3118651441 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef1173700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec144340 0x7feeec139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 -- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec0603f0 msgr2=0x7feeec0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec0603f0 0x7feeec0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 -- 172.21.15.185:0/3118651441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec144340 msgr2=0x7feeec139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec144340 0x7feeec139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 -- 172.21.15.185:0/3118651441 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec13ec50 con 0x7feeec0b37e0 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feeebfff700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7feed400beb0 tx=0x7feed4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feee8ff9700 1 -- 172.21.15.185:0/3118651441 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4011d10 con 0x7feeec0b37e0 2020-11-16T22:16:59.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feee8ff9700 1 -- 172.21.15.185:0/3118651441 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4011e90 con 0x7feeec0b37e0 2020-11-16T22:16:59.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec13e5e0 con 0x7feeec0b37e0 2020-11-16T22:16:59.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feee8ff9700 1 -- 172.21.15.185:0/3118651441 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed401e6b0 con 0x7feeec0b37e0 2020-11-16T22:16:59.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.295+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec139a00 con 0x7feeec0b37e0 2020-11-16T22:16:59.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.299+0000 7feee8ff9700 1 -- 172.21.15.185:0/3118651441 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feed4025030 con 0x7feeec0b37e0 2020-11-16T22:16:59.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.299+0000 7feee8ff9700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 0x7feec8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.299+0000 7feee8ff9700 1 -- 172.21.15.185:0/3118651441 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feed406f1e0 con 0x7feeec0b37e0 2020-11-16T22:16:59.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.299+0000 7feef0972700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 0x7feec8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.299+0000 7feef0972700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 0x7feec8057520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7feedc00b680 tx=0x7feedc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 Processor -- start 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- start start 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x2441c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 0x24093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x22984b0 con 0x2405d30 2020-11-16T22:16:59.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x234c930 con 0x21ee7b0 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x220c6f0 con 0x2180bb0 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58884/0 (socket says 172.21.15.185:58884) 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 -- 172.21.15.185:0/1253195918 learned_addr learned my addr 172.21.15.185:0/1253195918 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feeebfff700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x2441c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef1173700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 0x24093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 -- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 msgr2=0x2441c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x2441c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 -- 172.21.15.185:0/1253195918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 msgr2=0x24093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 0x24093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 -- 172.21.15.185:0/1253195918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22075a0 con 0x21ee7b0 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feeebfff700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x2441c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef1173700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 0x24093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:16:59.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feef0972700 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7feedc012d90 tx=0x7feedc013000).ready entity=mon.1 client_cookie=9914497a2179557a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feee1ffb700 1 -- 172.21.15.185:0/1253195918 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc010ab0 con 0x21ee7b0 2020-11-16T22:16:59.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feee1ffb700 1 -- 172.21.15.185:0/1253195918 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc010c10 con 0x21ee7b0 2020-11-16T22:16:59.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7feee1ffb700 1 -- 172.21.15.185:0/1253195918 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc01d400 con 0x21ee7b0 2020-11-16T22:16:59.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 msgr2=0x24464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7feedc012d90 tx=0x7feedc013000).stop 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- 172.21.15.185:0/1253195918 shutdown_connections 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x21ee7b0 0x24464b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2405d30 0x24093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 --2- 172.21.15.185:0/1253195918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x2441c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- 172.21.15.185:0/1253195918 >> 172.21.15.185:0/1253195918 conn(0x23bce40 msgr2=0x2445bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- 172.21.15.185:0/1253195918 shutdown_connections 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.303+0000 7fef01667740 1 -- 172.21.15.185:0/1253195918 wait complete. 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- start start 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 Processor -- start 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x2458460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 0x2458950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23396e0 con 0x21ee7b0 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2339820 con 0x2180bb0 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x233aa90 con 0x2405d30 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 10 client.0 ll_register_callbacks cb 0x232fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 0x2458950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58890/0 (socket says 172.21.15.185:58890) 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef0972700 1 -- 172.21.15.185:0/2420374007 learned_addr learned my addr 172.21.15.185:0/2420374007 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feeebfff700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x2458460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 -- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 msgr2=0x242b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 -- 172.21.15.185:0/2420374007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 msgr2=0x2458460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x2458460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 -- 172.21.15.185:0/2420374007 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x232a280 con 0x2405d30 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef0972700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef1173700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 0x2458950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7feee400b6e0 tx=0x7feee400b470).ready entity=mon.2 client_cookie=3859ed9ace95bce8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4016070 con 0x2405d30 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feee400be00 con 0x2405d30 2020-11-16T22:16:59.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2401e10 con 0x2405d30 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee400f7f0 con 0x2405d30 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 10 client.5375 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x215ffd0 con 0x2405d30 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2401e10 con 0x2405d30 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7feee401b450 con 0x2405d30 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 10 client.5375 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:59.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 10 client.5375 resolve_mds: resolved ID 'a' to GID 5183 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 20 client.5375 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x242ae60 0x2454870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 4 client.5375 mds_command: new command op to 5183 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 --> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x222ac50 con 0x242ae60 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feef0972700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x242ae60 0x2454870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.307+0000 7feebaffd700 10 client.5375 ms_handle_connect on v2:172.21.15.185:6834/2473550168 2020-11-16T22:16:59.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7feef0972700 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x242ae60 0x2454870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=40e7591aa5799a55 server_cookie=526e7a5cdc8e5109 in_seq=0 out_seq=0 2020-11-16T22:16:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7feebaffd700 1 -- 172.21.15.185:0/2420374007 <== mds.2 v2:172.21.15.185:6834/2473550168 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x222ac50 con 0x242ae60 2020-11-16T22:16:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7feebaffd700 10 client.5375 handle_command_reply: tid=0 2020-11-16T22:16:59.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 client.5375 shutdown 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 msgr2=0x2458950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 0x2458950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7feee400b6e0 tx=0x7feee400b470).stop 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 shutdown_connections 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x242b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x242ae60 0x2454870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2405d30 0x2458950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 --2- 172.21.15.185:0/2420374007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x2458460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7feebaffd700 0 client.5375 ms_handle_reset on v2:172.21.15.185:6834/2473550168 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 >> 172.21.15.185:0/2420374007 conn(0x23bce40 msgr2=0x2440b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 shutdown_connections 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 1 -- 172.21.15.185:0/2420374007 wait complete. 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.311+0000 7fef01667740 20 client.5375 trim_cache size 0 max 16384 2020-11-16T22:16:59.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 Processor -- start 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 -- start start 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 0x24093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x2458460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x243f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2311e40 con 0x21ee7b0 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24254d0 con 0x2180bb0 2020-11-16T22:16:59.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2428140 con 0x23bce40 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 0x24093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7feeebfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x2458460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 0x24093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52342/0 (socket says 172.21.15.185:52342) 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7feef0972700 1 -- 172.21.15.185:0/2184009998 learned_addr learned my addr 172.21.15.185:0/2184009998 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.327+0000 7feef1173700 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x243f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 -- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 msgr2=0x24093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 0x24093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 -- 172.21.15.185:0/2184009998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 msgr2=0x243f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x243f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 -- 172.21.15.185:0/2184009998 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00081d0 con 0x2180bb0 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feeebfff700 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x2458460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7feed4049030 tx=0x7feed4006e40).ready entity=mon.1 client_cookie=f90b1bba2c6a4501 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feee27fc700 1 -- 172.21.15.185:0/2184009998 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed406f030 con 0x2180bb0 2020-11-16T22:16:59.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feee27fc700 1 -- 172.21.15.185:0/2184009998 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40069f0 con 0x2180bb0 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7feee27fc700 1 -- 172.21.15.185:0/2184009998 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400b130 con 0x2180bb0 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 msgr2=0x2458460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x2458460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7feed4049030 tx=0x7feed4006e40).stop 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- 172.21.15.185:0/2184009998 shutdown_connections 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x243f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x23bce40 0x24093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 --2- 172.21.15.185:0/2184009998 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2180bb0 0x2458460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- 172.21.15.185:0/2184009998 >> 172.21.15.185:0/2184009998 conn(0x7feec0002320 msgr2=0x242a880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- 172.21.15.185:0/2184009998 shutdown_connections 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- 172.21.15.185:0/2184009998 wait complete. 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 -- start start 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.331+0000 7fef01667740 1 Processor -- start 2020-11-16T22:16:59.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x24ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x24aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 0x2404ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24446a0 con 0x21ee7b0 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x24ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x243f980 con 0x23bce40 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x243fb70 con 0x2180bb0 2020-11-16T22:16:59.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef0972700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x24ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52348/0 (socket says 172.21.15.185:52348) 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef0972700 1 -- 172.21.15.185:0/2694184919 learned_addr learned my addr 172.21.15.185:0/2694184919 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 0x2404ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feeebfff700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x24aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 10 client.0 ll_register_callbacks cb 0x2408dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 -- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 msgr2=0x24ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x24ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 -- 172.21.15.185:0/2694184919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 msgr2=0x24aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x24aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 -- 172.21.15.185:0/2694184919 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x219e720 con 0x23bce40 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef1173700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 0x2404ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7feee4007fd0 tx=0x7feee401f000).ready entity=mon.1 client_cookie=db844ef9a8ac33ed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee402a070 con 0x23bce40 2020-11-16T22:16:59.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feedc010c10 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feee400bb90 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee400b470 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 10 client.5809 fetch_fsmap learned FSMap version 38 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x21a9e00 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc010c10 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 38) v1 ==== 1485+0+0 (secure 0 0 0) 0x7feee401b450 con 0x23bce40 2020-11-16T22:16:59.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 10 client.5809 fetch_fsmap finished waiting for FSMap version 38 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 10 client.5809 resolve_mds: resolved ID 'a' to GID 5183 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 20 client.5809 populate_metadata read hostname 'smithi185' 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x2405d30 0x24053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 4 client.5809 mds_command: new command op to 5183 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 --> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x2231f50 con 0x2405d30 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef0972700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x2405d30 0x24053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:16:59.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feebaffd700 10 client.5809 ms_handle_connect on v2:172.21.15.185:6834/2473550168 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.335+0000 7feef0972700 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x2405d30 0x24053c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=e51203d8ef30e4d5 server_cookie=f20fb38a816d8ff7 in_seq=0 out_seq=0 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feebaffd700 1 -- 172.21.15.185:0/2694184919 <== mds.2 v2:172.21.15.185:6834/2473550168 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x2231f50 con 0x2405d30 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feebaffd700 10 client.5809 handle_command_reply: tid=0 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 client.5809 shutdown 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 msgr2=0x2404ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 0x2404ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7feee4007fd0 tx=0x7feee401f000).stop 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 shutdown_connections 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23bce40 0x2404ed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x2405d30 0x24053c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2180bb0 0x24ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feebaffd700 0 client.5809 ms_handle_reset on v2:172.21.15.185:6834/2473550168 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 --2- 172.21.15.185:0/2694184919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21ee7b0 0x24aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 >> 172.21.15.185:0/2694184919 conn(0x7feec0002320 msgr2=0x242a880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 shutdown_connections 2020-11-16T22:16:59.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 1 -- 172.21.15.185:0/2694184919 wait complete. 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7fef01667740 20 client.5809 trim_cache size 0 max 16384 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 msgr2=0x7feec8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 0x7feec8057520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7feedc00b680 tx=0x7feedc008040).stop 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 msgr2=0x7feeec0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7feed400beb0 tx=0x7feed4015000).stop 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 shutdown_connections 2020-11-16T22:16:59.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7feec8055060 0x7feec8057520 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeec144340 0x7feeec139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feeec0603f0 0x7feeec0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 --2- 172.21.15.185:0/3118651441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feeec0b37e0 0x7feeec0b0db0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 >> 172.21.15.185:0/3118651441 conn(0x7feeec063f60 msgr2=0x7feeec0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 shutdown_connections 2020-11-16T22:16:59.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:16:59.355+0000 7feef2bd6700 1 -- 172.21.15.185:0/3118651441 wait complete. 2020-11-16T22:16:59.933 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:16:59.933 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:16:59.933 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:16:59.933 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:16:59.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:16:59.934 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:16:59.992 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/ 2020-11-16T22:16:59.992 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4 2020-11-16T22:16:59.993 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5 2020-11-16T22:16:59.993 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:16:59.993 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:16:59.993 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:16:59.997 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:16:59.997 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:16:59.997 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:16:59.998 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:16:59.998 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:16:59.998 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:00.109 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627778 2020-11-16T22:17:00.298 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:17:00.299 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:17:00.299 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:17:00.300 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:17:00.300 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:17:00.300 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:00.412 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627779 2020-11-16T22:17:00.602 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:17:00.602 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:17:00.603 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:17:00.603 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:17:00.603 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:17:00.603 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:00.764 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627780 2020-11-16T22:17:00.954 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:17:00.954 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:17:00.954 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:17:00.955 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:17:00.955 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:17:00.955 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:01.067 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627781 2020-11-16T22:17:01.257 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:17:01.258 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:17:01.258 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:17:01.259 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:17:01.259 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:17:01.259 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:01.376 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627782 2020-11-16T22:17:01.566 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:17:01.566 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:17:01.567 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:17:01.567 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:17:01.567 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:17:01.568 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:17:01.729 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627783 2020-11-16T22:17:01.919 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:17:01.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.967+0000 7f9bfed7ac40 1 Processor -- start 2020-11-16T22:17:01.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.967+0000 7f9bfed7ac40 1 -- start start 2020-11-16T22:17:01.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a091e0 0x55bf82adf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a3da80 0x55bf82ad0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf829b71f0 con 0x55bf82a3da80 2020-11-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf82985be0 con 0x55bf82a091e0 2020-11-16T22:17:01.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf82adfe00 con 0x55bf82a3de60 2020-11-16T22:17:01.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52366/0 (socket says 172.21.15.185:52366) 2020-11-16T22:17:01.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 -- 172.21.15.185:0/809584655 learned_addr learned my addr 172.21.15.185:0/809584655 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:01.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9befb7f700 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a3da80 0x55bf82ad0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 -- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a091e0 msgr2=0x55bf82adf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:01.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a091e0 0x55bf82adf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:17:01.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 -- 172.21.15.185:0/809584655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a3da80 msgr2=0x55bf82ad0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:01.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a3da80 0x55bf82ad0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 -- 172.21.15.185:0/809584655 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf829af8a0 con 0x55bf82a3de60 2020-11-16T22:17:01.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bf0b81700 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x55bf82ad1570 tx=0x7f9be8015000).ready entity=mon.2 client_cookie=77a72ff823e1ca50 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:01.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9beeb7d700 1 -- 172.21.15.185:0/809584655 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8008b20 con 0x55bf82a3de60 2020-11-16T22:17:01.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9beeb7d700 1 -- 172.21.15.185:0/809584655 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be801c480 con 0x55bf82a3de60 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9beeb7d700 1 -- 172.21.15.185:0/809584655 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be801c800 con 0x55bf82a3de60 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 msgr2=0x55bf82ad4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x55bf82ad1570 tx=0x7f9be8015000).stop 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/809584655 shutdown_connections 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a091e0 0x55bf82adf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:17:01.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82ad4820 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/809584655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a3da80 0x55bf82ad0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/809584655 >> 172.21.15.185:0/809584655 conn(0x55bf82a49be0 msgr2=0x55bf82a48f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/809584655 shutdown_connections 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.971+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/809584655 wait complete. 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 Processor -- start 2020-11-16T22:17:01.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- start start 2020-11-16T22:17:01.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a091e0 0x55bf82ad2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a3da80 0x55bf82adda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82addf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf82a10ac0 con 0x55bf82a091e0 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf82ade430 con 0x55bf82a3da80 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf82add5f0 con 0x55bf82a3de60 2020-11-16T22:17:01.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bf0b81700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82addf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a3da80 0x55bf82adda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bf0b81700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82addf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52372/0 (socket says 172.21.15.185:52372) 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bf0b81700 1 -- 172.21.15.185:0/1233126288 learned_addr learned my addr 172.21.15.185:0/1233126288 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bf0380700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a091e0 0x55bf82ad2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 -- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 msgr2=0x55bf82addf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55bf82a3de60 0x55bf82addf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 -- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a091e0 msgr2=0x55bf82ad2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf82a091e0 0x55bf82ad2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 -- 172.21.15.185:0/1233126288 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf829ab280 con 0x55bf82a3da80 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9befb7f700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a3da80 0x55bf82adda50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x55bf82a3ced0 tx=0x7f9be000c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bed37a700 1 -- 172.21.15.185:0/1233126288 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000f040 con 0x55bf82a3da80 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bed37a700 1 -- 172.21.15.185:0/1233126288 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0010950 con 0x55bf82a3da80 2020-11-16T22:17:01.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf82989620 con 0x55bf82a3da80 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bed37a700 1 -- 172.21.15.185:0/1233126288 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0010cd0 con 0x55bf82a3da80 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.975+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf82ad3560 con 0x55bf82a3da80 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bed37a700 1 -- 172.21.15.185:0/1233126288 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9be001b020 con 0x55bf82a3da80 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bed37a700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bd0053ce0 0x7f9bd00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bed37a700 1 -- 172.21.15.185:0/1233126288 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9be006d1b0 con 0x55bf82a3da80 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bf0380700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bd0053ce0 0x7f9bd00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x55bf82ab9b90 0x55bf82af0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:01.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 --> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] -- osd_op(unknown.0.0:1 7.2 7:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x55bf82af0600 con 0x55bf82ab9b90 2020-11-16T22:17:01.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bf0380700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bd0053ce0 0x7f9bd00561a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f9bd8001d70 tx=0x7f9bd8000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:01.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bf0b81700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x55bf82ab9b90 0x55bf82af0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:01.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.979+0000 7f9bf0b81700 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x55bf82ab9b90 0x55bf82af0110 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:01.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bf0b81700 1 -- 172.21.15.185:0/1233126288 <== osd.4 v2:172.21.15.96:6800/15251 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v44'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9be8008430 con 0x55bf82ab9b90 2020-11-16T22:17:01.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x55bf82ab9b90 msgr2=0x55bf82af0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x55bf82ab9b90 0x55bf82af0110 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bd0053ce0 msgr2=0x7f9bd00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bd0053ce0 0x7f9bd00561a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f9bd8001d70 tx=0x7f9bd8000b80).stop 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a3da80 msgr2=0x55bf82adda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 --2- 172.21.15.185:0/1233126288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55bf82a3da80 0x55bf82adda50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x55bf82a3ced0 tx=0x7f9be000c040).stop 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bf0b81700 1 -- 172.21.15.185:0/1233126288 reap_dead start 2020-11-16T22:17:01.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 shutdown_connections 2020-11-16T22:17:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 >> 172.21.15.185:0/1233126288 conn(0x55bf82a49be0 msgr2=0x55bf82a48730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 shutdown_connections 2020-11-16T22:17:01.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:01.983+0000 7f9bfed7ac40 1 -- 172.21.15.185:0/1233126288 wait complete. 2020-11-16T22:17:01.972 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:17:02.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.059+0000 7f1f7b183c40 1 Processor -- start 2020-11-16T22:17:02.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.059+0000 7f1f7b183c40 1 -- start start 2020-11-16T22:17:02.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 0x562ada0d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ada040190 con 0x562ada0d2730 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ada0411f0 con 0x7f1f58003070 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ada00fbe0 con 0x562ada0d2b90 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6c789700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 0x562ada0d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52392/0 (socket says 172.21.15.185:52392) 2020-11-16T22:17:02.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3315918497 learned_addr learned my addr 172.21.15.185:0/3315918497 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6bf88700 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 msgr2=0x562ada0d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 0x562ada0d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3315918497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 msgr2=0x562ada0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3315918497 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ada035280 con 0x562ada0d2b90 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6c789700 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 0x562ada0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x562ada09ac80 tx=0x7f1f64001680).ready entity=mon.2 client_cookie=57fdbc5f46beb2f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6af86700 1 -- 172.21.15.185:0/3315918497 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6400dec0 con 0x562ada0d2b90 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6af86700 1 -- 172.21.15.185:0/3315918497 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64012400 con 0x562ada0d2b90 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f6af86700 1 -- 172.21.15.185:0/3315918497 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640127a0 con 0x562ada0d2b90 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 msgr2=0x562ada0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.063+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x562ada09ac80 tx=0x7f1f64001680).stop 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3315918497 shutdown_connections 2020-11-16T22:17:02.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1f58003070 0x562ada0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x562ada0d2b90 0x562ada0cd820 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3315918497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3315918497 >> 172.21.15.185:0/3315918497 conn(0x562ada0d3be0 msgr2=0x562ada0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3315918497 shutdown_connections 2020-11-16T22:17:02.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3315918497 wait complete. 2020-11-16T22:17:02.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 Processor -- start 2020-11-16T22:17:02.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- start start 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 0x562ada1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 0x562ada145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ada161d00 con 0x562ada0d2730 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ada15fcd0 con 0x562ada0d2b90 2020-11-16T22:17:02.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f7b183c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ada171c70 con 0x7f1f58003070 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6bf88700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 0x562ada1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6cf8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 0x562ada145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6c789700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6bf88700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 0x562ada1455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58954/0 (socket says 172.21.15.185:58954) 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6cf8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 0x562ada145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52400/0 (socket says 172.21.15.185:52400) 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.067+0000 7f1f6bf88700 1 -- 172.21.15.185:0/3741135777 learned_addr learned my addr 172.21.15.185:0/3741135777 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 -- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 msgr2=0x562ada145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 0x562ada145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 -- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 msgr2=0x562ada0c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ada0d2730 0x562ada0c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 -- 172.21.15.185:0/3741135777 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ada013620 con 0x562ada0d2b90 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1f58003070 0x562ada145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f6bf88700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 0x562ada1455e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1f5c012000 tx=0x7f1f5c018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f69783700 1 -- 172.21.15.185:0/3741135777 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00f040 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562ada0cfc90 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f69783700 1 -- 172.21.15.185:0/3741135777 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c014db0 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f69783700 1 -- 172.21.15.185:0/3741135777 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c01f620 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ada0c9e90 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f69783700 1 -- 172.21.15.185:0/3741135777 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1f5c025020 con 0x562ada0d2b90 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.071+0000 7f1f69783700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1f4c053ce0 0x7f1f4c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f69783700 1 -- 172.21.15.185:0/3741135777 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1f5c06e640 con 0x562ada0d2b90 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f6c789700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1f4c053ce0 0x7f1f4c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x562ada15ee50 0x562ada17a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 --> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] -- osd_op(unknown.0.0:1 7.2 7:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x562ada16e4d0 con 0x562ada15ee50 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f6c789700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1f4c053ce0 0x7f1f4c0561a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1f54006630 tx=0x7f1f54008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x562ada15ee50 0x562ada17a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.075+0000 7f1f6cf8a700 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x562ada15ee50 0x562ada17a370 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3741135777 <== osd.4 v2:172.21.15.96:6800/15251 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v44'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1f6400e040 con 0x562ada15ee50 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x562ada15ee50 msgr2=0x562ada17a370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.96:6800/15251,v1:172.21.15.96:6801/15251] conn(0x562ada15ee50 0x562ada17a370 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1f4c053ce0 msgr2=0x7f1f4c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1f4c053ce0 0x7f1f4c0561a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1f54006630 tx=0x7f1f54008040).stop 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 msgr2=0x562ada1455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 --2- 172.21.15.185:0/3741135777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x562ada0d2b90 0x562ada1455e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1f5c012000 tx=0x7f1f5c018000).stop 2020-11-16T22:17:02.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f6cf8a700 1 -- 172.21.15.185:0/3741135777 reap_dead start 2020-11-16T22:17:02.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 shutdown_connections 2020-11-16T22:17:02.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 >> 172.21.15.185:0/3741135777 conn(0x562ada0d3be0 msgr2=0x562ada146380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 shutdown_connections 2020-11-16T22:17:02.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.079+0000 7f1f7b183c40 1 -- 172.21.15.185:0/3741135777 wait complete. 2020-11-16T22:17:02.265 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:17:02.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 Processor -- start 2020-11-16T22:17:02.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 -- start start 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f78bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f8220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f701190 con 0x55ff4f78b850 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f7021f0 con 0x55ff4f790440 2020-11-16T22:17:02.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f6d0be0 con 0x55ff4f818ca0 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a6b3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f8220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a5b3a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a6b3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f8220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52414/0 (socket says 172.21.15.185:52414) 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a6b3c700 1 -- 172.21.15.185:0/4043024127 learned_addr learned my addr 172.21.15.185:0/4043024127 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a5b3a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58972/0 (socket says 172.21.15.185:58972) 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.347+0000 7ff7a633b700 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f78bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 msgr2=0x55ff4f8220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f8220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/4043024127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 msgr2=0x55ff4f78bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f78bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/4043024127 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4f6f6280 con 0x55ff4f790440 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7ff79c00b530 tx=0x7ff79c002840).ready entity=mon.1 client_cookie=dd9c0f466af0f1af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a4b38700 1 -- 172.21.15.185:0/4043024127 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00d490 con 0x55ff4f790440 2020-11-16T22:17:02.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a4b38700 1 -- 172.21.15.185:0/4043024127 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c00d5f0 con 0x55ff4f790440 2020-11-16T22:17:02.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7a4b38700 1 -- 172.21.15.185:0/4043024127 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00d970 con 0x55ff4f790440 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 msgr2=0x55ff4f78c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7ff79c00b530 tx=0x7ff79c002840).stop 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/4043024127 shutdown_connections 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f8220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f78bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/4043024127 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f78c760 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/4043024127 >> 172.21.15.185:0/4043024127 conn(0x55ff4f791670 msgr2=0x55ff4f742c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/4043024127 shutdown_connections 2020-11-16T22:17:02.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/4043024127 wait complete. 2020-11-16T22:17:02.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 Processor -- start 2020-11-16T22:17:02.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.351+0000 7ff7b4d35c40 1 -- start start 2020-11-16T22:17:02.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f81fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f81bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f819c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f81b300 con 0x55ff4f78b850 2020-11-16T22:17:02.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f8332d0 con 0x55ff4f790440 2020-11-16T22:17:02.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f819260 con 0x55ff4f818ca0 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a6b3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f819c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f81bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a6b3c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f819c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52422/0 (socket says 172.21.15.185:52422) 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a6b3c700 1 -- 172.21.15.185:0/122114667 learned_addr learned my addr 172.21.15.185:0/122114667 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a633b700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f81fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 msgr2=0x55ff4f819c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ff4f818ca0 0x55ff4f819c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/122114667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 msgr2=0x55ff4f81fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff4f78b850 0x55ff4f81fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 -- 172.21.15.185:0/122114667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4f6d4620 con 0x55ff4f790440 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7a5b3a700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f81bfa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x55ff4f754020 tx=0x55ff4f81db90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff796ffd700 1 -- 172.21.15.185:0/122114667 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c020070 con 0x55ff4f790440 2020-11-16T22:17:02.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff796ffd700 1 -- 172.21.15.185:0/122114667 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c010880 con 0x55ff4f790440 2020-11-16T22:17:02.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff4f819500 con 0x55ff4f790440 2020-11-16T22:17:02.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff796ffd700 1 -- 172.21.15.185:0/122114667 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00d970 con 0x55ff4f790440 2020-11-16T22:17:02.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.355+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff4f81c490 con 0x55ff4f790440 2020-11-16T22:17:02.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff796ffd700 1 -- 172.21.15.185:0/122114667 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff79c01c430 con 0x55ff4f790440 2020-11-16T22:17:02.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff796ffd700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff788053ce0 0x7ff7880561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff796ffd700 1 -- 172.21.15.185:0/122114667 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff79c06de70 con 0x55ff4f790440 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7a633b700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff788053ce0 0x7ff7880561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55ff4f83d7e0 0x55ff4f83fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7a633b700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff788053ce0 0x7ff7880561a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ff798005d40 tx=0x7ff798015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 7.5 7:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x55ff4f8400c0 con 0x55ff4f83d7e0 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7a6b3c700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55ff4f83d7e0 0x55ff4f83fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.359+0000 7ff7a6b3c700 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55ff4f83d7e0 0x55ff4f83fbd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7a6b3c700 1 -- 172.21.15.185:0/122114667 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v44'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff7a000e750 con 0x55ff4f83d7e0 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55ff4f83d7e0 msgr2=0x55ff4f83fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55ff4f83d7e0 0x55ff4f83fbd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff788053ce0 msgr2=0x7ff7880561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff788053ce0 0x7ff7880561a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ff798005d40 tx=0x7ff798015000).stop 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 msgr2=0x55ff4f81bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 --2- 172.21.15.185:0/122114667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ff4f790440 0x55ff4f81bfa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x55ff4f754020 tx=0x55ff4f81db90).stop 2020-11-16T22:17:02.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7a6b3c700 1 -- 172.21.15.185:0/122114667 reap_dead start 2020-11-16T22:17:02.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 shutdown_connections 2020-11-16T22:17:02.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 >> 172.21.15.185:0/122114667 conn(0x55ff4f791670 msgr2=0x55ff4f742c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 shutdown_connections 2020-11-16T22:17:02.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.363+0000 7ff7b4d35c40 1 -- 172.21.15.185:0/122114667 wait complete. 2020-11-16T22:17:02.550 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:17:02.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.635+0000 7f083e4a7c40 1 Processor -- start 2020-11-16T22:17:02.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.635+0000 7f083e4a7c40 1 -- start start 2020-11-16T22:17:02.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f2997dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f299750620 0x55f2997eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2996c5190 con 0x55f299750620 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2996c61f0 con 0x55f29974ce60 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f299694be0 con 0x55f29974ca80 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082f2ac700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082f2ac700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:58992/0 (socket says 172.21.15.185:58992) 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082f2ac700 1 -- 172.21.15.185:0/2000367001 learned_addr learned my addr 172.21.15.185:0/2000367001 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f2997dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f08302ae700 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f299750620 0x55f2997eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 -- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 msgr2=0x55f2997dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 -- 172.21.15.185:0/2000367001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f299750620 msgr2=0x55f2997eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f299750620 0x55f2997eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 -- 172.21.15.185:0/2000367001 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2996ba280 con 0x55f29974ca80 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082f2ac700 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082faad700 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f2997dd0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f081800b500 tx=0x7f0818001760).ready entity=mon.2 client_cookie=e481b815eeb634e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082e2aa700 1 -- 172.21.15.185:0/2000367001 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800d490 con 0x55f29974ca80 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082e2aa700 1 -- 172.21.15.185:0/2000367001 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800d5f0 con 0x55f29974ca80 2020-11-16T22:17:02.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.639+0000 7f082e2aa700 1 -- 172.21.15.185:0/2000367001 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800d970 con 0x55f29974ca80 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 msgr2=0x55f2997dd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f2997dd0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f081800b500 tx=0x7f0818001760).stop 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2000367001 shutdown_connections 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f299750620 0x55f2997eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f2997dd0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2000367001 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f29974ce60 0x55f2997dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2000367001 >> 172.21.15.185:0/2000367001 conn(0x55f299758be0 msgr2=0x55f299757730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2000367001 shutdown_connections 2020-11-16T22:17:02.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2000367001 wait complete. 2020-11-16T22:17:02.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 Processor -- start 2020-11-16T22:17:02.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.643+0000 7f083e4a7c40 1 -- start start 2020-11-16T22:17:02.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f299755ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f29974ce60 0x55f2997e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f299750620 0x55f299751cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2997561d0 con 0x55f29974ce60 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2997e2fc0 con 0x55f299750620 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f299753dd0 con 0x55f29974ca80 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f299755ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f08302ae700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f299750620 0x55f299751cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f299755ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52440/0 (socket says 172.21.15.185:52440) 2020-11-16T22:17:02.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 -- 172.21.15.185:0/2071972812 learned_addr learned my addr 172.21.15.185:0/2071972812 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082f2ac700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f29974ce60 0x55f2997e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 -- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f299750620 msgr2=0x55f299751cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55f299750620 0x55f299751cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 -- 172.21.15.185:0/2071972812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f29974ce60 msgr2=0x55f2997e1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f29974ce60 0x55f2997e1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 -- 172.21.15.185:0/2071972812 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f299698620 con 0x55f29974ca80 2020-11-16T22:17:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082faad700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f299755ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0818001870 tx=0x7f0818001760).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 -- 172.21.15.185:0/2071972812 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010a80 con 0x55f29974ca80 2020-11-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 -- 172.21.15.185:0/2071972812 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818010be0 con 0x55f29974ca80 2020-11-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 -- 172.21.15.185:0/2071972812 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081801d460 con 0x55f29974ca80 2020-11-16T22:17:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f299755840 con 0x55f29974ca80 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2997521c0 con 0x55f29974ca80 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 -- 172.21.15.185:0/2071972812 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0818023020 con 0x55f29974ca80 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0810053ce0 0x7f08100561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082caa7700 1 -- 172.21.15.185:0/2071972812 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f081806e240 con 0x55f29974ca80 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.647+0000 7f082f2ac700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0810053ce0 0x7f08100561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.651+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55f2997541a0 0x55f299753640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.651+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- osd_op(unknown.0.0:1 7.1 7:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x55f2997ff030 con 0x55f2997541a0 2020-11-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.651+0000 7f08302ae700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55f2997541a0 0x55f299753640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.651+0000 7f082f2ac700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0810053ce0 0x7f08100561a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x55f299704860 tx=0x7f08200018f0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.651+0000 7f08302ae700 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55f2997541a0 0x55f299753640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f08302ae700 1 -- 172.21.15.185:0/2071972812 <== osd.2 v2:172.21.15.185:6800/15193 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v44'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f082800a560 con 0x55f2997541a0 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55f2997541a0 msgr2=0x55f299753640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55f2997541a0 0x55f299753640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0810053ce0 msgr2=0x7f08100561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0810053ce0 0x7f08100561a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x55f299704860 tx=0x7f08200018f0).stop 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 msgr2=0x55f299755ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 --2- 172.21.15.185:0/2071972812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55f29974ca80 0x55f299755ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f0818001870 tx=0x7f0818001760).stop 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f08302ae700 1 -- 172.21.15.185:0/2071972812 reap_dead start 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 shutdown_connections 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 >> 172.21.15.185:0/2071972812 conn(0x55f299758be0 msgr2=0x55f29974e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 shutdown_connections 2020-11-16T22:17:02.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.655+0000 7f083e4a7c40 1 -- 172.21.15.185:0/2071972812 wait complete. 2020-11-16T22:17:02.842 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:17:02.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 Processor -- start 2020-11-16T22:17:02.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 -- start start 2020-11-16T22:17:02.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e388094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3880d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e386ee1f0 con 0x555e387779d0 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e386bcbe0 con 0x555e38778d20 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e3880c440 con 0x555e3880d720 2020-11-16T22:17:02.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52456/0 (socket says 172.21.15.185:52456) 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 -- 172.21.15.185:0/1716236814 learned_addr learned my addr 172.21.15.185:0/1716236814 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581051700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e388094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd580850700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3880d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 -- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 msgr2=0x555e3880d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3880d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 -- 172.21.15.185:0/1716236814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 msgr2=0x555e388094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e388094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 -- 172.21.15.185:0/1716236814 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e386e68a0 con 0x555e3880d720 2020-11-16T22:17:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581051700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e388094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:02.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd581852700 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd57c002c40 tx=0x7fd57c005d00).ready entity=mon.2 client_cookie=5fcf6d5856eb4df5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.931+0000 7fd5737fe700 1 -- 172.21.15.185:0/1716236814 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00d9d0 con 0x555e3880d720 2020-11-16T22:17:02.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd5737fe700 1 -- 172.21.15.185:0/1716236814 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00db30 con 0x555e3880d720 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd5737fe700 1 -- 172.21.15.185:0/1716236814 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00deb0 con 0x555e3880d720 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 msgr2=0x555e3877bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd57c002c40 tx=0x7fd57c005d00).stop 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/1716236814 shutdown_connections 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3877bfa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e388094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/1716236814 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3880d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/1716236814 >> 172.21.15.185:0/1716236814 conn(0x555e3877a470 msgr2=0x555e388085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/1716236814 shutdown_connections 2020-11-16T22:17:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/1716236814 wait complete. 2020-11-16T22:17:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 Processor -- start 2020-11-16T22:17:02.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.935+0000 7fd58fa4bc40 1 -- start start 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e3880cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3881b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3881e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3880db00 con 0x555e387779d0 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3880dd30 con 0x555e38778d20 2020-11-16T22:17:02.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e3881ae10 con 0x555e3880d720 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3881e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd580850700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3881b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3881e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52464/0 (socket says 172.21.15.185:52464) 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 -- 172.21.15.185:0/2158219819 learned_addr learned my addr 172.21.15.185:0/2158219819 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581051700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e3880cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 -- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 msgr2=0x555e3881b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3881b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 -- 172.21.15.185:0/2158219819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 msgr2=0x555e3880cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e387779d0 0x555e3880cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 -- 172.21.15.185:0/2158219819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e386e2280 con 0x555e3880d720 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd580850700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x555e38778d20 0x555e3881b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd581852700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3881e110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fd57c005ae0 tx=0x7fd57c018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd571ffb700 1 -- 172.21.15.185:0/2158219819 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01c620 con 0x555e3880d720 2020-11-16T22:17:02.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd571ffb700 1 -- 172.21.15.185:0/2158219819 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c020070 con 0x555e3880d720 2020-11-16T22:17:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555e386c0620 con 0x555e3880d720 2020-11-16T22:17:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd571ffb700 1 -- 172.21.15.185:0/2158219819 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01c7c0 con 0x555e3880d720 2020-11-16T22:17:02.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.939+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e3881ab40 con 0x555e3880d720 2020-11-16T22:17:03.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd571ffb700 1 -- 172.21.15.185:0/2158219819 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd57c01cbb0 con 0x555e3880d720 2020-11-16T22:17:03.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd571ffb700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd564053ce0 0x7fd5640561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd571ffb700 1 -- 172.21.15.185:0/2158219819 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd57c06eb00 con 0x555e3880d720 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd581051700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd564053ce0 0x7fd5640561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x555e3881b7a0 0x555e388274c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 7.5 7:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x555e388279b0 con 0x555e3881b7a0 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd580850700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x555e3881b7a0 0x555e388274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd581051700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd564053ce0 0x7fd5640561a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x555e387536c0 tx=0x7fd574000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.943+0000 7fd580850700 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x555e3881b7a0 0x555e388274c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd580850700 1 -- 172.21.15.185:0/2158219819 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v44'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd578000b80 con 0x555e3881b7a0 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x555e3881b7a0 msgr2=0x555e388274c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x555e3881b7a0 0x555e388274c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd564053ce0 msgr2=0x7fd5640561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd564053ce0 0x7fd5640561a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x555e387536c0 tx=0x7fd574000b80).stop 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 msgr2=0x555e3881e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.134 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 --2- 172.21.15.185:0/2158219819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x555e3880d720 0x555e3881e110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fd57c005ae0 tx=0x7fd57c018000).stop 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd581852700 1 -- 172.21.15.185:0/2158219819 reap_dead start 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 shutdown_connections 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 >> 172.21.15.185:0/2158219819 conn(0x555e3877a470 msgr2=0x555e3877cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 shutdown_connections 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:02.947+0000 7fd58fa4bc40 1 -- 172.21.15.185:0/2158219819 wait complete. 2020-11-16T22:17:03.135 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:17:03.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.219+0000 7feebadf0c40 1 Processor -- start 2020-11-16T22:17:03.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.219+0000 7feebadf0c40 1 -- start start 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f6765fe40 0x563f6764b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f676f3490 0x563f6764bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f675d6190 con 0x563f676f3490 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f675d71f0 con 0x563f6762aa40 2020-11-16T22:17:03.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f675a5be0 con 0x563f6765fe40 2020-11-16T22:17:03.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeabbf5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f6765fe40 0x563f6764b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59032/0 (socket says 172.21.15.185:59032) 2020-11-16T22:17:03.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 -- 172.21.15.185:0/988469796 learned_addr learned my addr 172.21.15.185:0/988469796 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeacbf7700 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f676f3490 0x563f6764bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 -- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f6765fe40 msgr2=0x563f6764b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f6765fe40 0x563f6764b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 -- 172.21.15.185:0/988469796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f676f3490 msgr2=0x563f6764bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f676f3490 0x563f6764bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 -- 172.21.15.185:0/988469796 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f675cb280 con 0x563f6762aa40 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeac3f6700 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7feea4001ad0 tx=0x7feea4006930).ready entity=mon.1 client_cookie=fa3d02050daef743 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeaabf3700 1 -- 172.21.15.185:0/988469796 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400b5e0 con 0x563f6762aa40 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeaabf3700 1 -- 172.21.15.185:0/988469796 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400b740 con 0x563f6762aa40 2020-11-16T22:17:03.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feeaabf3700 1 -- 172.21.15.185:0/988469796 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400bb00 con 0x563f6762aa40 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 -- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 msgr2=0x563f67628540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7feea4001ad0 tx=0x7feea4006930).stop 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 -- 172.21.15.185:0/988469796 shutdown_connections 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67628540 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f676f3490 0x563f6764bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.223+0000 7feebadf0c40 1 --2- 172.21.15.185:0/988469796 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f6765fe40 0x563f6764b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- 172.21.15.185:0/988469796 >> 172.21.15.185:0/988469796 conn(0x563f67663370 msgr2=0x563f67649b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- 172.21.15.185:0/988469796 shutdown_connections 2020-11-16T22:17:03.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- 172.21.15.185:0/988469796 wait complete. 2020-11-16T22:17:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 Processor -- start 2020-11-16T22:17:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- start start 2020-11-16T22:17:03.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67703e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f6765fe40 0x563f676f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 0x563f676f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f676f74a0 con 0x563f6765fe40 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f676f6890 con 0x563f6762aa40 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f6764adc0 con 0x563f676f3490 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 0x563f676f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 0x563f676f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52486/0 (socket says 172.21.15.185:52486) 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 -- 172.21.15.185:0/3049046247 learned_addr learned my addr 172.21.15.185:0/3049046247 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeac3f6700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67703e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeabbf5700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f6765fe40 0x563f676f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 -- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 msgr2=0x563f67703e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563f6762aa40 0x563f67703e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 -- 172.21.15.185:0/3049046247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f6765fe40 msgr2=0x563f676f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f6765fe40 0x563f676f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 -- 172.21.15.185:0/3049046247 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f675a9620 con 0x563f676f3490 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feeacbf7700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 0x563f676f07d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fee9c00b640 tx=0x7fee9c001700).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feea93f0700 1 -- 172.21.15.185:0/3049046247 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f410 con 0x563f676f3490 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feea93f0700 1 -- 172.21.15.185:0/3049046247 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00f570 con 0x563f676f3490 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563f676f6ac0 con 0x563f676f3490 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.227+0000 7feea93f0700 1 -- 172.21.15.185:0/3049046247 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f910 con 0x563f676f3490 2020-11-16T22:17:03.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f676f7000 con 0x563f676f3490 2020-11-16T22:17:03.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feea93f0700 1 -- 172.21.15.185:0/3049046247 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fee9c01f070 con 0x563f676f3490 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feea93f0700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fee94053ce0 0x7fee940561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feeac3f6700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fee94053ce0 0x7fee940561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feea93f0700 1 -- 172.21.15.185:0/3049046247 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fee9c06cf20 con 0x563f676f3490 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feebadf0c40 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x563f676f9a00 0x563f676f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 7.5 7:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e44) v8 -- 0x563f676fa300 con 0x563f676f9a00 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feeac3f6700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fee94053ce0 0x7fee940561a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7feea40012f0 tx=0x7feea4016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feeabbf5700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x563f676f9a00 0x563f676f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.231+0000 7feeabbf5700 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x563f676f9a00 0x563f676f9e10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feeabbf5700 1 -- 172.21.15.185:0/3049046247 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v44'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fee98000b80 con 0x563f676f9a00 2020-11-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x563f676f9a00 msgr2=0x563f676f9e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x563f676f9a00 0x563f676f9e10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fee94053ce0 msgr2=0x7fee940561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fee94053ce0 0x7fee940561a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7feea40012f0 tx=0x7feea4016000).stop 2020-11-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 msgr2=0x563f676f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 --2- 172.21.15.185:0/3049046247 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563f676f3490 0x563f676f07d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fee9c00b640 tx=0x7fee9c001700).stop 2020-11-16T22:17:03.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feeacbf7700 1 -- 172.21.15.185:0/3049046247 reap_dead start 2020-11-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 shutdown_connections 2020-11-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 >> 172.21.15.185:0/3049046247 conn(0x563f67663370 msgr2=0x563f676f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 shutdown_connections 2020-11-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.235+0000 7feebadf0c40 1 -- 172.21.15.185:0/3049046247 wait complete. 2020-11-16T22:17:03.426 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:03.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 Processor -- start 2020-11-16T22:17:03.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- start start 2020-11-16T22:17:03.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdb780b17e0 0x7fdb78067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb78098cc0 con 0x7fdb78066240 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb780b3950 con 0x7fdb780b17e0 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b3bf0 con 0x7fdb780b2160 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52496/0 (socket says 172.21.15.185:52496) 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 -- 172.21.15.185:0/797050939 learned_addr learned my addr 172.21.15.185:0/797050939 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7659c700 1 --2- 172.21.15.185:0/797050939 >> v2:172.21.15.96:3300/0 conn(0x7fdb780b17e0 0x7fdb78067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 -- 172.21.15.185:0/797050939 >> v2:172.21.15.96:3300/0 conn(0x7fdb780b17e0 msgr2=0x7fdb78067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 --2- 172.21.15.185:0/797050939 >> v2:172.21.15.96:3300/0 conn(0x7fdb780b17e0 0x7fdb78067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 -- 172.21.15.185:0/797050939 >> v1:172.21.15.96:6789/0 conn(0x7fdb78066240 legacy=0x7fdb78065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 -- 172.21.15.185:0/797050939 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78067b10 con 0x7fdb780b2160 2020-11-16T22:17:03.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb75d9b700 1 --2- 172.21.15.185:0/797050939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fdb60006fd0 tx=0x7fdb60006eb0).ready entity=mon.2 client_cookie=9e89d6285c710f5f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7559a700 1 -- 172.21.15.185:0/797050939 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60009810 con 0x7fdb780b2160 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7559a700 1 -- 172.21.15.185:0/797050939 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60009970 con 0x7fdb780b2160 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7559a700 1 -- 172.21.15.185:0/797050939 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60009cf0 con 0x7fdb780b2160 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/797050939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 msgr2=0x7fdb7814bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/797050939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fdb60006fd0 tx=0x7fdb60006eb0).stop 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/797050939 shutdown_connections 2020-11-16T22:17:03.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/797050939 >> v2:172.21.15.96:3300/0 conn(0x7fdb780b17e0 0x7fdb78067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/797050939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb7814bde0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/797050939 >> 172.21.15.185:0/797050939 conn(0x7fdb780604a0 msgr2=0x7fdb780608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/797050939 shutdown_connections 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.611+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/797050939 wait complete. 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 Processor -- start 2020-11-16T22:17:03.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 -- start start 2020-11-16T22:17:03.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb78066240 0x7fdb78157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdb780b17e0 0x7fdb7815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b0d00 con 0x7fdb78066240 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780ad250 con 0x7fdb780b17e0 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7cfb0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7815b5f0 con 0x7fdb780b2160 2020-11-16T22:17:03.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb7659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb78066240 0x7fdb78157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb75d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdb780b17e0 0x7fdb7815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52502/0 (socket says 172.21.15.185:52502) 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 -- 172.21.15.185:0/2223325993 learned_addr learned my addr 172.21.15.185:0/2223325993 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 -- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdb780b17e0 msgr2=0x7fdb7815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdb780b17e0 0x7fdb7815bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 -- 172.21.15.185:0/2223325993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb78066240 msgr2=0x7fdb78157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb78066240 0x7fdb78157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 -- 172.21.15.185:0/2223325993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7815b890 con 0x7fdb780b2160 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.615+0000 7fdb76d9d700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdb7000c710 tx=0x7fdb70007af0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70013410 con 0x7fdb780b2160 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb781583b0 con 0x7fdb780b2160 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70013570 con 0x7fdb780b2160 2020-11-16T22:17:03.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70013910 con 0x7fdb780b2160 2020-11-16T22:17:03.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb780ab680 con 0x7fdb780b2160 2020-11-16T22:17:03.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7815ba70 con 0x7fdb780b2160 2020-11-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdb70010030 con 0x7fdb780b2160 2020-11-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 0x7fdb54057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdb7006d9e0 con 0x7fdb780b2160 2020-11-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb7659c700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 0x7fdb54057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.619+0000 7fdb7659c700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 0x7fdb54057520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdb780b5e20 tx=0x7fdb680091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.631+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdb7815ba70 con 0x7fdb780b2160 2020-11-16T22:17:03.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb7815ba70 con 0x7fdb780b2160 2020-11-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb6e7fc700 1 -- 172.21.15.185:0/2223325993 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2522 (secure 0 0 0) 0x7fdb7815ba70 con 0x7fdb780b2160 2020-11-16T22:17:03.814 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:03.815 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:59.983598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 msgr2=0x7fdb54057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 0x7fdb54057520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdb780b5e20 tx=0x7fdb680091d0).stop 2020-11-16T22:17:03.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 msgr2=0x7fdb780ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdb7000c710 tx=0x7fdb70007af0).stop 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 shutdown_connections 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdb54055060 0x7fdb54057520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdb780b2160 0x7fdb780ac2b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb78066240 0x7fdb78157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 --2- 172.21.15.185:0/2223325993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdb780b17e0 0x7fdb7815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 >> 172.21.15.185:0/2223325993 conn(0x7fdb780604a0 msgr2=0x7fdb7814c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 shutdown_connections 2020-11-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.831+0000 7fdb7cfb0700 1 -- 172.21.15.185:0/2223325993 wait complete. 2020-11-16T22:17:03.818 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 39 2020-11-16T22:17:03.828 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub start /d1/d2/d3 recursive force 2020-11-16T22:17:03.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 Processor -- start 2020-11-16T22:17:03.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 -- start start 2020-11-16T22:17:03.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff8800acc10 0x7ff8801432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8800abf90 con 0x7ff8800669a0 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8800ac230 con 0x7ff8800acc10 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff884cd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800ac4d0 con 0x7ff880061e60 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52510/0 (socket says 172.21.15.185:52510) 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 -- 172.21.15.185:0/1395976166 learned_addr learned my addr 172.21.15.185:0/1395976166 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff8800669a0 0x7ff880066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52502/0 (socket says 172.21.15.185:52502) 2020-11-16T22:17:03.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87effd700 1 --2- 172.21.15.185:0/1395976166 >> v2:172.21.15.96:3300/0 conn(0x7ff8800acc10 0x7ff8801432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 -- 172.21.15.185:0/1395976166 >> v2:172.21.15.96:3300/0 conn(0x7ff8800acc10 msgr2=0x7ff8801432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87e7fc700 1 -- 172.21.15.185:0/1395976166 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408695237 0 0) 0x7ff8800abf90 con 0x7ff8800669a0 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1395976166 >> v2:172.21.15.96:3300/0 conn(0x7ff8800acc10 0x7ff8801432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 -- 172.21.15.185:0/1395976166 >> v1:172.21.15.185:6789/0 conn(0x7ff8800669a0 legacy=0x7ff880066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.975+0000 7ff87ffff700 1 -- 172.21.15.185:0/1395976166 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800ac770 con 0x7ff880061e60 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1395976166 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff878006fd0 tx=0x7ff878006eb0).ready entity=mon.2 client_cookie=31962f90afc1676e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff87e7fc700 1 -- 172.21.15.185:0/1395976166 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878009810 con 0x7ff880061e60 2020-11-16T22:17:03.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff87e7fc700 1 -- 172.21.15.185:0/1395976166 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878009970 con 0x7ff880061e60 2020-11-16T22:17:03.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff87e7fc700 1 -- 172.21.15.185:0/1395976166 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878009cf0 con 0x7ff880061e60 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 -- 172.21.15.185:0/1395976166 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 msgr2=0x7ff8801437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1395976166 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff878006fd0 tx=0x7ff878006eb0).stop 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 -- 172.21.15.185:0/1395976166 shutdown_connections 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1395976166 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff880061e60 0x7ff8801437d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1395976166 >> v2:172.21.15.96:3300/0 conn(0x7ff8800acc10 0x7ff8801432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 -- 172.21.15.185:0/1395976166 >> 172.21.15.185:0/1395976166 conn(0x7ff880060130 msgr2=0x7ff880060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 -- 172.21.15.185:0/1395976166 shutdown_connections 2020-11-16T22:17:03.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 -- 172.21.15.185:0/1395976166 wait complete. 2020-11-16T22:17:03.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.979+0000 7ff884cd8700 1 Processor -- start 2020-11-16T22:17:03.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- start start 2020-11-16T22:17:03.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8800669a0 0x7ff880140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 0x7ff880141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880141930 con 0x7ff8800669a0 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800ae950 con 0x7ff880061e60 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800aebf0 con 0x7ff8800ae260 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59068/0 (socket says 172.21.15.185:59068) 2020-11-16T22:17:03.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87f7fe700 1 -- 172.21.15.185:0/1131890665 learned_addr learned my addr 172.21.15.185:0/1131890665 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 0x7ff880141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87effd700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8800669a0 0x7ff880140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 -- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 msgr2=0x7ff8800ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 -- 172.21.15.185:0/1131890665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8800669a0 msgr2=0x7ff880140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8800669a0 0x7ff880140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 -- 172.21.15.185:0/1131890665 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800aee90 con 0x7ff8800ae260 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 0x7ff880141440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff878012000 tx=0x7ff878015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff8777fe700 1 -- 172.21.15.185:0/1131890665 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878009810 con 0x7ff8800ae260 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff8777fe700 1 -- 172.21.15.185:0/1131890665 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878006b40 con 0x7ff8800ae260 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff8777fe700 1 -- 172.21.15.185:0/1131890665 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87801d820 con 0x7ff8800ae260 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8800af130 con 0x7ff8800ae260 2020-11-16T22:17:03.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.983+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8800adbd0 con 0x7ff8800ae260 2020-11-16T22:17:03.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.987+0000 7ff8777fe700 1 -- 172.21.15.185:0/1131890665 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff87801a040 con 0x7ff8800ae260 2020-11-16T22:17:03.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.987+0000 7ff8777fe700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 0x7ff850057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.987+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 0x7ff850057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.987+0000 7ff8777fe700 1 -- 172.21.15.185:0/1131890665 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff87806eff0 con 0x7ff8800ae260 2020-11-16T22:17:03.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.987+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 0x7ff850057520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff870001520 tx=0x7ff870007e60).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 Processor -- start 2020-11-16T22:17:03.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- start start 2020-11-16T22:17:03.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xe88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xe86f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xd14560 con 0xc6a860 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xdc89e0 con 0xbfcc60 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xc887a0 con 0xe88e50 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xe86f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xe86f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59078/0 (socket says 172.21.15.185:59078) 2020-11-16T22:17:03.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52524/0 (socket says 172.21.15.185:52524) 2020-11-16T22:17:03.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87effd700 1 -- 172.21.15.185:0/2941214037 learned_addr learned my addr 172.21.15.185:0/2941214037 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xe88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 -- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 msgr2=0xe86f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xe86f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 -- 172.21.15.185:0/2941214037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 msgr2=0xe88960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xe88960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 -- 172.21.15.185:0/2941214037 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc83650 con 0xe88e50 2020-11-16T22:17:03.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff87ffff700 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7ff878021040 tx=0x7ff878011000).ready entity=mon.2 client_cookie=36d5c6f63e9c624d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff874ff9700 1 -- 172.21.15.185:0/2941214037 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878048b00 con 0xe88e50 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff874ff9700 1 -- 172.21.15.185:0/2941214037 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878048920 con 0xe88e50 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff874ff9700 1 -- 172.21.15.185:0/2941214037 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87801dbd0 con 0xe88e50 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 msgr2=0xe87430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7ff878021040 tx=0x7ff878011000).stop 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- 172.21.15.185:0/2941214037 shutdown_connections 2020-11-16T22:17:03.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe87430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xe88960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 --2- 172.21.15.185:0/2941214037 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xe86f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- 172.21.15.185:0/2941214037 >> 172.21.15.185:0/2941214037 conn(0xe38ef0 msgr2=0xe85e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- 172.21.15.185:0/2941214037 shutdown_connections 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.991+0000 7ff8949cb740 1 -- 172.21.15.185:0/2941214037 wait complete. 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- start start 2020-11-16T22:17:03.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 Processor -- start 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xebc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xed0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe8a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xdb5790 con 0xc6a860 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xdb58d0 con 0xbfcc60 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xdb6b40 con 0xe88e50 2020-11-16T22:17:03.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 10 client.0 ll_register_callbacks cb 0xdabb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xebc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe8a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xebc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59082/0 (socket says 172.21.15.185:59082) 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87f7fe700 1 -- 172.21.15.185:0/1680840018 learned_addr learned my addr 172.21.15.185:0/1680840018 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87effd700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xed0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 -- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 msgr2=0xebc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xebc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 -- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 msgr2=0xed0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xed0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 -- 172.21.15.185:0/1680840018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xda6330 con 0xe88e50 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87effd700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xed0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe8a020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7ff8780454c0 tx=0x7ff878029000).ready entity=mon.2 client_cookie=aa40e79bed4a82ec server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:03.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878044db0 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff878044f10 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe69770 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8780450d0 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 10 client.5441 fetch_fsmap learned FSMap version 39 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0xbdc080 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe69770 con 0xe88e50 2020-11-16T22:17:03.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 39) v1 ==== 1489+0+0 (secure 0 0 0) 0x7ff878048130 con 0xe88e50 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 10 client.5441 fetch_fsmap finished waiting for FSMap version 39 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 10 client.5441 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 20 client.5441 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xebe670 0xecc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 4 client.5441 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xca6d00 con 0xebe670 2020-11-16T22:17:03.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xebe670 0xecc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xebe670 0xecc880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c9b28d89ac0da5a0 server_cookie=a3c9b80791800cc in_seq=0 out_seq=0 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.995+0000 7ff86dffb700 10 client.5441 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff86dffb700 1 -- 172.21.15.185:0/1680840018 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xca6d00 con 0xebe670 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff86dffb700 10 client.5441 handle_command_reply: tid=0 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 client.5441 shutdown 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 msgr2=0xe8a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:03.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe8a020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7ff8780454c0 tx=0x7ff878029000).stop 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 shutdown_connections 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xebe670 0xecc880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbfcc60 0xebc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff86dffb700 0 client.5441 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe88e50 0xe8a020 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1680840018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc6a860 0xed0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:03.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 >> 172.21.15.185:0/1680840018 conn(0xe38ef0 msgr2=0xeb8810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:03.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 shutdown_connections 2020-11-16T22:17:03.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 1 -- 172.21.15.185:0/1680840018 wait complete. 2020-11-16T22:17:03.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:03.999+0000 7ff8949cb740 20 client.5441 trim_cache size 0 max 16384 2020-11-16T22:17:03.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 Processor -- start 2020-11-16T22:17:03.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 -- start start 2020-11-16T22:17:04.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe38ef0 0xeb5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 0xe89440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xe697a0 con 0xbfcc60 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xd8def0 con 0xe38ef0 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe88320 con 0xc6a860 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52534/0 (socket says 172.21.15.185:52534) 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 -- 172.21.15.185:0/1772884160 learned_addr learned my addr 172.21.15.185:0/1772884160 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe38ef0 0xeb5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87effd700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 0xe89440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 -- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe38ef0 msgr2=0xeb5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe38ef0 0xeb5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 -- 172.21.15.185:0/1772884160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 msgr2=0xe89440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 0xe89440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 -- 172.21.15.185:0/1772884160 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848008160 con 0xc6a860 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87effd700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 0xe89440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff87ffff700 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7ff878048a90 tx=0x7ff87803a000).ready entity=mon.2 client_cookie=1afa85b5fed725ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8757fa700 1 -- 172.21.15.185:0/1772884160 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878044db0 con 0xc6a860 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8757fa700 1 -- 172.21.15.185:0/1772884160 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878044f10 con 0xc6a860 2020-11-16T22:17:04.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.015+0000 7ff8757fa700 1 -- 172.21.15.185:0/1772884160 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87806d7c0 con 0xc6a860 2020-11-16T22:17:04.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 msgr2=0xe89930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7ff878048a90 tx=0x7ff87803a000).stop 2020-11-16T22:17:04.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/1772884160 shutdown_connections 2020-11-16T22:17:04.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe38ef0 0xeb5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc6a860 0xe89930 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- 172.21.15.185:0/1772884160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xbfcc60 0xe89440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/1772884160 >> 172.21.15.185:0/1772884160 conn(0x7ff848002320 msgr2=0xe8a020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/1772884160 shutdown_connections 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/1772884160 wait complete. 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- start start 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 Processor -- start 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc6a860 0xf1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe38ef0 0xf1e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xec7040 con 0xe38ef0 2020-11-16T22:17:04.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xec5100 con 0xc6a860 2020-11-16T22:17:04.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe86820 con 0xbfcc60 2020-11-16T22:17:04.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 10 client.0 ll_register_callbacks cb 0xe8b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:04.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52540/0 (socket says 172.21.15.185:52540) 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 -- 172.21.15.185:0/805773710 learned_addr learned my addr 172.21.15.185:0/805773710 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87effd700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc6a860 0xf1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87ffff700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe38ef0 0xf1e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 -- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc6a860 msgr2=0xf1e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc6a860 0xf1e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 -- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe38ef0 msgr2=0xf1e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe38ef0 0xf1e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 -- 172.21.15.185:0/805773710 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc1a7d0 con 0xbfcc60 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87f7fe700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7ff8780488b0 tx=0x7ff87000a000).ready entity=mon.2 client_cookie=d7fee0535c7b00af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870027070 con 0xbfcc60 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff878048920 con 0xbfcc60 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff870016600 con 0xbfcc60 2020-11-16T22:17:04.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff87001c040 con 0xbfcc60 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 10 client.5447 fetch_fsmap learned FSMap version 39 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0xc25eb0 con 0xbfcc60 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878048920 con 0xbfcc60 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 39) v1 ==== 1489+0+0 (secure 0 0 0) 0x7ff87002ca50 con 0xbfcc60 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 10 client.5447 fetch_fsmap finished waiting for FSMap version 39 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 10 client.5447 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:04.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 20 client.5447 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xe88e50 0xeb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 4 client.5447 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0xcae000 con 0xe88e50 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.019+0000 7ff87effd700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xe88e50 0xeb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff86dffb700 10 client.5447 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:04.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff87effd700 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xe88e50 0xeb6e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=38f350b578a2f2ae server_cookie=f6cda16b910e7ade in_seq=0 out_seq=0 2020-11-16T22:17:04.217 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:17:04.026+0000 7fbbc3cd6700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v21 ap=1 f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1) n(v4 rc2020-11-16T22:16:39.383300+0000 6=0+6)/n(v3 rc2020-11-16T22:16:39.383300+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@5},l=5297 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x564599fc8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:16:39.383300+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.217 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:17:04.034+0000 7fbbc9ce2700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v3 rc2020-11-16T22:16:39.383300+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5297=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564599fc8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:16:39.383300+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.217 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:17:04.030+0000 7fa378334700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v2 rc2020-11-16T22:16:39.383300+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@4},l=5297 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a8e907e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:16:39.383300+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.422 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:17:04.034+0000 7fa37e340700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v1 rc2020-11-16T22:16:39.383300+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5297=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a8e907ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:16:39.383300+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff86dffb700 1 -- 172.21.15.185:0/805773710 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0xcae000 con 0xe88e50 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff86dffb700 10 client.5447 handle_command_reply: tid=0 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 client.5447 shutdown 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 msgr2=0xe8a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7ff8780488b0 tx=0x7ff87000a000).stop 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 shutdown_connections 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe38ef0 0xf1e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbfcc60 0xe8a350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0xe88e50 0xeb6e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 --2- 172.21.15.185:0/805773710 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc6a860 0xf1e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 >> 172.21.15.185:0/805773710 conn(0x7ff848002320 msgr2=0xebd460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff86dffb700 0 client.5447 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 shutdown_connections 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 1 -- 172.21.15.185:0/805773710 wait complete. 2020-11-16T22:17:04.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff8949cb740 20 client.5447 trim_cache size 0 max 16384 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 msgr2=0x7ff850057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 0x7ff850057520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7ff870001520 tx=0x7ff870007e60).stop 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 msgr2=0x7ff880141440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 0x7ff880141440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7ff878012000 tx=0x7ff878015000).stop 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 shutdown_connections 2020-11-16T22:17:04.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff850055060 0x7ff850057520 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff880061e60 0x7ff8800ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff8800ae260 0x7ff880141440 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 --2- 172.21.15.185:0/1131890665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8800669a0 0x7ff880140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 >> 172.21.15.185:0/1131890665 conn(0x7ff880060130 msgr2=0x7ff880066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 shutdown_connections 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.023+0000 7ff884cd8700 1 -- 172.21.15.185:0/1131890665 wait complete. 2020-11-16T22:17:04.426 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:04.427 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0, 2020-11-16T22:17:04.427 INFO:teuthology.orchestra.run.smithi185.stdout: "scrub_tag": "d3471dec-506f-4ac4-84f7-91a754072318", 2020-11-16T22:17:04.427 INFO:teuthology.orchestra.run.smithi185.stdout: "mode": "asynchronous" 2020-11-16T22:17:04.427 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:04.427 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:17:04.027+0000 7fe58c150700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1) n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@4},l=5297 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5589c51b3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.427 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:17:04.027+0000 7fe58c150700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/AsLsXsFsx@2},l=5297 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5589c51b3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:17:04.428 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 Processor -- start 2020-11-16T22:17:04.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 -- start start 2020-11-16T22:17:04.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 0x7fdf500b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf50061fe0 con 0x7fdf500ad110 2020-11-16T22:17:04.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500b0fe0 con 0x7fdf500b1d30 2020-11-16T22:17:04.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf56151700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500b2b60 con 0x7fdf500b3090 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5514f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 0x7fdf500b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5494e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fdf500b1d30 0x7fdf5014b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52544/0 (socket says 172.21.15.185:52544) 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5494e700 1 -- 172.21.15.185:0/1070851696 learned_addr learned my addr 172.21.15.185:0/1070851696 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5514f700 1 -- 172.21.15.185:0/1070851696 >> v1:172.21.15.185:6789/0 conn(0x7fdf500b1d30 legacy=0x7fdf5014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5514f700 1 -- 172.21.15.185:0/1070851696 >> v1:172.21.15.96:6789/0 conn(0x7fdf500ad110 legacy=0x7fdf501477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5514f700 1 -- 172.21.15.185:0/1070851696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500b1280 con 0x7fdf500b3090 2020-11-16T22:17:04.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf5514f700 1 --2- 172.21.15.185:0/1070851696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 0x7fdf500b1720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fdf4000b9d0 tx=0x7fdf40006df0).ready entity=mon.2 client_cookie=7fadb33ecaf0d86c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf4ffff700 1 -- 172.21.15.185:0/1070851696 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009870 con 0x7fdf500b3090 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf4ffff700 1 -- 172.21.15.185:0/1070851696 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf400099d0 con 0x7fdf500b3090 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.575+0000 7fdf4ffff700 1 -- 172.21.15.185:0/1070851696 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009d50 con 0x7fdf500b3090 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- 172.21.15.185:0/1070851696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 msgr2=0x7fdf500b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 --2- 172.21.15.185:0/1070851696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 0x7fdf500b1720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fdf4000b9d0 tx=0x7fdf40006df0).stop 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- 172.21.15.185:0/1070851696 shutdown_connections 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 --2- 172.21.15.185:0/1070851696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b3090 0x7fdf500b1720 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- 172.21.15.185:0/1070851696 >> 172.21.15.185:0/1070851696 conn(0x7fdf50060130 msgr2=0x7fdf50060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- 172.21.15.185:0/1070851696 shutdown_connections 2020-11-16T22:17:04.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- 172.21.15.185:0/1070851696 wait complete. 2020-11-16T22:17:04.564 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 Processor -- start 2020-11-16T22:17:04.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.579+0000 7fdf56151700 1 -- start start 2020-11-16T22:17:04.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf500ad110 0x7fdf50147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 0x7fdf500affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 0x7fdf501458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50147a10 con 0x7fdf500ad110 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50157e10 con 0x7fdf500b3090 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501580b0 con 0x7fdf500b1d30 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf5494e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 0x7fdf500affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf5494e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 0x7fdf500affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52556/0 (socket says 172.21.15.185:52556) 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf5494e700 1 -- 172.21.15.185:0/2722938686 learned_addr learned my addr 172.21.15.185:0/2722938686 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 0x7fdf501458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf5514f700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf500ad110 0x7fdf50147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 -- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 msgr2=0x7fdf500affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 0x7fdf500affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 -- 172.21.15.185:0/2722938686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf500ad110 msgr2=0x7fdf50147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf500ad110 0x7fdf50147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 -- 172.21.15.185:0/2722938686 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50158350 con 0x7fdf500b3090 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf55950700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 0x7fdf501458c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdf4800c6e0 tx=0x7fdf48007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f470 con 0x7fdf500b3090 2020-11-16T22:17:04.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800f5d0 con 0x7fdf500b3090 2020-11-16T22:17:04.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf501585f0 con 0x7fdf500b3090 2020-11-16T22:17:04.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f950 con 0x7fdf500b3090 2020-11-16T22:17:04.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.583+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf50158b10 con 0x7fdf500b3090 2020-11-16T22:17:04.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50158780 con 0x7fdf500b3090 2020-11-16T22:17:04.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdf48019030 con 0x7fdf500b3090 2020-11-16T22:17:04.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf4d7fa700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 0x7fdf2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf5514f700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 0x7fdf2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf4806d090 con 0x7fdf500b3090 2020-11-16T22:17:04.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.587+0000 7fdf5514f700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 0x7fdf2c057520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fdf4000b150 tx=0x7fdf40015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.591+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdf48043290 con 0x7fdf500b3090 2020-11-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf5005d200 con 0x7fdf500b3090 2020-11-16T22:17:04.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf4d7fa700 1 -- 172.21.15.185:0/2722938686 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2522 (secure 0 0 0) 0x7fdf480468b0 con 0x7fdf500b3090 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 msgr2=0x7fdf2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 0x7fdf2c057520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fdf4000b150 tx=0x7fdf40015000).stop 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 msgr2=0x7fdf501458c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.795+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 0x7fdf501458c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdf4800c6e0 tx=0x7fdf48007aa0).stop 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 shutdown_connections 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf2c055060 0x7fdf2c057520 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf500b3090 0x7fdf501458c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf500ad110 0x7fdf50147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 --2- 172.21.15.185:0/2722938686 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf500b1d30 0x7fdf500affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 >> 172.21.15.185:0/2722938686 conn(0x7fdf50060130 msgr2=0x7fdf50147ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 shutdown_connections 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.799+0000 7fdf56151700 1 -- 172.21.15.185:0/2722938686 wait complete. 2020-11-16T22:17:04.838 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 39 2020-11-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:59.983598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:04.839 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub abort 2020-11-16T22:17:04.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 Processor -- start 2020-11-16T22:17:04.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 -- start start 2020-11-16T22:17:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdc2c141690 0x7fdc2c067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c145050 con 0x7fdc2c0b1f20 2020-11-16T22:17:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0b3f50 con 0x7fdc2c0b7690 2020-11-16T22:17:04.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc30e7d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b41f0 con 0x7fdc2c141690 2020-11-16T22:17:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdc2c141690 0x7fdc2c067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdc2c141690 0x7fdc2c067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59118/0 (socket says 172.21.15.185:59118) 2020-11-16T22:17:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 -- 172.21.15.185:0/1976167063 learned_addr learned my addr 172.21.15.185:0/1976167063 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2b7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fdc2c0b7690 0x7fdc2c144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52556/0 (socket says 172.21.15.185:52556) 2020-11-16T22:17:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 -- 172.21.15.185:0/1976167063 >> v1:172.21.15.185:6789/0 conn(0x7fdc2c0b7690 legacy=0x7fdc2c144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 -- 172.21.15.185:0/1976167063 >> v1:172.21.15.96:6789/0 conn(0x7fdc2c0b1f20 legacy=0x7fdc2c144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 -- 172.21.15.185:0/1976167063 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0b4490 con 0x7fdc2c141690 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2affd700 1 --2- 172.21.15.185:0/1976167063 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c141690 0x7fdc2c067420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdc20006fd0 tx=0x7fdc20006eb0).ready entity=mon.1 client_cookie=2f8c903dac71de59 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.991+0000 7fdc2a7fc700 1 -- 172.21.15.185:0/1976167063 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc200098d0 con 0x7fdc2c141690 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc2a7fc700 1 -- 172.21.15.185:0/1976167063 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc20009a30 con 0x7fdc2c141690 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc2a7fc700 1 -- 172.21.15.185:0/1976167063 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009db0 con 0x7fdc2c141690 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- 172.21.15.185:0/1976167063 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c141690 msgr2=0x7fdc2c067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/1976167063 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c141690 0x7fdc2c067420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdc20006fd0 tx=0x7fdc20006eb0).stop 2020-11-16T22:17:04.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- 172.21.15.185:0/1976167063 shutdown_connections 2020-11-16T22:17:04.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/1976167063 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c141690 0x7fdc2c067420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- 172.21.15.185:0/1976167063 >> 172.21.15.185:0/1976167063 conn(0x7fdc2c064b90 msgr2=0x7fdc2c063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- 172.21.15.185:0/1976167063 shutdown_connections 2020-11-16T22:17:04.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- 172.21.15.185:0/1976167063 wait complete. 2020-11-16T22:17:04.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 Processor -- start 2020-11-16T22:17:04.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.995+0000 7fdc30e7d700 1 -- start start 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 0x7fdc2c143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 0x7fdc2c142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b0590 con 0x7fdc2c141690 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c061040 con 0x7fdc2c0b1f20 2020-11-16T22:17:04.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c153310 con 0x7fdc2c0b7690 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 0x7fdc2c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52568/0 (socket says 172.21.15.185:52568) 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 0x7fdc2c143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59124/0 (socket says 172.21.15.185:59124) 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 -- 172.21.15.185:0/3802877903 learned_addr learned my addr 172.21.15.185:0/3802877903 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2bfff700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 0x7fdc2c142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 -- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 msgr2=0x7fdc2c143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 0x7fdc2c143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 -- 172.21.15.185:0/3802877903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 msgr2=0x7fdc2c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 0x7fdc2c142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 -- 172.21.15.185:0/3802877903 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c1535b0 con 0x7fdc2c0b7690 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2bfff700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 0x7fdc2c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fdc2000b1b0 tx=0x7fdc20015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc0b7fe700 1 -- 172.21.15.185:0/3802877903 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2001e460 con 0x7fdc2c0b7690 2020-11-16T22:17:04.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc0b7fe700 1 -- 172.21.15.185:0/3802877903 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2001e5c0 con 0x7fdc2c0b7690 2020-11-16T22:17:04.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c153850 con 0x7fdc2c0b7690 2020-11-16T22:17:04.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc0b7fe700 1 -- 172.21.15.185:0/3802877903 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2001e980 con 0x7fdc2c0b7690 2020-11-16T22:17:04.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:04.999+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c153f90 con 0x7fdc2c0b7690 2020-11-16T22:17:04.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.003+0000 7fdc0b7fe700 1 -- 172.21.15.185:0/3802877903 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdc20025030 con 0x7fdc2c0b7690 2020-11-16T22:17:04.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.003+0000 7fdc0b7fe700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 0x7fdc04057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.003+0000 7fdc0b7fe700 1 -- 172.21.15.185:0/3802877903 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdc2006f760 con 0x7fdc2c0b7690 2020-11-16T22:17:04.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.003+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 0x7fdc04057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.003+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 0x7fdc04057520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fdc1c007fd0 tx=0x7fdc1c007f00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 Processor -- start 2020-11-16T22:17:04.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- start start 2020-11-16T22:17:04.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 0x17ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1542ba0 0x17cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x165a4a0 con 0x1542ba0 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x170e920 con 0x17ced90 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x15ce6e0 con 0x15b07a0 2020-11-16T22:17:04.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 0x17ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59136/0 (socket says 172.21.15.185:59136) 2020-11-16T22:17:04.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2bfff700 1 -- 172.21.15.185:0/251320361 learned_addr learned my addr 172.21.15.185:0/251320361 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:04.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2affd700 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1542ba0 0x17cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:04.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 msgr2=0x17cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/251320361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1542ba0 msgr2=0x17cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1542ba0 0x17cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/251320361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15c9590 con 0x15b07a0 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 0x17ce8a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fdc1c001d40 tx=0x7fdc1c00a000).ready entity=mon.2 client_cookie=8ef805463e66b66d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc08ff9700 1 -- 172.21.15.185:0/251320361 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c017860 con 0x15b07a0 2020-11-16T22:17:04.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc08ff9700 1 -- 172.21.15.185:0/251320361 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0179c0 con 0x15b07a0 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc08ff9700 1 -- 172.21.15.185:0/251320361 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c017d40 con 0x15b07a0 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 msgr2=0x17ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 0x17ce8a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fdc1c001d40 tx=0x7fdc1c00a000).stop 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- 172.21.15.185:0/251320361 shutdown_connections 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x15b07a0 0x17ce8a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 --2- 172.21.15.185:0/251320361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1542ba0 0x17cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:04.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.007+0000 7fdc40b70740 1 -- 172.21.15.185:0/251320361 >> 172.21.15.185:0/251320361 conn(0x177ee30 msgr2=0x17cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:04.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- 172.21.15.185:0/251320361 shutdown_connections 2020-11-16T22:17:04.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- 172.21.15.185:0/251320361 wait complete. 2020-11-16T22:17:04.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- start start 2020-11-16T22:17:04.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 Processor -- start 2020-11-16T22:17:04.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x17fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:04.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16fb6d0 con 0x15b07a0 2020-11-16T22:17:05.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x16fb810 con 0x17ced90 2020-11-16T22:17:05.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x16fca80 con 0x1542ba0 2020-11-16T22:17:05.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 10 client.0 ll_register_callbacks cb 0x16f1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:05.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52582/0 (socket says 172.21.15.185:52582) 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/3274285241 learned_addr learned my addr 172.21.15.185:0/3274285241 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x17fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 msgr2=0x17fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 msgr2=0x17fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x17fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/3274285241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16ec270 con 0x1542ba0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdc1c017e90 tx=0x7fdc1c015000).ready entity=mon.2 client_cookie=5f12cada993e1c6d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c027070 con 0x1542ba0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17af6b0 con 0x1542ba0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdc1c0149f0 con 0x1542ba0 2020-11-16T22:17:05.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1c014dc0 con 0x1542ba0 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 10 client.5462 fetch_fsmap learned FSMap version 39 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x1521fc0 con 0x1542ba0 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17af6b0 con 0x1542ba0 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.011+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 39) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fdc1c01e020 con 0x1542ba0 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 10 client.5462 fetch_fsmap finished waiting for FSMap version 39 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 10 client.5462 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:05.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 20 client.5462 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17cf680 0x17cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 4 client.5462 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x15ecc40 con 0x17cf680 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17cf680 0x17cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc2affd700 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17cf680 0x17cfc70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8f732bb166a8d47f server_cookie=5f09251fa6a58355 in_seq=0 out_seq=0 2020-11-16T22:17:05.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc01ffb700 10 client.5462 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:05.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc01ffb700 1 -- 172.21.15.185:0/3274285241 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x15ecc40 con 0x17cf680 2020-11-16T22:17:05.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc01ffb700 10 client.5462 handle_command_reply: tid=0 2020-11-16T22:17:05.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 client.5462 shutdown 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 msgr2=0x17d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fdc1c017e90 tx=0x7fdc1c015000).stop 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 shutdown_connections 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17cf680 0x17cfc70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17ced90 0x17fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x17d2320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 --2- 172.21.15.185:0/3274285241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x17fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 >> 172.21.15.185:0/3274285241 conn(0x177ee30 msgr2=0x17cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc01ffb700 0 client.5462 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 shutdown_connections 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 1 -- 172.21.15.185:0/3274285241 wait complete. 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.015+0000 7fdc40b70740 20 client.5462 trim_cache size 0 max 16384 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 Processor -- start 2020-11-16T22:17:05.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- start start 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x177ee30 0x1815e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x1811ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d3e30 con 0x15b07a0 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x17c9990 con 0x177ee30 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x15ecc40 con 0x1542ba0 2020-11-16T22:17:05.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x177ee30 0x1815e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52590/0 (socket says 172.21.15.185:52590) 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 -- 172.21.15.185:0/4092615309 learned_addr learned my addr 172.21.15.185:0/4092615309 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2bfff700 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x1811ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 -- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x177ee30 msgr2=0x1815e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x177ee30 0x1815e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 -- 172.21.15.185:0/4092615309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 msgr2=0x1811ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x1811ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 -- 172.21.15.185:0/4092615309 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8008160 con 0x1542ba0 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2bfff700 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x1811ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc2affd700 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fdc20021040 tx=0x7fdc20011000).ready entity=mon.2 client_cookie=d6d927873a03308a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc097fa700 1 -- 172.21.15.185:0/4092615309 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2001edd0 con 0x1542ba0 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc097fa700 1 -- 172.21.15.185:0/4092615309 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc200478f0 con 0x1542ba0 2020-11-16T22:17:05.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc097fa700 1 -- 172.21.15.185:0/4092615309 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20048ca0 con 0x1542ba0 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 msgr2=0x18119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fdc20021040 tx=0x7fdc20011000).stop 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- 172.21.15.185:0/4092615309 shutdown_connections 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x177ee30 0x1815e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x1811ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 --2- 172.21.15.185:0/4092615309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1542ba0 0x18119f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- 172.21.15.185:0/4092615309 >> 172.21.15.185:0/4092615309 conn(0x7fdbf8002320 msgr2=0x17caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- 172.21.15.185:0/4092615309 shutdown_connections 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.035+0000 7fdc40b70740 1 -- 172.21.15.185:0/4092615309 wait complete. 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 -- start start 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 Processor -- start 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x185eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x177ee30 0x18686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1801e30 con 0x15b07a0 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x18020d0 con 0x1542ba0 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1819af0 con 0x177ee30 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59152/0 (socket says 172.21.15.185:59152) 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x177ee30 0x18686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/2383721039 learned_addr learned my addr 172.21.15.185:0/2383721039 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 10 client.0 ll_register_callbacks cb 0x18047e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2affd700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x185eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x177ee30 msgr2=0x18686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x177ee30 0x18686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 msgr2=0x185eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x185eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 -- 172.21.15.185:0/2383721039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1560710 con 0x1542ba0 2020-11-16T22:17:05.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc2b7fe700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fdc1c020400 tx=0x7fdc1c027000).ready entity=mon.1 client_cookie=fc0fa56b5e2fa843 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0176e0 con 0x1542ba0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdc1c0149f0 con 0x1542ba0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc1c0179c0 con 0x1542ba0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1c014610 con 0x1542ba0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.039+0000 7fdc40b70740 10 client.5908 fetch_fsmap learned FSMap version 39 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x156bdf0 con 0x1542ba0 2020-11-16T22:17:05.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0179c0 con 0x1542ba0 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 39) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fdc1c01e040 con 0x1542ba0 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 10 client.5908 fetch_fsmap finished waiting for FSMap version 39 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 10 client.5908 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 20 client.5908 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17ced90 0x1868bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 4 client.5908 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,a,b,o,r,t,",}] 2020-11-16T22:17:05.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " s c r u b a b o r t " }) v1 -- 0x15f3f40 con 0x17ced90 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc2affd700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17ced90 0x1868bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 10 client.5908 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc2affd700 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17ced90 0x1868bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ffaa8ccaad5c6c11 server_cookie=6d08d14307a4c6c in_seq=0 out_seq=0 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 1 -- 172.21.15.185:0/2383721039 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x15f3f40 con 0x17ced90 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 10 client.5908 handle_command_reply: tid=0 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 client.5908 shutdown 2020-11-16T22:17:05.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 msgr2=0x185eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fdc1c020400 tx=0x7fdc1c027000).stop 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 shutdown_connections 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x17ced90 0x1868bb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1542ba0 0x185eb00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x177ee30 0x18686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 --2- 172.21.15.185:0/2383721039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x15b07a0 0x185eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 >> 172.21.15.185:0/2383721039 conn(0x7fdbf8002320 msgr2=0x17c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc01ffb700 0 client.5908 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 shutdown_connections 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 1 -- 172.21.15.185:0/2383721039 wait complete. 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc40b70740 20 client.5908 trim_cache size 0 max 16384 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 msgr2=0x7fdc04057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 0x7fdc04057520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fdc1c007fd0 tx=0x7fdc1c007f00).stop 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 msgr2=0x7fdc2c0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fdc2000b1b0 tx=0x7fdc20015000).stop 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 shutdown_connections 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc04055060 0x7fdc04057520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdc2c0b1f20 0x7fdc2c143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c141690 0x7fdc2c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 --2- 172.21.15.185:0/3802877903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdc2c0b7690 0x7fdc2c0b0f40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 >> 172.21.15.185:0/3802877903 conn(0x7fdc2c064b90 msgr2=0x7fdc2c0b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 shutdown_connections 2020-11-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.043+0000 7fdc30e7d700 1 -- 172.21.15.185:0/3802877903 wait complete. 2020-11-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:05.437 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:05.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 Processor -- start 2020-11-16T22:17:05.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 -- start start 2020-11-16T22:17:05.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0d18067070 0x7f0d180b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d180682b0 0x7f0d18143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d18067510 con 0x7f0d180b7470 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180677b0 con 0x7f0d180682b0 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1f138700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180b17a0 con 0x7f0d18067070 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1d935700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0d18067070 0x7f0d180b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1d935700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0d18067070 0x7f0d180b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59160/0 (socket says 172.21.15.185:59160) 2020-11-16T22:17:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1d935700 1 -- 172.21.15.185:0/1463319272 learned_addr learned my addr 172.21.15.185:0/1463319272 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1e136700 1 --1- 172.21.15.185:0/1463319272 >> v1:172.21.15.185:6789/0 conn(0x7f0d180b7470 0x7f0d180b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52598/0 (socket says 172.21.15.185:52598) 2020-11-16T22:17:05.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.787+0000 7f0d1e937700 1 --2- 172.21.15.185:0/1463319272 >> v2:172.21.15.96:3300/0 conn(0x7f0d180682b0 0x7f0d18143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 -- 172.21.15.185:0/1463319272 >> v2:172.21.15.96:3300/0 conn(0x7f0d180682b0 msgr2=0x7f0d18143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 --2- 172.21.15.185:0/1463319272 >> v2:172.21.15.96:3300/0 conn(0x7f0d180682b0 0x7f0d18143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 -- 172.21.15.185:0/1463319272 >> v1:172.21.15.185:6789/0 conn(0x7f0d180b7470 legacy=0x7f0d180b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 -- 172.21.15.185:0/1463319272 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180b1a40 con 0x7f0d18067070 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d134700 1 -- 172.21.15.185:0/1463319272 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681411143 0 0) 0x7f0d18067510 con 0x7f0d180b7470 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 --2- 172.21.15.185:0/1463319272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d180b7850 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0d180b32d0 tx=0x7f0d14009290).ready entity=mon.1 client_cookie=dddc658de9b0791c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d134700 1 -- 172.21.15.185:0/1463319272 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400bbc0 con 0x7f0d18067070 2020-11-16T22:17:05.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d134700 1 -- 172.21.15.185:0/1463319272 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1400bd20 con 0x7f0d18067070 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d134700 1 -- 172.21.15.185:0/1463319272 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14010620 con 0x7f0d18067070 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- 172.21.15.185:0/1463319272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 msgr2=0x7f0d180b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- 172.21.15.185:0/1463319272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d180b7850 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0d180b32d0 tx=0x7f0d14009290).stop 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- 172.21.15.185:0/1463319272 shutdown_connections 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- 172.21.15.185:0/1463319272 >> v2:172.21.15.96:3300/0 conn(0x7f0d180682b0 0x7f0d18143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- 172.21.15.185:0/1463319272 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d180b7850 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- 172.21.15.185:0/1463319272 >> 172.21.15.185:0/1463319272 conn(0x7f0d180637b0 msgr2=0x7f0d180b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- 172.21.15.185:0/1463319272 shutdown_connections 2020-11-16T22:17:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- 172.21.15.185:0/1463319272 wait complete. 2020-11-16T22:17:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 Processor -- start 2020-11-16T22:17:05.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- start start 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d181571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 0x7f0d1813cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d180b7470 0x7f0d180b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18111d10 con 0x7f0d180b7470 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181567c0 con 0x7f0d18067070 2020-11-16T22:17:05.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1f138700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18156a60 con 0x7f0d180682b0 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1e136700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d181571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1d935700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 0x7f0d1813cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1e136700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d181571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59166/0 (socket says 172.21.15.185:59166) 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1e136700 1 -- 172.21.15.185:0/2514472017 learned_addr learned my addr 172.21.15.185:0/2514472017 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.791+0000 7f0d1e937700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d180b7470 0x7f0d180b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 -- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 msgr2=0x7f0d181571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d181571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 -- 172.21.15.185:0/2514472017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d180b7470 msgr2=0x7f0d180b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d180b7470 0x7f0d180b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 -- 172.21.15.185:0/2514472017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18156d00 con 0x7f0d180682b0 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1d935700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 0x7f0d1813cde0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0d180b32d0 tx=0x7f0d14017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14019680 con 0x7f0d180682b0 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1401e070 con 0x7f0d180682b0 2020-11-16T22:17:05.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140197e0 con 0x7f0d180682b0 2020-11-16T22:17:05.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d180b6ee0 con 0x7f0d180682b0 2020-11-16T22:17:05.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1813c470 con 0x7f0d180682b0 2020-11-16T22:17:05.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18156e90 con 0x7f0d180682b0 2020-11-16T22:17:05.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0d1400e040 con 0x7f0d180682b0 2020-11-16T22:17:05.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 0x7f0cf0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:05.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d1406fa50 con 0x7f0d180682b0 2020-11-16T22:17:05.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.795+0000 7f0d1e136700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 0x7f0cf0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:05.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.799+0000 7f0d1e136700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 0x7f0cf0057520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f0d08007180 tx=0x7f0d08007a30).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:05.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.799+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0d14045d30 con 0x7f0d180682b0 2020-11-16T22:17:05.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d1805d200 con 0x7f0d180682b0 2020-11-16T22:17:05.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/2514472017 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f0d14028070 con 0x7f0d180682b0 2020-11-16T22:17:05.983 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:05.983 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:16:59.983598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":8,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":8,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:05.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 msgr2=0x7f0cf0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 0x7f0cf0057520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f0d08007180 tx=0x7f0d08007a30).stop 2020-11-16T22:17:05.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 msgr2=0x7f0d1813cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:05.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 0x7f0d1813cde0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f0d180b32d0 tx=0x7f0d14017000).stop 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 shutdown_connections 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cf0055060 0x7f0cf0057520 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d18067070 0x7f0d181571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d180b7470 0x7f0d180b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 --2- 172.21.15.185:0/2514472017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d180682b0 0x7f0d1813cde0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:05.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 >> 172.21.15.185:0/2514472017 conn(0x7f0d180637b0 msgr2=0x7f0d18138cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:05.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 shutdown_connections 2020-11-16T22:17:05.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:05.999+0000 7f0d1f138700 1 -- 172.21.15.185:0/2514472017 wait complete. 2020-11-16T22:17:05.986 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 39 2020-11-16T22:17:05.997 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:17:06.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.131+0000 7fb523fff700 1 Processor -- start 2020-11-16T22:17:06.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.131+0000 7fb523fff700 1 -- start start 2020-11-16T22:17:06.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 0x7fb524145350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5240b47b0 con 0x7fb5240b3ec0 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5240b4a50 con 0x7fb5240af7f0 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240b4cf0 con 0x7fb524063fd0 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52618/0 (socket says 172.21.15.185:52618) 2020-11-16T22:17:06.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 -- 172.21.15.185:0/1501403009 learned_addr learned my addr 172.21.15.185:0/1501403009 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb5227fc700 1 --2- 172.21.15.185:0/1501403009 >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 0x7fb524145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 -- 172.21.15.185:0/1501403009 >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 msgr2=0x7fb524145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 --2- 172.21.15.185:0/1501403009 >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 0x7fb524145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 -- 172.21.15.185:0/1501403009 >> v1:172.21.15.96:6789/0 conn(0x7fb5240b3ec0 legacy=0x7fb524145840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 -- 172.21.15.185:0/1501403009 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524142260 con 0x7fb524063fd0 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb5227fc700 1 --2- 172.21.15.185:0/1501403009 >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 0x7fb524145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb522ffd700 1 --2- 172.21.15.185:0/1501403009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb514006fd0 tx=0x7fb514006eb0).ready entity=mon.2 client_cookie=50e57d9e519fc4e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb521ffb700 1 -- 172.21.15.185:0/1501403009 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140098d0 con 0x7fb524063fd0 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb521ffb700 1 -- 172.21.15.185:0/1501403009 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514009a30 con 0x7fb524063fd0 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb521ffb700 1 -- 172.21.15.185:0/1501403009 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514009db0 con 0x7fb524063fd0 2020-11-16T22:17:06.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- 172.21.15.185:0/1501403009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 msgr2=0x7fb5240b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 --2- 172.21.15.185:0/1501403009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fb514006fd0 tx=0x7fb514006eb0).stop 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- 172.21.15.185:0/1501403009 shutdown_connections 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 --2- 172.21.15.185:0/1501403009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb524063fd0 0x7fb5240b1270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 --2- 172.21.15.185:0/1501403009 >> v2:172.21.15.96:3300/0 conn(0x7fb5240af7f0 0x7fb524145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- 172.21.15.185:0/1501403009 >> 172.21.15.185:0/1501403009 conn(0x7fb524067940 msgr2=0x7fb524067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- 172.21.15.185:0/1501403009 shutdown_connections 2020-11-16T22:17:06.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- 172.21.15.185:0/1501403009 wait complete. 2020-11-16T22:17:06.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 Processor -- start 2020-11-16T22:17:06.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.135+0000 7fb523fff700 1 -- start start 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 0x7fb5240b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 0x7fb52413ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 0x7fb52413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240adbf0 con 0x7fb5240b3ec0 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240ad350 con 0x7fb524063fd0 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5241554d0 con 0x7fb5240af7f0 2020-11-16T22:17:06.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 0x7fb5240b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 0x7fb52413ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 0x7fb52413ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52626/0 (socket says 172.21.15.185:52626) 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 0x7fb5240b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59180/0 (socket says 172.21.15.185:59180) 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5227fc700 1 -- 172.21.15.185:0/152042099 learned_addr learned my addr 172.21.15.185:0/152042099 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 0x7fb52413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 -- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 msgr2=0x7fb52413ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 0x7fb52413ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 -- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 msgr2=0x7fb5240b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 0x7fb5240b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 -- 172.21.15.185:0/152042099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524155770 con 0x7fb5240b3ec0 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb5237fe700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 0x7fb52413b050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fb51c00ca60 tx=0x7fb51c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb51affd700 1 -- 172.21.15.185:0/152042099 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00ce30 con 0x7fb5240b3ec0 2020-11-16T22:17:06.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb51affd700 1 -- 172.21.15.185:0/152042099 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c00a730 con 0x7fb5240b3ec0 2020-11-16T22:17:06.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb524151410 con 0x7fb5240b3ec0 2020-11-16T22:17:06.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb51affd700 1 -- 172.21.15.185:0/152042099 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00aab0 con 0x7fb5240b3ec0 2020-11-16T22:17:06.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.139+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5240b3480 con 0x7fb5240b3ec0 2020-11-16T22:17:06.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.143+0000 7fb51affd700 1 -- 172.21.15.185:0/152042099 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb51c019030 con 0x7fb5240b3ec0 2020-11-16T22:17:06.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.143+0000 7fb51affd700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 0x7fb500057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.143+0000 7fb522ffd700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 0x7fb500057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.143+0000 7fb51affd700 1 -- 172.21.15.185:0/152042099 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb51c06d500 con 0x7fb5240b3ec0 2020-11-16T22:17:06.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.143+0000 7fb522ffd700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 0x7fb500057520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fb514006ef0 tx=0x7fb514015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 Processor -- start 2020-11-16T22:17:06.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- start start 2020-11-16T22:17:06.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1d5e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d5eda0 0x1d5d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bea4b0 con 0x1d5eda0 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c9e930 con 0x1b407b0 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b5e6f0 con 0x1ad2bb0 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52632/0 (socket says 172.21.15.185:52632) 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1d5e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 -- 172.21.15.185:0/2631505652 learned_addr learned my addr 172.21.15.185:0/2631505652 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5237fe700 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d5eda0 0x1d5d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 -- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 msgr2=0x1d5e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1d5e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 -- 172.21.15.185:0/2631505652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d5eda0 msgr2=0x1d5d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d5eda0 0x1d5d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 -- 172.21.15.185:0/2631505652 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b595a0 con 0x1ad2bb0 2020-11-16T22:17:06.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5227fc700 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb50c00bdf0 tx=0x7fb50c008040).ready entity=mon.2 client_cookie=af208976c5c2623a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb4fbfff700 1 -- 172.21.15.185:0/2631505652 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00b4e0 con 0x1ad2bb0 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb4fbfff700 1 -- 172.21.15.185:0/2631505652 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c010d10 con 0x1ad2bb0 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb4fbfff700 1 -- 172.21.15.185:0/2631505652 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c014680 con 0x1ad2bb0 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 msgr2=0x1d5ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb50c00bdf0 tx=0x7fb50c008040).stop 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- 172.21.15.185:0/2631505652 shutdown_connections 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1d5e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d5eda0 0x1d5d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2631505652 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d5ce90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- 172.21.15.185:0/2631505652 >> 172.21.15.185:0/2631505652 conn(0x1d0ee40 msgr2=0x1d5bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- 172.21.15.185:0/2631505652 shutdown_connections 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.147+0000 7fb5381b4740 1 -- 172.21.15.185:0/2631505652 wait complete. 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- start start 2020-11-16T22:17:06.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 Processor -- start 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ad2bb0 0x1d5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c8b6e0 con 0x1ad2bb0 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c8b820 con 0x1b407b0 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c8ca90 con 0x1d5eda0 2020-11-16T22:17:06.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 10 client.0 ll_register_callbacks cb 0x1c81ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59194/0 (socket says 172.21.15.185:59194) 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52642/0 (socket says 172.21.15.185:52642) 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 -- 172.21.15.185:0/3802538609 learned_addr learned my addr 172.21.15.185:0/3802538609 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb522ffd700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ad2bb0 0x1d5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 -- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 msgr2=0x1da6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 -- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ad2bb0 msgr2=0x1d5b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ad2bb0 0x1d5b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 -- 172.21.15.185:0/3802538609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c7c280 con 0x1b407b0 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5237fe700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5227fc700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fb50c00bdc0 tx=0x7fb50c008040).ready entity=mon.1 client_cookie=19b8a505ebd221c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:06.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c014c80 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb50c014de0 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d3f6c0 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb50c01d680 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 10 client.5926 fetch_fsmap learned FSMap version 39 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x1ab1fd0 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d3f6c0 con 0x1b407b0 2020-11-16T22:17:06.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 39) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb50c031020 con 0x1b407b0 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 10 client.5926 fetch_fsmap finished waiting for FSMap version 39 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 10 client.5926 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 20 client.5926 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d942f0 0x1d8d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 4 client.5926 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1b7cc50 con 0x1d942f0 2020-11-16T22:17:06.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.151+0000 7fb522ffd700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d942f0 0x1d8d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb4f97fa700 10 client.5926 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb522ffd700 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d942f0 0x1d8d110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=8383a53c619dc9c9 server_cookie=4145d41b99baac00 in_seq=0 out_seq=0 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb4f97fa700 1 -- 172.21.15.185:0/3802538609 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1b7cc50 con 0x1d942f0 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb4f97fa700 10 client.5926 handle_command_reply: tid=0 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 client.5926 shutdown 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 msgr2=0x1da26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fb50c00bdc0 tx=0x7fb50c008040).stop 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 shutdown_connections 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d942f0 0x1d8d110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d5eda0 0x1da6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb4f97fa700 0 client.5926 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ad2bb0 0x1d5b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 --2- 172.21.15.185:0/3802538609 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b407b0 0x1da26b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 >> 172.21.15.185:0/3802538609 conn(0x1d0ee40 msgr2=0x1d5ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 shutdown_connections 2020-11-16T22:17:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 1 -- 172.21.15.185:0/3802538609 wait complete. 2020-11-16T22:17:06.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.155+0000 7fb5381b4740 20 client.5926 trim_cache size 0 max 16384 2020-11-16T22:17:06.148 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:17:06.170+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:17:06.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 Processor -- start 2020-11-16T22:17:06.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- start start 2020-11-16T22:17:06.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d90c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1d5a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c63e40 con 0x1b407b0 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d3f6c0 con 0x1d0ee40 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da3850 con 0x1ad2bb0 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d90c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59202/0 (socket says 172.21.15.185:59202) 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb522ffd700 1 -- 172.21.15.185:0/1552229797 learned_addr learned my addr 172.21.15.185:0/1552229797 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5237fe700 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1d5a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 -- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 msgr2=0x1d61070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 -- 172.21.15.185:0/1552229797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 msgr2=0x1d5a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1d5a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 -- 172.21.15.185:0/1552229797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0008160 con 0x1ad2bb0 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb522ffd700 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5227fc700 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d90c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb50c00bdc0 tx=0x7fb50c021040).ready entity=mon.2 client_cookie=ee913ac447115126 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb518ff9700 1 -- 172.21.15.185:0/1552229797 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c014e90 con 0x1ad2bb0 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb518ff9700 1 -- 172.21.15.185:0/1552229797 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c01eb70 con 0x1ad2bb0 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb518ff9700 1 -- 172.21.15.185:0/1552229797 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c01e450 con 0x1ad2bb0 2020-11-16T22:17:06.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 msgr2=0x1d90c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d90c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb50c00bdc0 tx=0x7fb50c021040).stop 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- 172.21.15.185:0/1552229797 shutdown_connections 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d61070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1d5a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 --2- 172.21.15.185:0/1552229797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1d90c70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- 172.21.15.185:0/1552229797 >> 172.21.15.185:0/1552229797 conn(0x7fb4f0002320 msgr2=0x1d933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- 172.21.15.185:0/1552229797 shutdown_connections 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.179+0000 7fb5381b4740 1 -- 172.21.15.185:0/1552229797 wait complete. 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- start start 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 Processor -- start 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1df84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d5b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1daa2a0 con 0x1b407b0 2020-11-16T22:17:06.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da9b10 con 0x1d0ee40 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d5e800 con 0x1ad2bb0 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 10 client.0 ll_register_callbacks cb 0x1d58a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d5b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52652/0 (socket says 172.21.15.185:52652) 2020-11-16T22:17:06.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 -- 172.21.15.185:0/2801266817 learned_addr learned my addr 172.21.15.185:0/2801266817 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5227fc700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1df84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 -- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 msgr2=0x1df8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 -- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 msgr2=0x1df84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1df84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 -- 172.21.15.185:0/2801266817 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1af0720 con 0x1d0ee40 2020-11-16T22:17:06.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5237fe700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d5b920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fb51c001390 tx=0x7fb51c06d040).ready entity=mon.1 client_cookie=c21a4a518fa487a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0434f0 con 0x1d0ee40 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb50c010d10 con 0x1d0ee40 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb51c046380 con 0x1d0ee40 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb51c046750 con 0x1d0ee40 2020-11-16T22:17:06.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 10 client.5932 fetch_fsmap learned FSMap version 40 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x1afbe00 con 0x1d0ee40 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c010d10 con 0x1d0ee40 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 40) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb51c024040 con 0x1d0ee40 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 10 client.5932 fetch_fsmap finished waiting for FSMap version 40 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 10 client.5932 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:06.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 20 client.5932 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d5eda0 0x1d5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 4 client.5932 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1b83f50 con 0x1d5eda0 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d5eda0 0x1d5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb4f97fa700 10 client.5932 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.183+0000 7fb522ffd700 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d5eda0 0x1d5be10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=a067b79bf0f2572c server_cookie=2282c079624a34ac in_seq=0 out_seq=0 2020-11-16T22:17:06.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb4f97fa700 1 -- 172.21.15.185:0/2801266817 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1b83f50 con 0x1d5eda0 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb4f97fa700 10 client.5932 handle_command_reply: tid=0 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 client.5932 shutdown 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 msgr2=0x1d5b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d5b920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fb51c001390 tx=0x7fb51c06d040).stop 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 shutdown_connections 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1d5eda0 0x1d5be10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d0ee40 0x1d5b920 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ad2bb0 0x1df8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 --2- 172.21.15.185:0/2801266817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b407b0 0x1df84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 >> 172.21.15.185:0/2801266817 conn(0x7fb4f0002320 msgr2=0x1d8e310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb4f97fa700 0 client.5932 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 shutdown_connections 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 1 -- 172.21.15.185:0/2801266817 wait complete. 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb5381b4740 20 client.5932 trim_cache size 0 max 16384 2020-11-16T22:17:06.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 msgr2=0x7fb500057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 0x7fb500057520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fb514006ef0 tx=0x7fb514015000).stop 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 msgr2=0x7fb52413b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 0x7fb52413b050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fb51c00ca60 tx=0x7fb51c007e20).stop 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 shutdown_connections 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb500055060 0x7fb500057520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb524063fd0 0x7fb5240b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5240b3ec0 0x7fb52413b050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 --2- 172.21.15.185:0/152042099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5240af7f0 0x7fb52413ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:06.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 >> 172.21.15.185:0/152042099 conn(0x7fb524067940 msgr2=0x7fb5240b6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:06.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 shutdown_connections 2020-11-16T22:17:06.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:06.187+0000 7fb523fff700 1 -- 172.21.15.185:0/152042099 wait complete. 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout: "d3471dec-506f-4ac4-84f7-91a754072318": { 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:17:06.571 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "d3471dec-506f-4ac4-84f7-91a754072318", 2020-11-16T22:17:06.572 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:17:06.572 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:06.572 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:06.572 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:06.572 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:17:11.573 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:11.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 Processor -- start 2020-11-16T22:17:11.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 -- start start 2020-11-16T22:17:11.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f39d8067070 0x7f39d8143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f39d80682b0 0x7f39d80b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d8067510 con 0x7f39d8067070 2020-11-16T22:17:11.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80677b0 con 0x7f39d80682b0 2020-11-16T22:17:11.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.723+0000 7f39de8a4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80b17a0 con 0x7f39d80b7470 2020-11-16T22:17:11.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39dd8a2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f39d80682b0 0x7f39d80b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f39d80682b0 0x7f39d80b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59220/0 (socket says 172.21.15.185:59220) 2020-11-16T22:17:11.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 -- 172.21.15.185:0/2813788431 learned_addr learned my addr 172.21.15.185:0/2813788431 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:11.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39dd0a1700 1 --2- 172.21.15.185:0/2813788431 >> v2:172.21.15.96:3300/0 conn(0x7f39d8067070 0x7f39d8143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 -- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 msgr2=0x7f39d80b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 --2- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d80b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 -- 172.21.15.185:0/2813788431 >> v2:172.21.15.96:3300/0 conn(0x7f39d8067070 msgr2=0x7f39d8143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 --2- 172.21.15.185:0/2813788431 >> v2:172.21.15.96:3300/0 conn(0x7f39d8067070 0x7f39d8143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 -- 172.21.15.185:0/2813788431 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80b1a40 con 0x7f39d80682b0 2020-11-16T22:17:11.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de0a3700 1 --2- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d80682b0 0x7f39d80b1ee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f39d8065a30 tx=0x7f39d0006f40).ready entity=mon.1 client_cookie=326e1450e8380971 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:11.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39dc8a0700 1 -- 172.21.15.185:0/2813788431 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000cad0 con 0x7f39d80682b0 2020-11-16T22:17:11.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39dc8a0700 1 -- 172.21.15.185:0/2813788431 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d000cc30 con 0x7f39d80682b0 2020-11-16T22:17:11.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39dc8a0700 1 -- 172.21.15.185:0/2813788431 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0010410 con 0x7f39d80682b0 2020-11-16T22:17:11.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 -- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d80682b0 msgr2=0x7f39d80b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:11.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 --2- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d80682b0 0x7f39d80b1ee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f39d8065a30 tx=0x7f39d0006f40).stop 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 -- 172.21.15.185:0/2813788431 shutdown_connections 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 --2- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d80682b0 0x7f39d80b1ee0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 --2- 172.21.15.185:0/2813788431 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d80b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 --2- 172.21.15.185:0/2813788431 >> v2:172.21.15.96:3300/0 conn(0x7f39d8067070 0x7f39d8143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 -- 172.21.15.185:0/2813788431 >> 172.21.15.185:0/2813788431 conn(0x7f39d80637b0 msgr2=0x7f39d80b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 -- 172.21.15.185:0/2813788431 shutdown_connections 2020-11-16T22:17:11.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.727+0000 7f39de8a4700 1 -- 172.21.15.185:0/2813788431 wait complete. 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 Processor -- start 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 -- start start 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 0x7f39d80afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39d80682b0 0x7f39d8139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80b63a0 con 0x7f39d80682b0 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d814f3c0 con 0x7f39d8067070 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de8a4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d814f660 con 0x7f39d80b7470 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39dd8a2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 0x7f39d80afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52670/0 (socket says 172.21.15.185:52670) 2020-11-16T22:17:11.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39dd8a2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 0x7f39d80afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59222/0 (socket says 172.21.15.185:59222) 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 -- 172.21.15.185:0/3040702649 learned_addr learned my addr 172.21.15.185:0/3040702649 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39dd0a1700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39d80682b0 0x7f39d8139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 -- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 msgr2=0x7f39d80afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 0x7f39d80afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 -- 172.21.15.185:0/3040702649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39d80682b0 msgr2=0x7f39d8139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39d80682b0 0x7f39d8139680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.731+0000 7f39de0a3700 1 -- 172.21.15.185:0/3040702649 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d814f900 con 0x7f39d80b7470 2020-11-16T22:17:11.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39de0a3700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f39d000a330 tx=0x7f39d80b1bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d001f070 con 0x7f39d80b7470 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0010410 con 0x7f39d80b7470 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00198b0 con 0x7f39d80b7470 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d814fba0 con 0x7f39d80b7470 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d80b0340 con 0x7f39d80b7470 2020-11-16T22:17:11.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f39d000e040 con 0x7f39d80b7470 2020-11-16T22:17:11.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d80b22f0 con 0x7f39d80b7470 2020-11-16T22:17:11.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 0x7f39b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:11.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39dd8a2700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 0x7f39b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:11.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.735+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f39d00706b0 con 0x7f39d80b7470 2020-11-16T22:17:11.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.739+0000 7f39dd8a2700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 0x7f39b0057520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f39c8006fd0 tx=0x7f39c8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:11.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.743+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f39d0046800 con 0x7f39d80b7470 2020-11-16T22:17:11.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.939+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39d814fd30 con 0x7f39d80b7470 2020-11-16T22:17:11.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.939+0000 7f39cdffb700 1 -- 172.21.15.185:0/3040702649 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f39d0029090 con 0x7f39d80b7470 2020-11-16T22:17:11.924 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:11.924 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:11.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 msgr2=0x7f39b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:11.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 0x7f39b0057520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f39c8006fd0 tx=0x7f39c8006df0).stop 2020-11-16T22:17:11.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 msgr2=0x7f39d8139b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:11.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f39d000a330 tx=0x7f39d80b1bd0).stop 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 shutdown_connections 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39b0055060 0x7f39b0057520 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39d8067070 0x7f39d80afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39d80b7470 0x7f39d8139b70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 --2- 172.21.15.185:0/3040702649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39d80682b0 0x7f39d8139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:11.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 >> 172.21.15.185:0/3040702649 conn(0x7f39d80637b0 msgr2=0x7f39d80b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:11.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 shutdown_connections 2020-11-16T22:17:11.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:11.943+0000 7f39de8a4700 1 -- 172.21.15.185:0/3040702649 wait complete. 2020-11-16T22:17:11.928 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:11.938 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:17:11.950 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:17:11.970+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:17:12.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 Processor -- start 2020-11-16T22:17:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 -- start start 2020-11-16T22:17:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f000c0b2e80 0x7f000c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f000c0b3920 0x7f000c143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c13dcf0 0x7f000c0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0ac070 con 0x7f000c0b2e80 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0ac310 con 0x7f000c0b3920 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f00109fc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0ac5b0 con 0x7f000c13dcf0 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f000c0b3920 0x7f000c143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f000c0b3920 0x7f000c143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59230/0 (socket says 172.21.15.185:59230) 2020-11-16T22:17:12.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 -- 172.21.15.185:0/4141588659 learned_addr learned my addr 172.21.15.185:0/4141588659 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c13dcf0 0x7f000c0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000ad9d700 1 --2- 172.21.15.185:0/4141588659 >> v2:172.21.15.96:3300/0 conn(0x7f000c0b2e80 0x7f000c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 -- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c13dcf0 msgr2=0x7f000c0b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 --2- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c13dcf0 0x7f000c0b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 -- 172.21.15.185:0/4141588659 >> v2:172.21.15.96:3300/0 conn(0x7f000c0b2e80 msgr2=0x7f000c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 --2- 172.21.15.185:0/4141588659 >> v2:172.21.15.96:3300/0 conn(0x7f000c0b2e80 0x7f000c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 -- 172.21.15.185:0/4141588659 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0ac850 con 0x7f000c0b3920 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f000a59c700 1 --2- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c0b3920 0x7f000c143ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f000c0b38c0 tx=0x7efff4006eb0).ready entity=mon.1 client_cookie=3984003f2c4d27f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f0009d9b700 1 -- 172.21.15.185:0/4141588659 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4009810 con 0x7f000c0b3920 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f0009d9b700 1 -- 172.21.15.185:0/4141588659 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4009970 con 0x7f000c0b3920 2020-11-16T22:17:12.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.079+0000 7f0009d9b700 1 -- 172.21.15.185:0/4141588659 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4009cf0 con 0x7f000c0b3920 2020-11-16T22:17:12.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c0b3920 msgr2=0x7f000c143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c0b3920 0x7f000c143ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f000c0b38c0 tx=0x7efff4006eb0).stop 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- 172.21.15.185:0/4141588659 shutdown_connections 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c13dcf0 0x7f000c0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- 172.21.15.185:0/4141588659 >> v2:172.21.15.96:3300/0 conn(0x7f000c0b2e80 0x7f000c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- 172.21.15.185:0/4141588659 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c0b3920 0x7f000c143ef0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- 172.21.15.185:0/4141588659 >> 172.21.15.185:0/4141588659 conn(0x7f000c060130 msgr2=0x7f000c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- 172.21.15.185:0/4141588659 shutdown_connections 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- 172.21.15.185:0/4141588659 wait complete. 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 Processor -- start 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- start start 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 0x7f000c1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c0b3920 0x7f000c13b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153610 con 0x7f000c0b2e80 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153820 con 0x7f000c13dcf0 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f00109fc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c153ac0 con 0x7f000c0b3920 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c0b3920 0x7f000c13b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59240/0 (socket says 172.21.15.185:59240) 2020-11-16T22:17:12.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 -- 172.21.15.185:0/2061141293 learned_addr learned my addr 172.21.15.185:0/2061141293 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000ad9d700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 0x7f000c1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 -- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c0b3920 msgr2=0x7f000c13b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c0b3920 0x7f000c13b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 -- 172.21.15.185:0/2061141293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 msgr2=0x7f000c1454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 0x7f000c1454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 -- 172.21.15.185:0/2061141293 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c153270 con 0x7f000c13dcf0 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000ad9d700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 0x7f000c1454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:12.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.083+0000 7f000b59e700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f000400ef20 tx=0x7f00040091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f0002ffd700 1 -- 172.21.15.185:0/2061141293 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004009690 con 0x7f000c13dcf0 2020-11-16T22:17:12.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f0002ffd700 1 -- 172.21.15.185:0/2061141293 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400cbc0 con 0x7f000c13dcf0 2020-11-16T22:17:12.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f0002ffd700 1 -- 172.21.15.185:0/2061141293 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040123f0 con 0x7f000c13dcf0 2020-11-16T22:17:12.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c1459e0 con 0x7f000c13dcf0 2020-11-16T22:17:12.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c154100 con 0x7f000c13dcf0 2020-11-16T22:17:12.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f0002ffd700 1 -- 172.21.15.185:0/2061141293 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0004022070 con 0x7f000c13dcf0 2020-11-16T22:17:12.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.087+0000 7f0002ffd700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 0x7effdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 0x7effdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f0002ffd700 1 -- 172.21.15.185:0/2061141293 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f000406e520 con 0x7f000c13dcf0 2020-11-16T22:17:12.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 0x7effdc057520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7efffc00d4a0 tx=0x7efffc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 Processor -- start 2020-11-16T22:17:12.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 -- start start 2020-11-16T22:17:12.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xdb4bb0 0x103ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 0x103f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xecc4b0 con 0xdb4bb0 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xf80930 con 0xe227b0 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe406f0 con 0x1040da0 2020-11-16T22:17:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59244/0 (socket says 172.21.15.185:59244) 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 0x103f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 -- 172.21.15.185:0/4226147322 learned_addr learned my addr 172.21.15.185:0/4226147322 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000a59c700 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xdb4bb0 0x103ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 -- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 msgr2=0x103f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 0x103f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 -- 172.21.15.185:0/4226147322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xdb4bb0 msgr2=0x103ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xdb4bb0 0x103ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.091+0000 7f000ad9d700 1 -- 172.21.15.185:0/4226147322 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe3b5a0 con 0xe227b0 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f000b59e700 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 0x103f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f000ad9d700 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7efffc012de0 tx=0x7efffc012000).ready entity=mon.1 client_cookie=9f327c54da399cd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7efffbfff700 1 -- 172.21.15.185:0/4226147322 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc012640 con 0xe227b0 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7efffbfff700 1 -- 172.21.15.185:0/4226147322 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc011cd0 con 0xe227b0 2020-11-16T22:17:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7efffbfff700 1 -- 172.21.15.185:0/4226147322 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc01e640 con 0xe227b0 2020-11-16T22:17:12.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 msgr2=0x10408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7efffc012de0 tx=0x7efffc012000).stop 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- 172.21.15.185:0/4226147322 shutdown_connections 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x10408b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1040da0 0x103f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 --2- 172.21.15.185:0/4226147322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xdb4bb0 0x103ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- 172.21.15.185:0/4226147322 >> 172.21.15.185:0/4226147322 conn(0xff0e40 msgr2=0x103ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- 172.21.15.185:0/4226147322 shutdown_connections 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- 172.21.15.185:0/4226147322 wait complete. 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 -- start start 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.095+0000 7f001fc8e740 1 Processor -- start 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1040da0 0x1088310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xf6d6e0 con 0x1040da0 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xf6d820 con 0xdb4bb0 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xf6ea90 con 0xe227b0 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52696/0 (socket says 172.21.15.185:52696) 2020-11-16T22:17:12.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59250/0 (socket says 172.21.15.185:59250) 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 10 client.0 ll_register_callbacks cb 0xf63ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 -- 172.21.15.185:0/1952935593 learned_addr learned my addr 172.21.15.185:0/1952935593 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000b59e700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1040da0 0x1088310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 -- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 msgr2=0x103c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 -- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1040da0 msgr2=0x1088310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1040da0 0x1088310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 -- 172.21.15.185:0/1952935593 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xf5e280 con 0xe227b0 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000ad9d700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000a59c700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7efff4010000 tx=0x7efff4015000).ready entity=mon.2 client_cookie=7f322e187219a74c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4024070 con 0xe227b0 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10216c0 con 0xe227b0 2020-11-16T22:17:12.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efff4011820 con 0xe227b0 2020-11-16T22:17:12.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff4011bf0 con 0xe227b0 2020-11-16T22:17:12.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 10 client.5507 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:12.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0xd93fd0 con 0xe227b0 2020-11-16T22:17:12.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10216c0 con 0xe227b0 2020-11-16T22:17:12.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7efff401fa90 con 0xe227b0 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 10 client.5507 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 10 client.5507 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 20 client.5507 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1088800 0x106f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 4 client.5507 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:12.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xe5ec50 con 0x1088800 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000ad9d700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1088800 0x106f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7efff97fa700 10 client.5507 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.099+0000 7f000ad9d700 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1088800 0x106f0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=df1413f2c206928e server_cookie=cb70b5487b5be1a0 in_seq=0 out_seq=0 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7efff97fa700 1 -- 172.21.15.185:0/1952935593 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xe5ec50 con 0x1088800 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7efff97fa700 10 client.5507 handle_command_reply: tid=0 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 client.5507 shutdown 2020-11-16T22:17:12.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 msgr2=0x1084670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7efff4010000 tx=0x7efff4015000).stop 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 shutdown_connections 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1088800 0x106f0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x103c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7efff97fa700 0 client.5507 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1040da0 0x1088310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 --2- 172.21.15.185:0/1952935593 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe227b0 0x1084670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 >> 172.21.15.185:0/1952935593 conn(0xff0e40 msgr2=0x103ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 shutdown_connections 2020-11-16T22:17:12.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 1 -- 172.21.15.185:0/1952935593 wait complete. 2020-11-16T22:17:12.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.103+0000 7f001fc8e740 20 client.5507 trim_cache size 0 max 16384 2020-11-16T22:17:12.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.119+0000 7f001fc8e740 1 Processor -- start 2020-11-16T22:17:12.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.119+0000 7f001fc8e740 1 -- start start 2020-11-16T22:17:12.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xff0e40 0x1035870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xf45e40 con 0xff0e40 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x10434c0 con 0xe227b0 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xf9d680 con 0xdb4bb0 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59262/0 (socket says 172.21.15.185:59262) 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000b59e700 1 -- 172.21.15.185:0/3095128128 learned_addr learned my addr 172.21.15.185:0/3095128128 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52704/0 (socket says 172.21.15.185:52704) 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000ad9d700 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xff0e40 0x1035870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 -- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 msgr2=0x1043660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 -- 172.21.15.185:0/3095128128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xff0e40 msgr2=0x1035870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xff0e40 0x1035870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 -- 172.21.15.185:0/3095128128 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1088670 con 0xdb4bb0 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000b59e700 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7efff4010000 tx=0x7efff4024000).ready entity=mon.2 client_cookie=d7ffed87668a6db8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f0000c96700 1 -- 172.21.15.185:0/3095128128 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011bf0 con 0xdb4bb0 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f0000c96700 1 -- 172.21.15.185:0/3095128128 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4011db0 con 0xdb4bb0 2020-11-16T22:17:12.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f0000c96700 1 -- 172.21.15.185:0/3095128128 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff401f690 con 0xdb4bb0 2020-11-16T22:17:12.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 msgr2=0x103ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7efff4010000 tx=0x7efff4024000).stop 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- 172.21.15.185:0/3095128128 shutdown_connections 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe227b0 0x1043660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xff0e40 0x1035870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- 172.21.15.185:0/3095128128 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xdb4bb0 0x103ed00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- 172.21.15.185:0/3095128128 >> 172.21.15.185:0/3095128128 conn(0x7effd4002320 msgr2=0x1076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- 172.21.15.185:0/3095128128 shutdown_connections 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- 172.21.15.185:0/3095128128 wait complete. 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- start start 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 Processor -- start 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe227b0 0x107d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xff0e40 0x107d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x107e7f0 con 0xe227b0 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1076a20 con 0xdb4bb0 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x103f430 con 0xff0e40 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f001fc8e740 10 client.0 ll_register_callbacks cb 0x10736b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000b59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xff0e40 0x107d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59264/0 (socket says 172.21.15.185:59264) 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000ad9d700 1 -- 172.21.15.185:0/710428988 learned_addr learned my addr 172.21.15.185:0/710428988 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.123+0000 7f000a59c700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe227b0 0x107d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 -- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xff0e40 msgr2=0x107d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xff0e40 0x107d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 -- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe227b0 msgr2=0x107d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe227b0 0x107d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 -- 172.21.15.185:0/710428988 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd4008150 con 0xdb4bb0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000ad9d700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7efffc00d270 tx=0x7efffc02e040).ready entity=mon.1 client_cookie=6ddd11cc6204838d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc031040 con 0xdb4bb0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efffc01e9f0 con 0xdb4bb0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efffc011cd0 con 0xdb4bb0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efffc01edc0 con 0xdb4bb0 2020-11-16T22:17:12.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 10 client.5956 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0xdd2720 con 0xdb4bb0 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc011cd0 con 0xdb4bb0 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7efffc02cca0 con 0xdb4bb0 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 10 client.5956 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 10 client.5956 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 20 client.5956 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:12.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1040da0 0x10e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 4 client.5956 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xe65f50 con 0x1040da0 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000a59c700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1040da0 0x10e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f000a59c700 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1040da0 0x10e3340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=8ed1c7e721f15bd0 server_cookie=7078b684c84c8d0f in_seq=0 out_seq=0 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 10 client.5956 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 1 -- 172.21.15.185:0/710428988 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0xe65f50 con 0x1040da0 2020-11-16T22:17:12.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 10 client.5956 handle_command_reply: tid=0 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 client.5956 shutdown 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 msgr2=0x10e07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7efffc00d270 tx=0x7efffc02e040).stop 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 shutdown_connections 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x1040da0 0x10e3340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xdb4bb0 0x10e07b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7efff97fa700 0 client.5956 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xff0e40 0x107d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 --2- 172.21.15.185:0/710428988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe227b0 0x107d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 >> 172.21.15.185:0/710428988 conn(0x7effd4002320 msgr2=0x1070310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 shutdown_connections 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 1 -- 172.21.15.185:0/710428988 wait complete. 2020-11-16T22:17:12.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.127+0000 7f001fc8e740 20 client.5956 trim_cache size 0 max 16384 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "d3471dec-506f-4ac4-84f7-91a754072318": { 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "d3471dec-506f-4ac4-84f7-91a754072318", 2020-11-16T22:17:12.122 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:17:12.123 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:12.123 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:12.123 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:12.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 msgr2=0x7effdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 0x7effdc057520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7efffc00d4a0 tx=0x7efffc008040).stop 2020-11-16T22:17:12.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 msgr2=0x7f000c13b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:12.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f000400ef20 tx=0x7f00040091d0).stop 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 shutdown_connections 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7effdc055060 0x7effdc057520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f000c13dcf0 0x7f000c13b9f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f000c0b2e80 0x7f000c1454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 --2- 172.21.15.185:0/2061141293 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f000c0b3920 0x7f000c13b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 >> 172.21.15.185:0/2061141293 conn(0x7f000c060130 msgr2=0x7f000c13a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:12.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 shutdown_connections 2020-11-16T22:17:12.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:12.131+0000 7f00109fc700 1 -- 172.21.15.185:0/2061141293 wait complete. 2020-11-16T22:17:12.131 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:17:17.133 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:17.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 Processor -- start 2020-11-16T22:17:17.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 -- start start 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 0x7f8ec40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec4147f90 con 0x7f8ec40acee0 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40af020 con 0x7f8ec40b72e0 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ecb3d2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40b4810 con 0x7f8ec40b64b0 2020-11-16T22:17:17.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59272/0 (socket says 172.21.15.185:59272) 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 -- 172.21.15.185:0/3619866241 learned_addr learned my addr 172.21.15.185:0/3619866241 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8eca3d0700 1 --2- 172.21.15.185:0/3619866241 >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 0x7f8ec40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 -- 172.21.15.185:0/3619866241 >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 msgr2=0x7f8ec40b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 --2- 172.21.15.185:0/3619866241 >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 0x7f8ec40b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 -- 172.21.15.185:0/3619866241 >> v1:172.21.15.185:6789/0 conn(0x7f8ec40acee0 legacy=0x7f8ec4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.283+0000 7f8ec9bcf700 1 -- 172.21.15.185:0/3619866241 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4148460 con 0x7f8ec40b64b0 2020-11-16T22:17:17.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8eca3d0700 1 --2- 172.21.15.185:0/3619866241 >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 0x7f8ec40b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:17.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ec9bcf700 1 --2- 172.21.15.185:0/3619866241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8ec40b71a0 tx=0x7f8eb4006eb0).ready entity=mon.1 client_cookie=4c3c06d640dad0a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ec93ce700 1 -- 172.21.15.185:0/3619866241 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40098d0 con 0x7f8ec40b64b0 2020-11-16T22:17:17.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ec93ce700 1 -- 172.21.15.185:0/3619866241 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4009a30 con 0x7f8ec40b64b0 2020-11-16T22:17:17.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ec93ce700 1 -- 172.21.15.185:0/3619866241 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4009db0 con 0x7f8ec40b64b0 2020-11-16T22:17:17.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3619866241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 msgr2=0x7f8ec40b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3619866241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8ec40b71a0 tx=0x7f8eb4006eb0).stop 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3619866241 shutdown_connections 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3619866241 >> v2:172.21.15.96:3300/0 conn(0x7f8ec40b72e0 0x7f8ec40b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3619866241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec40b6aa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3619866241 >> 172.21.15.185:0/3619866241 conn(0x7f8ec4067860 msgr2=0x7f8ec4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3619866241 shutdown_connections 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3619866241 wait complete. 2020-11-16T22:17:17.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 Processor -- start 2020-11-16T22:17:17.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.287+0000 7f8ecb3d2700 1 -- start start 2020-11-16T22:17:17.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec40acee0 0x7f8ec40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 0x7f8ec40b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec41414f0 con 0x7f8ec40acee0 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4156b20 con 0x7f8ec40b64b0 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4156dc0 con 0x7f8ec40b72e0 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ec9bcf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 0x7f8ec40b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ec9bcf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59280/0 (socket says 172.21.15.185:59280) 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ec9bcf700 1 -- 172.21.15.185:0/3817529231 learned_addr learned my addr 172.21.15.185:0/3817529231 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8eca3d0700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec40acee0 0x7f8ec40afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 -- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 msgr2=0x7f8ec4157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 -- 172.21.15.185:0/3817529231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec40acee0 msgr2=0x7f8ec40afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec40acee0 0x7f8ec40afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 -- 172.21.15.185:0/3817529231 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40b0390 con 0x7f8ec40b72e0 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ec9bcf700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecabd1700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 0x7f8ec40b06d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8ebc00c810 tx=0x7f8ebc007e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00e560 con 0x7f8ec40b72e0 2020-11-16T22:17:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc00e6c0 con 0x7f8ec40b72e0 2020-11-16T22:17:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00ea40 con 0x7f8ec40b72e0 2020-11-16T22:17:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec4141790 con 0x7f8ec40b72e0 2020-11-16T22:17:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.291+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec40b3350 con 0x7f8ec40b72e0 2020-11-16T22:17:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec405d200 con 0x7f8ec40b72e0 2020-11-16T22:17:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8ebc010070 con 0x7f8ec40b72e0 2020-11-16T22:17:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8eb27fc700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 0x7f8e9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ebc06d860 con 0x7f8ec40b72e0 2020-11-16T22:17:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8eca3d0700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 0x7f8e9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.295+0000 7f8eca3d0700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 0x7f8e9c057520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f8ec000a610 tx=0x7f8ec0006f90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.299+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8ebc043ac0 con 0x7f8ec40b72e0 2020-11-16T22:17:17.333 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:17.340 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:17.342 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:17.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.503+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ec40603a0 con 0x7f8ec40b72e0 2020-11-16T22:17:17.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.503+0000 7f8eb27fc700 1 -- 172.21.15.185:0/3817529231 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f8ebc0470e0 con 0x7f8ec40b72e0 2020-11-16T22:17:17.489 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:17.489 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:17.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 msgr2=0x7f8e9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 0x7f8e9c057520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f8ec000a610 tx=0x7f8ec0006f90).stop 2020-11-16T22:17:17.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 msgr2=0x7f8ec40b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 0x7f8ec40b06d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8ebc00c810 tx=0x7f8ebc007e80).stop 2020-11-16T22:17:17.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 shutdown_connections 2020-11-16T22:17:17.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e9c055060 0x7f8e9c057520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ec40b72e0 0x7f8ec40b06d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ec40acee0 0x7f8ec40afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 --2- 172.21.15.185:0/3817529231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ec40b64b0 0x7f8ec4157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 >> 172.21.15.185:0/3817529231 conn(0x7f8ec4067860 msgr2=0x7f8ec4144880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 shutdown_connections 2020-11-16T22:17:17.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.507+0000 7f8ecb3d2700 1 -- 172.21.15.185:0/3817529231 wait complete. 2020-11-16T22:17:17.493 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:17.504 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 Processor -- start 2020-11-16T22:17:17.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- start start 2020-11-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2b00b2e80 0x7fd2b00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 0x7fd2b0143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b00ac070 con 0x7fd2b013dcf0 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00ac310 con 0x7fd2b00b3920 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00ac5b0 con 0x7fd2b00b2e80 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2b00b2e80 0x7fd2b00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd2b00b2e80 0x7fd2b00b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59286/0 (socket says 172.21.15.185:59286) 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 -- 172.21.15.185:0/2322118500 learned_addr learned my addr 172.21.15.185:0/2322118500 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/2322118500 >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 0x7fd2b0143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 -- 172.21.15.185:0/2322118500 >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 msgr2=0x7fd2b0143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/2322118500 >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 0x7fd2b0143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 -- 172.21.15.185:0/2322118500 >> v1:172.21.15.96:6789/0 conn(0x7fd2b013dcf0 legacy=0x7fd2b00b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 -- 172.21.15.185:0/2322118500 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00ac850 con 0x7fd2b00b2e80 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/2322118500 >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 0x7fd2b0143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/2322118500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b00b2e80 0x7fd2b00b3260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fd2b0064930 tx=0x7fd2a0009290).ready entity=mon.1 client_cookie=e776dc6ebb2c0300 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b50b8700 1 -- 172.21.15.185:0/2322118500 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000bb80 con 0x7fd2b00b2e80 2020-11-16T22:17:17.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b50b8700 1 -- 172.21.15.185:0/2322118500 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000bce0 con 0x7fd2b00b2e80 2020-11-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b50b8700 1 -- 172.21.15.185:0/2322118500 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0011610 con 0x7fd2b00b2e80 2020-11-16T22:17:17.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- 172.21.15.185:0/2322118500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b00b2e80 msgr2=0x7fd2b00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/2322118500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b00b2e80 0x7fd2b00b3260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fd2b0064930 tx=0x7fd2a0009290).stop 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- 172.21.15.185:0/2322118500 shutdown_connections 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/2322118500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b00b2e80 0x7fd2b00b3260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/2322118500 >> v2:172.21.15.96:3300/0 conn(0x7fd2b00b3920 0x7fd2b0143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- 172.21.15.185:0/2322118500 >> 172.21.15.185:0/2322118500 conn(0x7fd2b0060130 msgr2=0x7fd2b0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- 172.21.15.185:0/2322118500 shutdown_connections 2020-11-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.647+0000 7fd2b70bc700 1 -- 172.21.15.185:0/2322118500 wait complete. 2020-11-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 Processor -- start 2020-11-16T22:17:17.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- start start 2020-11-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b00b2e80 0x7fd2b0146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2b00b3920 0x7fd2b00ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b013eba0 con 0x7fd2b00b2e80 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0147b60 con 0x7fd2b013dcf0 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0157470 con 0x7fd2b00b3920 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59294/0 (socket says 172.21.15.185:59294) 2020-11-16T22:17:17.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 -- 172.21.15.185:0/1838968896 learned_addr learned my addr 172.21.15.185:0/1838968896 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2b00b3920 0x7fd2b00ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b00b2e80 0x7fd2b0146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 -- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2b00b3920 msgr2=0x7fd2b00ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2b00b3920 0x7fd2b00ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 -- 172.21.15.185:0/1838968896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b00b2e80 msgr2=0x7fd2b0146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b00b2e80 0x7fd2b0146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 -- 172.21.15.185:0/1838968896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0147df0 con 0x7fd2b013dcf0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fd2a800c760 tx=0x7fd2a8007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2ae7fc700 1 -- 172.21.15.185:0/1838968896 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e490 con 0x7fd2b013dcf0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2ae7fc700 1 -- 172.21.15.185:0/1838968896 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800e610 con 0x7fd2b013dcf0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0148090 con 0x7fd2b013dcf0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2ae7fc700 1 -- 172.21.15.185:0/1838968896 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e9d0 con 0x7fd2b013dcf0 2020-11-16T22:17:17.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.651+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b00affc0 con 0x7fd2b013dcf0 2020-11-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.655+0000 7fd2ae7fc700 1 -- 172.21.15.185:0/1838968896 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd2a8010030 con 0x7fd2b013dcf0 2020-11-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.655+0000 7fd2ae7fc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 0x7fd28c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.655+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 0x7fd28c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.655+0000 7fd2ae7fc700 1 -- 172.21.15.185:0/1838968896 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2a8002e40 con 0x7fd2b013dcf0 2020-11-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.655+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 0x7fd28c057520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd2a0009fd0 tx=0x7fd2a0016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 Processor -- start 2020-11-16T22:17:17.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 -- start start 2020-11-16T22:17:17.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x22cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22ccda0 0x22cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21584b0 con 0x2040bb0 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x220c930 con 0x20ae7b0 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20cc6f0 con 0x22ccda0 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59300/0 (socket says 172.21.15.185:59300) 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 -- 172.21.15.185:0/133017905 learned_addr learned my addr 172.21.15.185:0/133017905 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22ccda0 0x22cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x22cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 -- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22ccda0 msgr2=0x22cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22ccda0 0x22cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 -- 172.21.15.185:0/133017905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 msgr2=0x22cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x22cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.659+0000 7fd2b60ba700 1 -- 172.21.15.185:0/133017905 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20c75a0 con 0x20ae7b0 2020-11-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fd2a0011000 tx=0x7fd2a001d000).ready entity=mon.1 client_cookie=8efb73e300ff06d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2a77fe700 1 -- 172.21.15.185:0/133017905 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a001c650 con 0x20ae7b0 2020-11-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2a77fe700 1 -- 172.21.15.185:0/133017905 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a001c7b0 con 0x20ae7b0 2020-11-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2a77fe700 1 -- 172.21.15.185:0/133017905 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a001cb30 con 0x20ae7b0 2020-11-16T22:17:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 msgr2=0x22cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fd2a0011000 tx=0x7fd2a001d000).stop 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- 172.21.15.185:0/133017905 shutdown_connections 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x22cc8b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22ccda0 0x22cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/133017905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x22cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- 172.21.15.185:0/133017905 >> 172.21.15.185:0/133017905 conn(0x227ce40 msgr2=0x22c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- 172.21.15.185:0/133017905 shutdown_connections 2020-11-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- 172.21.15.185:0/133017905 wait complete. 2020-11-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- start start 2020-11-16T22:17:17.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 Processor -- start 2020-11-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22ccda0 0x22fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f96e0 con 0x22ccda0 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f9820 con 0x20ae7b0 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x21faa90 con 0x2040bb0 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2c6daf740 10 client.0 ll_register_callbacks cb 0x21efad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b60ba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b60ba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52750/0 (socket says 172.21.15.185:52750) 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59308/0 (socket says 172.21.15.185:59308) 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b60ba700 1 -- 172.21.15.185:0/975006280 learned_addr learned my addr 172.21.15.185:0/975006280 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.663+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22ccda0 0x22fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 -- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 msgr2=0x2300880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 -- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22ccda0 msgr2=0x22fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22ccda0 0x22fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 -- 172.21.15.185:0/975006280 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21ea280 con 0x2040bb0 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fd2a00090f0 tx=0x7fd2a0027000).ready entity=mon.2 client_cookie=61da7e9897f24e33 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a001c650 con 0x2040bb0 2020-11-16T22:17:17.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22ad6c0 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2a000be50 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a0027b00 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 10 client.5525 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x201ffd0 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0042020 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd2a0027c60 con 0x2040bb0 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 10 client.5525 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:17.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 10 client.5525 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:17.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 20 client.5525 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:17.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2300d70 0x2314450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 4 client.5525 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:17.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x20eac50 con 0x2300d70 2020-11-16T22:17:17.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2300d70 0x2314450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 10 client.5525 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:17.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2300d70 0x2314450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=c86571e6d69f3e2c server_cookie=50ce3034de4667ce in_seq=0 out_seq=0 2020-11-16T22:17:17.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/975006280 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20eac50 con 0x2300d70 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2a4ff9700 10 client.5525 handle_command_reply: tid=0 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.667+0000 7fd2c6daf740 1 client.5525 shutdown 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 msgr2=0x2310640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fd2a00090f0 tx=0x7fd2a0027000).stop 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 shutdown_connections 2020-11-16T22:17:17.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x2300d70 0x2314450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22ccda0 0x22fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x2310640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/975006280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2300880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2a4ff9700 0 client.5525 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 >> 172.21.15.185:0/975006280 conn(0x227ce40 msgr2=0x22c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 shutdown_connections 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 1 -- 172.21.15.185:0/975006280 wait complete. 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.671+0000 7fd2c6daf740 20 client.5525 trim_cache size 0 max 16384 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 Processor -- start 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 -- start start 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 0x22c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2310be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d1e40 con 0x227ce40 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22c7380 con 0x20ae7b0 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x22ce230 con 0x2040bb0 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b68bb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2310be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52762/0 (socket says 172.21.15.185:52762) 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 -- 172.21.15.185:0/2273138316 learned_addr learned my addr 172.21.15.185:0/2273138316 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 0x22c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 -- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 msgr2=0x2310be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2310be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 -- 172.21.15.185:0/2273138316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 msgr2=0x22c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 0x22c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 -- 172.21.15.185:0/2273138316 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284008160 con 0x2040bb0 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 0x22c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:17.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x22c8c80 tx=0x7fd298016040).ready entity=mon.2 client_cookie=4ba9772c96f98c92 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.691+0000 7fd2a7fff700 1 -- 172.21.15.185:0/2273138316 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298014ba0 con 0x2040bb0 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a7fff700 1 -- 172.21.15.185:0/2273138316 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29800c040 con 0x2040bb0 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a7fff700 1 -- 172.21.15.185:0/2273138316 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29801d470 con 0x2040bb0 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 msgr2=0x22fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x22c8c80 tx=0x7fd298016040).stop 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/2273138316 shutdown_connections 2020-11-16T22:17:17.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20ae7b0 0x2310be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227ce40 0x22c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/2273138316 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2040bb0 0x22fa6f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/2273138316 >> 172.21.15.185:0/2273138316 conn(0x7fd284002320 msgr2=0x2310370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/2273138316 shutdown_connections 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/2273138316 wait complete. 2020-11-16T22:17:17.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- start start 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 Processor -- start 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x2310730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x22cb320 con 0x2040bb0 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2302630 con 0x227ce40 2020-11-16T22:17:17.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x23028a0 con 0x20ae7b0 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 10 client.0 ll_register_callbacks cb 0x22ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59324/0 (socket says 172.21.15.185:59324) 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b58b9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52766/0 (socket says 172.21.15.185:52766) 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 -- 172.21.15.185:0/308903956 learned_addr learned my addr 172.21.15.185:0/308903956 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:17.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x2310730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 -- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 msgr2=0x2362110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 -- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 msgr2=0x2310730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x2310730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 -- 172.21.15.185:0/308903956 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x205e720 con 0x227ce40 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b58b9700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2b68bb700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd2a8047760 tx=0x7fd2a800cf90).ready entity=mon.1 client_cookie=6fe478c3c1d8efad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:17.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a802f070 con 0x227ce40 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2a801ae20 con 0x227ce40 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a001c7b0 con 0x227ce40 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a8047160 con 0x227ce40 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 10 client.5983 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x2069e00 con 0x227ce40 2020-11-16T22:17:17.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.695+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a001c7b0 con 0x227ce40 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd2a8043850 con 0x227ce40 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 10 client.5983 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 10 client.5983 resolve_mds: resolved ID 'c' to GID 5475 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 20 client.5983 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x22ccda0 0x2366320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 4 client.5983 mds_command: new command op to 5475 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:17:17.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 --> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x20f1f50 con 0x22ccda0 2020-11-16T22:17:17.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x22ccda0 0x2366320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:17.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2a4ff9700 10 client.5983 ms_handle_connect on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:17.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b60ba700 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x22ccda0 0x2366320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=9a51f6600cc6c196 server_cookie=5b8b6aaccde4c182 in_seq=0 out_seq=0 2020-11-16T22:17:17.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2a4ff9700 1 -- 172.21.15.185:0/308903956 <== mds.1 v2:172.21.15.96:6833/4236989230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x20f1f50 con 0x22ccda0 2020-11-16T22:17:17.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2a4ff9700 10 client.5983 handle_command_reply: tid=0 2020-11-16T22:17:18.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 client.5983 shutdown 2020-11-16T22:17:18.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 msgr2=0x2362600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd2a8047760 tx=0x7fd2a800cf90).stop 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 shutdown_connections 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:6833/4236989230,v1:172.21.15.96:6835/4236989230] conn(0x22ccda0 0x2366320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227ce40 0x2362600 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2a4ff9700 0 client.5983 ms_handle_reset on v2:172.21.15.96:6833/4236989230 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2040bb0 0x2310730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 --2- 172.21.15.185:0/308903956 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20ae7b0 0x2362110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 >> 172.21.15.185:0/308903956 conn(0x7fd284002320 msgr2=0x22fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 shutdown_connections 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 1 -- 172.21.15.185:0/308903956 wait complete. 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2c6daf740 20 client.5983 trim_cache size 0 max 16384 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 msgr2=0x7fd28c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 0x7fd28c057520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd2a0009fd0 tx=0x7fd2a0016000).stop 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 msgr2=0x7fd2b0156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fd2a800c760 tx=0x7fd2a8007ba0).stop 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 shutdown_connections 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd28c055060 0x7fd28c057520 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd2b013dcf0 0x7fd2b0156b00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b00b2e80 0x7fd2b0146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 --2- 172.21.15.185:0/1838968896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd2b00b3920 0x7fd2b00ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 >> 172.21.15.185:0/1838968896 conn(0x7fd2b0060130 msgr2=0x7fd2b0148f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 shutdown_connections 2020-11-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:17.699+0000 7fd2b70bc700 1 -- 172.21.15.185:0/1838968896 wait complete. 2020-11-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:17:18.082 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:18.083 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:18.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.227+0000 7f0f0bfff700 1 Processor -- start 2020-11-16T22:17:18.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.227+0000 7f0f0bfff700 1 -- start start 2020-11-16T22:17:18.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 0x7f0f0c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 0x7f0f0c141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c066ad0 con 0x7f0f0c0b7470 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c140d10 con 0x7f0f0c141100 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c144d90 con 0x7f0f0c0b2920 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 0x7f0f0c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0affd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f0f0c0b7470 0x7f0f0c13e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52766/0 (socket says 172.21.15.185:52766) 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0affd700 1 -- 172.21.15.185:0/1252565618 learned_addr learned my addr 172.21.15.185:0/1252565618 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:18.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0b7fe700 1 --2- 172.21.15.185:0/1252565618 >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 0x7f0f0c141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1252565618 >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 msgr2=0x7f0f0c141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 --2- 172.21.15.185:0/1252565618 >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 0x7f0f0c141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1252565618 >> v1:172.21.15.185:6789/0 conn(0x7f0f0c0b7470 legacy=0x7f0f0c13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1252565618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c144ff0 con 0x7f0f0c0b2920 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0b7fe700 1 --2- 172.21.15.185:0/1252565618 >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 0x7f0f0c141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0a7fc700 1 --2- 172.21.15.185:0/1252565618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 0x7f0f0c0b7850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f0f0000b9a0 tx=0x7f0f00006df0).ready entity=mon.2 client_cookie=7b081ca8370448a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f09ffb700 1 -- 172.21.15.185:0/1252565618 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009840 con 0x7f0f0c0b2920 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f09ffb700 1 -- 172.21.15.185:0/1252565618 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f000099a0 con 0x7f0f0c0b2920 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f09ffb700 1 -- 172.21.15.185:0/1252565618 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009d20 con 0x7f0f0c0b2920 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1252565618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 msgr2=0x7f0f0c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1252565618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 0x7f0f0c0b7850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f0f0000b9a0 tx=0x7f0f00006df0).stop 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1252565618 shutdown_connections 2020-11-16T22:17:18.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1252565618 >> v2:172.21.15.96:3300/0 conn(0x7f0f0c141100 0x7f0f0c141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1252565618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b2920 0x7f0f0c0b7850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1252565618 >> 172.21.15.185:0/1252565618 conn(0x7f0f0c0637b0 msgr2=0x7f0f0c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1252565618 shutdown_connections 2020-11-16T22:17:18.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.231+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1252565618 wait complete. 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 Processor -- start 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 -- start start 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c0b2920 0x7f0f0c0af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 0x7f0f0c139870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad400 con 0x7f0f0c0b2920 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad6a0 con 0x7f0f0c141100 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad940 con 0x7f0f0c0b7470 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 0x7f0f0c139870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52782/0 (socket says 172.21.15.185:52782) 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1773132104 learned_addr learned my addr 172.21.15.185:0/1773132104 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0affd700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c0b2920 0x7f0f0c0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 msgr2=0x7f0f0c139870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 0x7f0f0c139870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1773132104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c0b2920 msgr2=0x7f0f0c0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c0b2920 0x7f0f0c0af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 -- 172.21.15.185:0/1773132104 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0adbe0 con 0x7f0f0c0b7470 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0b7fe700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 0x7f0f0c139870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:18.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0a7fc700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f0f0c144ed0 tx=0x7f0f00015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00011d50 con 0x7f0f0c0b7470 2020-11-16T22:17:18.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0001e460 con 0x7f0f0c0b7470 2020-11-16T22:17:18.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0afa00 con 0x7f0f0c0b7470 2020-11-16T22:17:18.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.235+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0001e7e0 con 0x7f0f0c0b7470 2020-11-16T22:17:18.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c13ad60 con 0x7f0f0c0b7470 2020-11-16T22:17:18.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c0add70 con 0x7f0f0c0b7470 2020-11-16T22:17:18.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0f00025030 con 0x7f0f0c0b7470 2020-11-16T22:17:18.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0ef2ffd700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 0x7f0ee8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0f0006f4b0 con 0x7f0f0c0b7470 2020-11-16T22:17:18.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0f0affd700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 0x7f0ee8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.239+0000 7f0f0affd700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 0x7f0ee8057520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0efc0072f0 tx=0x7f0efc007b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.247+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0f000457c0 con 0x7f0f0c0b7470 2020-11-16T22:17:18.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f0c05d200 con 0x7f0f0c0b7470 2020-11-16T22:17:18.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0ef2ffd700 1 -- 172.21.15.185:0/1773132104 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f0f00023020 con 0x7f0f0c0b7470 2020-11-16T22:17:18.431 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:18.431 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:18.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 msgr2=0x7f0ee8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 0x7f0ee8057520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f0efc0072f0 tx=0x7f0efc007b80).stop 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 msgr2=0x7f0f0c13a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f0f0c144ed0 tx=0x7f0f00015000).stop 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 shutdown_connections 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0ee8055060 0x7f0ee8057520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0f0c141100 0x7f0f0c139870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f0c0b2920 0x7f0f0c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 --2- 172.21.15.185:0/1773132104 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0f0c0b7470 0x7f0f0c13a870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 >> 172.21.15.185:0/1773132104 conn(0x7f0f0c0637b0 msgr2=0x7f0f0c0b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 shutdown_connections 2020-11-16T22:17:18.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.447+0000 7f0f0bfff700 1 -- 172.21.15.185:0/1773132104 wait complete. 2020-11-16T22:17:18.434 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:18.445 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 Processor -- start 2020-11-16T22:17:18.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 -- start start 2020-11-16T22:17:18.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc9700b2d40 0x7fc970140be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 0x7fc9700b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 0x7fc97013d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9701410d0 con 0x7fc970144a00 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9700b3610 con 0x7fc9700b2d40 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc974fe2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b2410 con 0x7fc9700b39e0 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc96f7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc9700b2d40 0x7fc970140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc96f7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc9700b2d40 0x7fc970140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59342/0 (socket says 172.21.15.185:59342) 2020-11-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc96effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 0x7fc9700b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc96ffff700 1 --2- 172.21.15.185:0/114927597 >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 0x7fc97013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.583+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114927597 learned_addr learned my addr 172.21.15.185:0/114927597 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 msgr2=0x7fc9700b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 0x7fc9700b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114927597 >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 msgr2=0x7fc97013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114927597 >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 0x7fc97013d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114927597 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9701442c0 con 0x7fc9700b2d40 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96effd700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 0x7fc9700b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96ffff700 1 --2- 172.21.15.185:0/114927597 >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 0x7fc97013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:18.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b2d40 0x7fc970140be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fc970064930 tx=0x7fc958006eb0).ready entity=mon.1 client_cookie=66e2de0ad5354050 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96e7fc700 1 -- 172.21.15.185:0/114927597 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958009720 con 0x7fc9700b2d40 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96e7fc700 1 -- 172.21.15.185:0/114927597 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958009880 con 0x7fc9700b2d40 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc96e7fc700 1 -- 172.21.15.185:0/114927597 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958009c00 con 0x7fc9700b2d40 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b2d40 msgr2=0x7fc970140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b2d40 0x7fc970140be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fc970064930 tx=0x7fc958006eb0).stop 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- 172.21.15.185:0/114927597 shutdown_connections 2020-11-16T22:17:18.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 --2- 172.21.15.185:0/114927597 >> v2:172.21.15.96:3300/0 conn(0x7fc970144a00 0x7fc97013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b2d40 0x7fc970140be0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 --2- 172.21.15.185:0/114927597 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b39e0 0x7fc9700b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- 172.21.15.185:0/114927597 >> 172.21.15.185:0/114927597 conn(0x7fc970060130 msgr2=0x7fc970060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- 172.21.15.185:0/114927597 shutdown_connections 2020-11-16T22:17:18.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- 172.21.15.185:0/114927597 wait complete. 2020-11-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 Processor -- start 2020-11-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.587+0000 7fc974fe2700 1 -- start start 2020-11-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b2d40 0x7fc970139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970144a00 0x7fc9700b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b6f30 con 0x7fc970144a00 2020-11-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b5a30 con 0x7fc9700b39e0 2020-11-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b6170 con 0x7fc9700b2d40 2020-11-16T22:17:18.577 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59350/0 (socket says 172.21.15.185:59350) 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 -- 172.21.15.185:0/819933802 learned_addr learned my addr 172.21.15.185:0/819933802 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b2d40 0x7fc970139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96ffff700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970144a00 0x7fc9700b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 -- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b2d40 msgr2=0x7fc970139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b2d40 0x7fc970139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 -- 172.21.15.185:0/819933802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970144a00 msgr2=0x7fc9700b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970144a00 0x7fc9700b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 -- 172.21.15.185:0/819933802 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9701407e0 con 0x7fc9700b39e0 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc96effd700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc96000ef50 tx=0x7fc960006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc9677fe700 1 -- 172.21.15.185:0/819933802 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000ca80 con 0x7fc9700b39e0 2020-11-16T22:17:18.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc9677fe700 1 -- 172.21.15.185:0/819933802 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96000cbe0 con 0x7fc9700b39e0 2020-11-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97013a430 con 0x7fc9700b39e0 2020-11-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc9677fe700 1 -- 172.21.15.185:0/819933802 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9600123f0 con 0x7fc9700b39e0 2020-11-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9700b54d0 con 0x7fc9700b39e0 2020-11-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.591+0000 7fc9677fe700 1 -- 172.21.15.185:0/819933802 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc960012760 con 0x7fc9700b39e0 2020-11-16T22:17:18.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.595+0000 7fc9677fe700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 0x7fc94c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.595+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 0x7fc94c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.595+0000 7fc9677fe700 1 -- 172.21.15.185:0/819933802 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc96006e360 con 0x7fc9700b39e0 2020-11-16T22:17:18.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.595+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 0x7fc94c057520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc95800b1b0 tx=0x7fc958015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.599+0000 7fc984cd5740 1 Processor -- start 2020-11-16T22:17:18.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.599+0000 7fc984cd5740 1 -- start start 2020-11-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24a3da0 0x24a2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x232f4b0 con 0x2217bb0 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x23e3930 con 0x24a3da0 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x22a36f0 con 0x22857b0 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52800/0 (socket says 172.21.15.185:52800) 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114106179 learned_addr learned my addr 172.21.15.185:0/114106179 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96ffff700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24a3da0 0x24a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96effd700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24a3da0 msgr2=0x24a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24a3da0 0x24a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114106179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 msgr2=0x24a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 -- 172.21.15.185:0/114106179 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x229e5a0 con 0x22857b0 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96effd700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc970141210 tx=0x7fc95801f000).ready entity=mon.2 client_cookie=c41e9258d5bcf0bd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc964ff9700 1 -- 172.21.15.185:0/114106179 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9580069d0 con 0x22857b0 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc964ff9700 1 -- 172.21.15.185:0/114106179 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800b6d0 con 0x22857b0 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc964ff9700 1 -- 172.21.15.185:0/114106179 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95801ecb0 con 0x22857b0 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 msgr2=0x24a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc970141210 tx=0x7fc95801f000).stop 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- 172.21.15.185:0/114106179 shutdown_connections 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24a3da0 0x24a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x24a38b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 --2- 172.21.15.185:0/114106179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- 172.21.15.185:0/114106179 >> 172.21.15.185:0/114106179 conn(0x2453e40 msgr2=0x24a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- 172.21.15.185:0/114106179 shutdown_connections 2020-11-16T22:17:18.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- 172.21.15.185:0/114106179 wait complete. 2020-11-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 -- start start 2020-11-16T22:17:18.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.603+0000 7fc984cd5740 1 Processor -- start 2020-11-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 0x24a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 0x24e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 0x24e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d06e0 con 0x22857b0 2020-11-16T22:17:18.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d0820 con 0x2217bb0 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x23d1a90 con 0x24a3da0 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 0x24a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 0x24a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59362/0 (socket says 172.21.15.185:59362) 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 0x24e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96f7fe700 1 -- 172.21.15.185:0/1270635830 learned_addr learned my addr 172.21.15.185:0/1270635830 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 10 client.0 ll_register_callbacks cb 0x23c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:18.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96effd700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 0x24e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 -- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 msgr2=0x24a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 0x24a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 -- 172.21.15.185:0/1270635830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 msgr2=0x24e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 0x24e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 -- 172.21.15.185:0/1270635830 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23c1280 con 0x24a3da0 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96effd700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 0x24e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:18.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96ffff700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 0x24e7da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc968014d30 tx=0x7fc9680091d0).ready entity=mon.2 client_cookie=daef9d16e6a99eb6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968012a00 con 0x24a3da0 2020-11-16T22:17:18.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc968012b60 con 0x24a3da0 2020-11-16T22:17:18.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24846c0 con 0x24a3da0 2020-11-16T22:17:18.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc968012d20 con 0x24a3da0 2020-11-16T22:17:18.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 10 client.5549 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x21f6fd0 con 0x24a3da0 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24846c0 con 0x24a3da0 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fc96801eab0 con 0x24a3da0 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 10 client.5549 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 10 client.5549 resolve_mds: resolved ID 'a' to GID 5183 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 20 client.5549 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:18.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24d92d0 0x24d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 4 client.5549 mds_command: new command op to 5183 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 --> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x22c1c50 con 0x24d92d0 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24d92d0 0x24d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24d92d0 0x24d1640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=5417a9d86ac5112 server_cookie=cecb1509327b1435 in_seq=0 out_seq=0 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.607+0000 7fc945ffb700 10 client.5549 ms_handle_connect on v2:172.21.15.185:6834/2473550168 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc945ffb700 1 -- 172.21.15.185:0/1270635830 <== mds.2 v2:172.21.15.185:6834/2473550168 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x22c1c50 con 0x24d92d0 2020-11-16T22:17:18.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc945ffb700 10 client.5549 handle_command_reply: tid=0 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 client.5549 shutdown 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 msgr2=0x24e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 0x24e7da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fc968014d30 tx=0x7fc9680091d0).stop 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 shutdown_connections 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2217bb0 0x24a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24d92d0 0x24d1640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24a3da0 0x24e7da0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc945ffb700 0 client.5549 ms_handle_reset on v2:172.21.15.185:6834/2473550168 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 --2- 172.21.15.185:0/1270635830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22857b0 0x24e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 >> 172.21.15.185:0/1270635830 conn(0x2453e40 msgr2=0x24a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 shutdown_connections 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 1 -- 172.21.15.185:0/1270635830 wait complete. 2020-11-16T22:17:18.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.611+0000 7fc984cd5740 20 client.5549 trim_cache size 0 max 16384 2020-11-16T22:17:18.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 Processor -- start 2020-11-16T22:17:18.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 -- start start 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 0x24a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 0x24d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23a8e40 con 0x2217bb0 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24a4180 con 0x22857b0 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x249eb00 con 0x2453e40 2020-11-16T22:17:18.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc96ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 0x24d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 0x24a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc96ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 0x24d8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59372/0 (socket says 172.21.15.185:59372) 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.631+0000 7fc96ffff700 1 -- 172.21.15.185:0/4121569951 learned_addr learned my addr 172.21.15.185:0/4121569951 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96effd700 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 -- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 msgr2=0x24d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 0x24d8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 -- 172.21.15.185:0/4121569951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 msgr2=0x24d8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:18.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24d8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:18.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 -- 172.21.15.185:0/4121569951 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c008160 con 0x2453e40 2020-11-16T22:17:18.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 0x24a1f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x24a0760 tx=0x7fc958013000).ready entity=mon.2 client_cookie=c66cf2499b7d01c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc9657fa700 1 -- 172.21.15.185:0/4121569951 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958035070 con 0x2453e40 2020-11-16T22:17:19.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc9657fa700 1 -- 172.21.15.185:0/4121569951 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958036040 con 0x2453e40 2020-11-16T22:17:19.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc9657fa700 1 -- 172.21.15.185:0/4121569951 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958039280 con 0x2453e40 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 msgr2=0x24a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 0x24a1f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x24a0760 tx=0x7fc958013000).stop 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- 172.21.15.185:0/4121569951 shutdown_connections 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22857b0 0x24d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2453e40 0x24a1f10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 --2- 172.21.15.185:0/4121569951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x24d8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- 172.21.15.185:0/4121569951 >> 172.21.15.185:0/4121569951 conn(0x7fc93c002320 msgr2=0x24f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- 172.21.15.185:0/4121569951 shutdown_connections 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- 172.21.15.185:0/4121569951 wait complete. 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 -- start start 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.635+0000 7fc984cd5740 1 Processor -- start 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x2533390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 0x253cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24a6440 con 0x2217bb0 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d62e0 con 0x2453e40 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d6580 con 0x22857b0 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 10 client.0 ll_register_callbacks cb 0x24a1690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 0x253cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52822/0 (socket says 172.21.15.185:52822) 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 -- 172.21.15.185:0/3190054876 learned_addr learned my addr 172.21.15.185:0/3190054876 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 0x253cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59380/0 (socket says 172.21.15.185:59380) 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x2533390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 -- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 msgr2=0x253cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 0x253cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 -- 172.21.15.185:0/3190054876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 msgr2=0x2533390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x2533390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 -- 172.21.15.185:0/3190054876 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2235720 con 0x22857b0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96effd700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc960006a30 tx=0x7fc96006e040).ready entity=mon.2 client_cookie=7b8e1c51ef93ecaf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96006e170 con 0x22857b0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc960043ff0 con 0x22857b0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc95800b6d0 con 0x22857b0 2020-11-16T22:17:19.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc96006cab0 con 0x22857b0 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 10 client.5555 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x2240e00 con 0x22857b0 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800b6d0 con 0x22857b0 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fc96006cc10 con 0x22857b0 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 10 client.5555 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 10 client.5555 resolve_mds: resolved ID 'a' to GID 5183 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 20 client.5555 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:19.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24a3da0 0x253d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 4 client.5555 mds_command: new command op to 5183 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 --> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x22c8f50 con 0x24a3da0 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24a3da0 0x253d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.639+0000 7fc945ffb700 10 client.5555 ms_handle_connect on v2:172.21.15.185:6834/2473550168 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc96f7fe700 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24a3da0 0x253d480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=3abd9b97016dfe33 server_cookie=258d4e4029e40851 in_seq=0 out_seq=0 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc945ffb700 1 -- 172.21.15.185:0/3190054876 <== mds.2 v2:172.21.15.185:6834/2473550168 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x22c8f50 con 0x24a3da0 2020-11-16T22:17:19.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc945ffb700 10 client.5555 handle_command_reply: tid=0 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 client.5555 shutdown 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 msgr2=0x2533880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc960006a30 tx=0x7fc96006e040).stop 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 shutdown_connections 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2453e40 0x253cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:6834/2473550168,v1:172.21.15.185:6835/2473550168] conn(0x24a3da0 0x253d480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2217bb0 0x2533390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 --2- 172.21.15.185:0/3190054876 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22857b0 0x2533880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 >> 172.21.15.185:0/3190054876 conn(0x7fc93c002320 msgr2=0x24988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc945ffb700 0 client.5555 ms_handle_reset on v2:172.21.15.185:6834/2473550168 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 shutdown_connections 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 1 -- 172.21.15.185:0/3190054876 wait complete. 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc984cd5740 20 client.5555 trim_cache size 0 max 16384 2020-11-16T22:17:19.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 msgr2=0x7fc94c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 0x7fc94c057520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc95800b1b0 tx=0x7fc958015000).stop 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 msgr2=0x7fc9700afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fc96000ef50 tx=0x7fc960006df0).stop 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 shutdown_connections 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc94c055060 0x7fc94c057520 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970144a00 0x7fc9700b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700b2d40 0x7fc970139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 --2- 172.21.15.185:0/819933802 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc9700b39e0 0x7fc9700afc20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 >> 172.21.15.185:0/819933802 conn(0x7fc970060130 msgr2=0x7fc970142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 shutdown_connections 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:18.643+0000 7fc974fe2700 1 -- 172.21.15.185:0/819933802 wait complete. 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:19.025 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:17:19.026 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:17:19.026 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:19.026 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:19.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 Processor -- start 2020-11-16T22:17:19.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 -- start start 2020-11-16T22:17:19.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9f640b72e0 0x7f9f640b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 0x7f9f640b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f64147f90 con 0x7f9f640acee0 2020-11-16T22:17:19.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f640af020 con 0x7f9f640b64b0 2020-11-16T22:17:19.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f640b4810 con 0x7f9f640b72e0 2020-11-16T22:17:19.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9f640b72e0 0x7f9f640b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f62d9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f9f640acee0 0x7f9f64144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52826/0 (socket says 172.21.15.185:52826) 2020-11-16T22:17:19.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f62d9d700 1 -- 172.21.15.185:0/1027622766 learned_addr learned my addr 172.21.15.185:0/1027622766 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:19.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f61d9b700 1 --2- 172.21.15.185:0/1027622766 >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 0x7f9f640b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 -- 172.21.15.185:0/1027622766 >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 msgr2=0x7f9f640b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 --2- 172.21.15.185:0/1027622766 >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 0x7f9f640b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 -- 172.21.15.185:0/1027622766 >> v1:172.21.15.185:6789/0 conn(0x7f9f640acee0 legacy=0x7f9f64144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 -- 172.21.15.185:0/1027622766 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64148460 con 0x7f9f640b72e0 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f61d9b700 1 --2- 172.21.15.185:0/1027622766 >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 0x7f9f640b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6259c700 1 --2- 172.21.15.185:0/1027622766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f640b72e0 0x7f9f640b76c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f9f64080a30 tx=0x7f9f5c0091a0).ready entity=mon.1 client_cookie=2d040aff14cc5c2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.175+0000 7f9f6159a700 1 -- 172.21.15.185:0/1027622766 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00bbc0 con 0x7f9f640b72e0 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6159a700 1 -- 172.21.15.185:0/1027622766 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00bd20 con 0x7f9f640b72e0 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6159a700 1 -- 172.21.15.185:0/1027622766 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c005c60 con 0x7f9f640b72e0 2020-11-16T22:17:19.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- 172.21.15.185:0/1027622766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f640b72e0 msgr2=0x7f9f640b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1027622766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f640b72e0 0x7f9f640b76c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f9f64080a30 tx=0x7f9f5c0091a0).stop 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- 172.21.15.185:0/1027622766 shutdown_connections 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1027622766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f640b72e0 0x7f9f640b76c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1027622766 >> v2:172.21.15.96:3300/0 conn(0x7f9f640b64b0 0x7f9f640b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- 172.21.15.185:0/1027622766 >> 172.21.15.185:0/1027622766 conn(0x7f9f64067860 msgr2=0x7f9f64067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- 172.21.15.185:0/1027622766 shutdown_connections 2020-11-16T22:17:19.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- 172.21.15.185:0/1027622766 wait complete. 2020-11-16T22:17:19.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 Processor -- start 2020-11-16T22:17:19.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.179+0000 7f9f6359e700 1 -- start start 2020-11-16T22:17:19.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f640b64b0 0x7f9f64153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b09e0 con 0x7f9f640b64b0 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b1070 con 0x7f9f64113fd0 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64153fe0 con 0x7f9f640acee0 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59394/0 (socket says 172.21.15.185:59394) 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 -- 172.21.15.185:0/1904222484 learned_addr learned my addr 172.21.15.185:0/1904222484 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52836/0 (socket says 172.21.15.185:52836) 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f61d9b700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f640b64b0 0x7f9f64153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 -- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 msgr2=0x7f9f64113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 -- 172.21.15.185:0/1904222484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f640b64b0 msgr2=0x7f9f64153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f640b64b0 0x7f9f64153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 -- 172.21.15.185:0/1904222484 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64154280 con 0x7f9f64113fd0 2020-11-16T22:17:19.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6259c700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:19.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f62d9d700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9f5400c6e0 tx=0x7f9f54007ba0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540072e0 con 0x7f9f64113fd0 2020-11-16T22:17:19.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54007440 con 0x7f9f64113fd0 2020-11-16T22:17:19.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f640b0bd0 con 0x7f9f64113fd0 2020-11-16T22:17:19.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54005820 con 0x7f9f64113fd0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f64141480 con 0x7f9f64113fd0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9f5401a060 con 0x7f9f64113fd0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f5a7fc700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 0x7f9f40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.183+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f640b0d60 con 0x7f9f64113fd0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.187+0000 7f9f6259c700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 0x7f9f40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.187+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f5406d340 con 0x7f9f64113fd0 2020-11-16T22:17:19.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.187+0000 7f9f6259c700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 0x7f9f40057520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f9f64068850 tx=0x7f9f5c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.191+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9f54046fd0 con 0x7f9f64113fd0 2020-11-16T22:17:19.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.391+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f6405d200 con 0x7f9f64113fd0 2020-11-16T22:17:19.376 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:19.376 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:19.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.391+0000 7f9f5a7fc700 1 -- 172.21.15.185:0/1904222484 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f9f54018070 con 0x7f9f64113fd0 2020-11-16T22:17:19.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 msgr2=0x7f9f40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 0x7f9f40057520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f9f64068850 tx=0x7f9f5c015000).stop 2020-11-16T22:17:19.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 msgr2=0x7f9f64153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9f5400c6e0 tx=0x7f9f54007ba0).stop 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 shutdown_connections 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f40055060 0x7f9f40057520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f64113fd0 0x7f9f64153970 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f640acee0 0x7f9f64113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 --2- 172.21.15.185:0/1904222484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f640b64b0 0x7f9f64153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 >> 172.21.15.185:0/1904222484 conn(0x7f9f64067860 msgr2=0x7f9f640b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 shutdown_connections 2020-11-16T22:17:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.395+0000 7f9f6359e700 1 -- 172.21.15.185:0/1904222484 wait complete. 2020-11-16T22:17:19.380 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:19.391 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:17:19.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 Processor -- start 2020-11-16T22:17:19.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 -- start start 2020-11-16T22:17:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b5d70 0x7fc2900ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc2900b4470 0x7fc2900b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc2901475d0 0x7fc290147a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2900b7290 con 0x7fc2901475d0 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2900b4bf0 con 0x7fc2900b4470 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900b74c0 con 0x7fc2900b5d70 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc2900b4470 0x7fc2900b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc2900b4470 0x7fc2900b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59398/0 (socket says 172.21.15.185:59398) 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 -- 172.21.15.185:0/265362380 learned_addr learned my addr 172.21.15.185:0/265362380 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28e59c700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b5d70 0x7fc2900ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28ed9d700 1 --2- 172.21.15.185:0/265362380 >> v2:172.21.15.96:3300/0 conn(0x7fc2901475d0 0x7fc290147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 -- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b5d70 msgr2=0x7fc2900ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b5d70 0x7fc2900ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 -- 172.21.15.185:0/265362380 >> v2:172.21.15.96:3300/0 conn(0x7fc2901475d0 msgr2=0x7fc290147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 --2- 172.21.15.185:0/265362380 >> v2:172.21.15.96:3300/0 conn(0x7fc2901475d0 0x7fc290147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 -- 172.21.15.185:0/265362380 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900b7760 con 0x7fc2900b4470 2020-11-16T22:17:19.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc287fff700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b4470 0x7fc2900b7c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fc2900b83b0 tx=0x7fc278006eb0).ready entity=mon.1 client_cookie=f3d54d71a24d438e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28dd9b700 1 -- 172.21.15.185:0/265362380 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278009810 con 0x7fc2900b4470 2020-11-16T22:17:19.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28dd9b700 1 -- 172.21.15.185:0/265362380 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278009970 con 0x7fc2900b4470 2020-11-16T22:17:19.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.535+0000 7fc28dd9b700 1 -- 172.21.15.185:0/265362380 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278009cf0 con 0x7fc2900b4470 2020-11-16T22:17:19.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b4470 msgr2=0x7fc2900b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b4470 0x7fc2900b7c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fc2900b83b0 tx=0x7fc278006eb0).stop 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/265362380 shutdown_connections 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- 172.21.15.185:0/265362380 >> v2:172.21.15.96:3300/0 conn(0x7fc2901475d0 0x7fc290147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b5d70 0x7fc2900ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- 172.21.15.185:0/265362380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b4470 0x7fc2900b7c00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/265362380 >> 172.21.15.185:0/265362380 conn(0x7fc29005ba80 msgr2=0x7fc29005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/265362380 shutdown_connections 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/265362380 wait complete. 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 Processor -- start 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- start start 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 0x7fc290144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2901475d0 0x7fc29013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29013fb80 con 0x7fc2901475d0 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900af540 con 0x7fc2900b5d70 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900af7e0 con 0x7fc2900b4470 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 0x7fc290144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59406/0 (socket says 172.21.15.185:59406) 2020-11-16T22:17:19.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 -- 172.21.15.185:0/3994994391 learned_addr learned my addr 172.21.15.185:0/3994994391 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 0x7fc290144560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52848/0 (socket says 172.21.15.185:52848) 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28ed9d700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2901475d0 0x7fc29013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 -- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 msgr2=0x7fc290144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 0x7fc290144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 -- 172.21.15.185:0/3994994391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2901475d0 msgr2=0x7fc29013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2901475d0 0x7fc29013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 -- 172.21.15.185:0/3994994391 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900afa80 con 0x7fc2900b5d70 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc287fff700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc2900b4d30 tx=0x7fc278015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc2867fc700 1 -- 172.21.15.185:0/3994994391 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278011c80 con 0x7fc2900b5d70 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc2867fc700 1 -- 172.21.15.185:0/3994994391 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278011de0 con 0x7fc2900b5d70 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900b2370 con 0x7fc2900b5d70 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc2867fc700 1 -- 172.21.15.185:0/3994994391 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27801e6b0 con 0x7fc2900b5d70 2020-11-16T22:17:19.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.539+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2900b09c0 con 0x7fc2900b5d70 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.543+0000 7fc2867fc700 1 -- 172.21.15.185:0/3994994391 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc278025030 con 0x7fc2900b5d70 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.543+0000 7fc2867fc700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 0x7fc26c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.543+0000 7fc2867fc700 1 -- 172.21.15.185:0/3994994391 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc27806f140 con 0x7fc2900b5d70 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.543+0000 7fc28e59c700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 0x7fc26c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.543+0000 7fc28e59c700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 0x7fc26c057520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc280006fd0 tx=0x7fc280006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 Processor -- start 2020-11-16T22:17:19.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- start start 2020-11-16T22:17:19.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x257be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x257dda0 0x257c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24094b0 con 0x22f1bb0 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24bd930 con 0x257dda0 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x237d6f0 con 0x235f7b0 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52856/0 (socket says 172.21.15.185:52856) 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x257dda0 0x257c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 -- 172.21.15.185:0/2336424523 learned_addr learned my addr 172.21.15.185:0/2336424523 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc287fff700 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x257be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 -- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x257dda0 msgr2=0x257c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x257dda0 0x257c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 -- 172.21.15.185:0/2336424523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 msgr2=0x257be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x257be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 -- 172.21.15.185:0/2336424523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23785a0 con 0x235f7b0 2020-11-16T22:17:19.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fc280014e80 tx=0x7fc280013000).ready entity=mon.2 client_cookie=bf572e676bf531e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2677fe700 1 -- 172.21.15.185:0/2336424523 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280011b20 con 0x235f7b0 2020-11-16T22:17:19.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2677fe700 1 -- 172.21.15.185:0/2336424523 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280011c80 con 0x235f7b0 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2677fe700 1 -- 172.21.15.185:0/2336424523 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280020410 con 0x235f7b0 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 msgr2=0x257d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fc280014e80 tx=0x7fc280013000).stop 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- 172.21.15.185:0/2336424523 shutdown_connections 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x257dda0 0x257c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257d8b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2336424523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x257be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- 172.21.15.185:0/2336424523 >> 172.21.15.185:0/2336424523 conn(0x252de40 msgr2=0x257add0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- 172.21.15.185:0/2336424523 shutdown_connections 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.547+0000 7fc2a342d740 1 -- 172.21.15.185:0/2336424523 wait complete. 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 -- start start 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 Processor -- start 2020-11-16T22:17:19.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2580d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x257dda0 0x257e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24aa6e0 con 0x257dda0 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24aa820 con 0x22f1bb0 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24aba90 con 0x235f7b0 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2580d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52864/0 (socket says 172.21.15.185:52864) 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 -- 172.21.15.185:0/2092897153 learned_addr learned my addr 172.21.15.185:0/2092897153 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc2a342d740 10 client.0 ll_register_callbacks cb 0x24a0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc28ed9d700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x257dda0 0x257e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 -- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 msgr2=0x2580d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2580d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 -- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x257dda0 msgr2=0x257e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x257dda0 0x257e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 -- 172.21.15.185:0/2092897153 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x249b280 con 0x235f7b0 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc287fff700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fc278048e90 tx=0x7fc27802a040).ready entity=mon.2 client_cookie=4e35394dd7b6d15e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.551+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780480f0 con 0x235f7b0 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc278048250 con 0x235f7b0 2020-11-16T22:17:19.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x255e6c0 con 0x235f7b0 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27801eaa0 con 0x235f7b0 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 10 client.5570 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x22d0fd0 con 0x235f7b0 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x255e6c0 con 0x235f7b0 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fc27801ec00 con 0x235f7b0 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 10 client.5570 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:19.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 10 client.5570 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 20 client.5570 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x2581220 0x25ac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 4 client.5570 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x239bc50 con 0x2581220 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x2581220 0x25ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 10 client.5570 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:19.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x2581220 0x25ac030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7c0e20419698e093 server_cookie=b45f0d87c7694785 in_seq=0 out_seq=0 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 1 -- 172.21.15.185:0/2092897153 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x239bc50 con 0x2581220 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 10 client.5570 handle_command_reply: tid=0 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 client.5570 shutdown 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 msgr2=0x257e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fc278048e90 tx=0x7fc27802a040).stop 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 shutdown_connections 2020-11-16T22:17:19.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x2581220 0x25ac030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2580d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x257dda0 0x257e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2092897153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x235f7b0 0x257e480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc264ff9700 0 client.5570 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 >> 172.21.15.185:0/2092897153 conn(0x252de40 msgr2=0x257be20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 shutdown_connections 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 1 -- 172.21.15.185:0/2092897153 wait complete. 2020-11-16T22:17:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.555+0000 7fc2a342d740 20 client.5570 trim_cache size 0 max 16384 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 Processor -- start 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 -- start start 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2572bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x235f7b0 0x25b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2482e40 con 0x235f7b0 2020-11-16T22:17:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x257f7b0 con 0x22f1bb0 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x257b580 con 0x252de40 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2572bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52872/0 (socket says 172.21.15.185:52872) 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 -- 172.21.15.185:0/2222985405 learned_addr learned my addr 172.21.15.185:0/2222985405 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28ed9d700 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x235f7b0 0x25b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 -- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 msgr2=0x2572bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2572bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 -- 172.21.15.185:0/2222985405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x235f7b0 msgr2=0x25b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x235f7b0 0x25b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 -- 172.21.15.185:0/2222985405 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258008160 con 0x252de40 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc28e59c700 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fc280014e50 tx=0x7fc280017000).ready entity=mon.2 client_cookie=9a46519e941d30f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc267fff700 1 -- 172.21.15.185:0/2222985405 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280006960 con 0x252de40 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc267fff700 1 -- 172.21.15.185:0/2222985405 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280020410 con 0x252de40 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.575+0000 7fc267fff700 1 -- 172.21.15.185:0/2222985405 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28002d0d0 con 0x252de40 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 msgr2=0x25726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fc280014e50 tx=0x7fc280017000).stop 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/2222985405 shutdown_connections 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x235f7b0 0x25b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x252de40 0x25726c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- 172.21.15.185:0/2222985405 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22f1bb0 0x2572bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/2222985405 >> 172.21.15.185:0/2222985405 conn(0x7fc258002320 msgr2=0x257efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/2222985405 shutdown_connections 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/2222985405 wait complete. 2020-11-16T22:17:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- start start 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 Processor -- start 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x25ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x25c1cf0 con 0x22f1bb0 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x25b0650 con 0x235f7b0 2020-11-16T22:17:19.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x25ad210 con 0x257dda0 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 10 client.0 ll_register_callbacks cb 0x2577a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc28ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59436/0 (socket says 172.21.15.185:59436) 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc28ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52878/0 (socket says 172.21.15.185:52878) 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 -- 172.21.15.185:0/3775668912 learned_addr learned my addr 172.21.15.185:0/3775668912 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:19.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc28e59c700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x25ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 -- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 msgr2=0x25bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 -- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 msgr2=0x25ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x25ba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 -- 172.21.15.185:0/3775668912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x230f720 con 0x235f7b0 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc28ed9d700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:19.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc287fff700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc2780490c0 tx=0x7fc27802f000).ready entity=mon.1 client_cookie=79840859b0723ce8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278048cd0 con 0x235f7b0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc278041c60 con 0x235f7b0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc280011c80 con 0x235f7b0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27804a030 con 0x235f7b0 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 10 client.6034 fetch_fsmap learned FSMap version 41 2020-11-16T22:17:19.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x231ae00 con 0x235f7b0 2020-11-16T22:17:19.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280011c80 con 0x235f7b0 2020-11-16T22:17:20.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 41) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fc27801e890 con 0x235f7b0 2020-11-16T22:17:20.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 10 client.6034 fetch_fsmap finished waiting for FSMap version 41 2020-11-16T22:17:20.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 10 client.6034 resolve_mds: resolved ID 'b' to GID 5542 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 20 client.6034 populate_metadata read hostname 'smithi185' 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x25b0f30 0x26130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 4 client.6034 mds_command: new command op to 5542 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.579+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 --> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x23a2f50 con 0x25b0f30 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28e59c700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x25b0f30 0x26130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc264ff9700 10 client.6034 ms_handle_connect on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28e59c700 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x25b0f30 0x26130b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4d4307ff2279228f server_cookie=390aa94b71b75022 in_seq=0 out_seq=0 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc264ff9700 1 -- 172.21.15.185:0/3775668912 <== mds.0 v2:172.21.15.96:6832/653581747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x23a2f50 con 0x25b0f30 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc264ff9700 10 client.6034 handle_command_reply: tid=0 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 client.6034 shutdown 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 msgr2=0x25bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc2780490c0 tx=0x7fc27802f000).stop 2020-11-16T22:17:20.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 shutdown_connections 2020-11-16T22:17:20.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x257dda0 0x25bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22f1bb0 0x25ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.96:6832/653581747,v1:172.21.15.96:6834/653581747] conn(0x25b0f30 0x26130b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 --2- 172.21.15.185:0/3775668912 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x235f7b0 0x25bb1d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 >> 172.21.15.185:0/3775668912 conn(0x7fc258002320 msgr2=0x25c1370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc264ff9700 0 client.6034 ms_handle_reset on v2:172.21.15.96:6832/653581747 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 shutdown_connections 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 1 -- 172.21.15.185:0/3775668912 wait complete. 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc2a342d740 20 client.6034 trim_cache size 0 max 16384 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 msgr2=0x7fc26c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 0x7fc26c057520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc280006fd0 tx=0x7fc280006df0).stop 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 msgr2=0x7fc29013f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc2900b4d30 tx=0x7fc278015000).stop 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 shutdown_connections 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc26c055060 0x7fc26c057520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2901475d0 0x7fc29013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2900b4470 0x7fc290144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 --2- 172.21.15.185:0/3994994391 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2900b5d70 0x7fc29013f1a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 >> 172.21.15.185:0/3994994391 conn(0x7fc29005ba80 msgr2=0x7fc290146380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 shutdown_connections 2020-11-16T22:17:20.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:19.583+0000 7fc28f59e700 1 -- 172.21.15.185:0/3994994391 wait complete. 2020-11-16T22:17:20.176 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:20.177 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:17:20.177 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:17:20.177 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:20.177 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-11-16T22:17:20.178 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 Processor -- start 2020-11-16T22:17:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 -- start start 2020-11-16T22:17:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 0x7f6a7c0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c1444d0 con 0x7f6a7c09d260 2020-11-16T22:17:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c144770 con 0x7f6a7c0b6b30 2020-11-16T22:17:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.323+0000 7f6a81d9d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c144a10 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7bfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6a7c0b6b30 0x7f6a7c09a070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52882/0 (socket says 172.21.15.185:52882) 2020-11-16T22:17:20.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7bfff700 1 -- 172.21.15.185:0/3222745548 learned_addr learned my addr 172.21.15.185:0/3222745548 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:20.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 0x7f6a7c0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15848001 0 0) 0x7f6a7c144770 con 0x7f6a7c0b6b30 2020-11-16T22:17:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a5c003240 con 0x7f6a7c0b6b30 2020-11-16T22:17:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534506115 0 0) 0x7f6a7c1444d0 con 0x7f6a7c09d260 2020-11-16T22:17:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a7c144770 con 0x7f6a7c09d260 2020-11-16T22:17:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/3222745548 >> v1:172.21.15.185:6789/0 conn(0x7f6a7c0b6b30 legacy=0x7f6a7c09a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/3222745548 >> v1:172.21.15.96:6789/0 conn(0x7f6a7c09d260 legacy=0x7f6a7c1437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/3222745548 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c144cb0 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7b7fe700 1 --2- 172.21.15.185:0/3222745548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 0x7f6a7c0b5f10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f6a6400b9d0 tx=0x7f6a64006df0).ready entity=mon.2 client_cookie=c468f167f9e15a05 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64009680 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a640097e0 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a7a7fc700 1 -- 172.21.15.185:0/3222745548 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64009b60 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 -- 172.21.15.185:0/3222745548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 msgr2=0x7f6a7c0b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/3222745548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 0x7f6a7c0b5f10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f6a6400b9d0 tx=0x7f6a64006df0).stop 2020-11-16T22:17:20.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 -- 172.21.15.185:0/3222745548 shutdown_connections 2020-11-16T22:17:20.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/3222745548 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c0b5b30 0x7f6a7c0b5f10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 -- 172.21.15.185:0/3222745548 >> 172.21.15.185:0/3222745548 conn(0x7f6a7c068330 msgr2=0x7f6a7c0ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 -- 172.21.15.185:0/3222745548 shutdown_connections 2020-11-16T22:17:20.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.327+0000 7f6a81d9d700 1 -- 172.21.15.185:0/3222745548 wait complete. 2020-11-16T22:17:20.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 Processor -- start 2020-11-16T22:17:20.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 -- start start 2020-11-16T22:17:20.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 0x7f6a7c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c1499b0 0x7f6a7c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b1e00 con 0x7f6a7c0b6b30 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b20a0 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a81d9d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b2340 con 0x7f6a7c1499b0 2020-11-16T22:17:20.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a7b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a7b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59446/0 (socket says 172.21.15.185:59446) 2020-11-16T22:17:20.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/1750347552 learned_addr learned my addr 172.21.15.185:0/1750347552 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:20.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.331+0000 7f6a7bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c1499b0 0x7f6a7c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7affd700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 0x7f6a7c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c1499b0 msgr2=0x7f6a7c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c1499b0 0x7f6a7c153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/1750347552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 msgr2=0x7f6a7c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 0x7f6a7c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 -- 172.21.15.185:0/1750347552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0b25e0 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7affd700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 0x7f6a7c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a7b7fe700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6a6400b150 tx=0x7f6a64015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64011d70 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6401e460 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c09f870 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6401e7e0 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c14a120 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6a64025030 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.335+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c14a310 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.339+0000 7f6a737fe700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 0x7f6a58057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.339+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a6406f530 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.339+0000 7f6a7affd700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 0x7f6a58057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.339+0000 7f6a7affd700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 0x7f6a58057550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6a7c09de30 tx=0x7f6a6c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.343+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6a64045710 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.539+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a7c0b2770 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.539+0000 7f6a737fe700 1 -- 172.21.15.185:0/1750347552 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f6a64023020 con 0x7f6a7c0b5b30 2020-11-16T22:17:20.523 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:20.523 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:20.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 msgr2=0x7f6a58057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 0x7f6a58057550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f6a7c09de30 tx=0x7f6a6c009040).stop 2020-11-16T22:17:20.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 msgr2=0x7f6a7c09f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f6a6400b150 tx=0x7f6a64015000).stop 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 shutdown_connections 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6a58055090 0x7f6a58057550 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6a7c0b5b30 0x7f6a7c09f380 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6a7c1499b0 0x7f6a7c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 --2- 172.21.15.185:0/1750347552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a7c0b6b30 0x7f6a7c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 >> 172.21.15.185:0/1750347552 conn(0x7f6a7c068330 msgr2=0x7f6a7c142e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 shutdown_connections 2020-11-16T22:17:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.543+0000 7f6a81d9d700 1 -- 172.21.15.185:0/1750347552 wait complete. 2020-11-16T22:17:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:20.538 DEBUG:tasks.cephfs.test_multimds_misc:current active=['b', 'c', 'a'] 2020-11-16T22:17:20.538 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:17:20.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 Processor -- start 2020-11-16T22:17:20.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 -- start start 2020-11-16T22:17:20.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f81040b1fa0 0x7f8104147970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81040b2a30 con 0x7f81040b0f70 2020-11-16T22:17:20.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81040b24f0 con 0x7f81040680a0 2020-11-16T22:17:20.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8103fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8104148570 con 0x7f81040b1fa0 2020-11-16T22:17:20.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f81040b1fa0 0x7f8104147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f81040b1fa0 0x7f8104147970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59454/0 (socket says 172.21.15.185:59454) 2020-11-16T22:17:20.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 -- 172.21.15.185:0/3942219726 learned_addr learned my addr 172.21.15.185:0/3942219726 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:20.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8101ffb700 1 -- 172.21.15.185:0/3942219726 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896930244 0 0) 0x7f81040b24f0 con 0x7f81040680a0 2020-11-16T22:17:20.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 -- 172.21.15.185:0/3942219726 >> v1:172.21.15.185:6789/0 conn(0x7f81040680a0 legacy=0x7f810414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 -- 172.21.15.185:0/3942219726 >> v1:172.21.15.96:6789/0 conn(0x7f81040b0f70 legacy=0x7f8104147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 -- 172.21.15.185:0/3942219726 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040b2790 con 0x7f81040b1fa0 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f81027fc700 1 --2- 172.21.15.185:0/3942219726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040b1fa0 0x7f8104147970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f80fc009d50 tx=0x7f80fc009290).ready entity=mon.1 client_cookie=5c6c17070708a26c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8101ffb700 1 -- 172.21.15.185:0/3942219726 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00bc80 con 0x7f81040b1fa0 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8101ffb700 1 -- 172.21.15.185:0/3942219726 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc00bde0 con 0x7f81040b1fa0 2020-11-16T22:17:20.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.675+0000 7f8101ffb700 1 -- 172.21.15.185:0/3942219726 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc011610 con 0x7f81040b1fa0 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3942219726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040b1fa0 msgr2=0x7f8104147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 --2- 172.21.15.185:0/3942219726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040b1fa0 0x7f8104147970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f80fc009d50 tx=0x7f80fc009290).stop 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3942219726 shutdown_connections 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 --2- 172.21.15.185:0/3942219726 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040b1fa0 0x7f8104147970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3942219726 >> 172.21.15.185:0/3942219726 conn(0x7f81040657c0 msgr2=0x7f8104065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3942219726 shutdown_connections 2020-11-16T22:17:20.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3942219726 wait complete. 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 Processor -- start 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- start start 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81040b0f70 0x7f81040ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81040b1fa0 0x7f81040ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81040ae770 con 0x7f81040b1fa0 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81040af7d0 con 0x7f81040680a0 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81040abe50 con 0x7f81040b0f70 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59460/0 (socket says 172.21.15.185:59460) 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 -- 172.21.15.185:0/3934918223 learned_addr learned my addr 172.21.15.185:0/3934918223 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f81027fc700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81040b0f70 0x7f81040ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f81037fe700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81040b1fa0 0x7f81040ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 -- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81040b0f70 msgr2=0x7f81040ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81040b0f70 0x7f81040ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 -- 172.21.15.185:0/3934918223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81040b1fa0 msgr2=0x7f81040ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81040b1fa0 0x7f81040ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 -- 172.21.15.185:0/3934918223 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040afa00 con 0x7f81040680a0 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8102ffd700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f80f800c730 tx=0x7f80f8007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800e560 con 0x7f81040680a0 2020-11-16T22:17:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81040afca0 con 0x7f81040680a0 2020-11-16T22:17:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800e6c0 con 0x7f81040680a0 2020-11-16T22:17:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800ea80 con 0x7f81040680a0 2020-11-16T22:17:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.679+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810414b220 con 0x7f81040680a0 2020-11-16T22:17:20.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810405d200 con 0x7f81040680a0 2020-11-16T22:17:20.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f80f8019030 con 0x7f81040680a0 2020-11-16T22:17:20.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f80eaffd700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 0x7f80d4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:20.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f81027fc700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 0x7f80d4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:20.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f80f806d980 con 0x7f81040680a0 2020-11-16T22:17:20.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.683+0000 7f81027fc700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 0x7f80d4057550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f80fc008000 tx=0x7f80fc016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:20.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.691+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f80f8043c00 con 0x7f81040680a0 2020-11-16T22:17:20.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.895+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8104061140 con 0x7f81040680a0 2020-11-16T22:17:20.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.895+0000 7f80eaffd700 1 -- 172.21.15.185:0/3934918223 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1689 (secure 0 0 0) 0x7f80f8047220 con 0x7f81040680a0 2020-11-16T22:17:20.880 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:20.880 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":213,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":112410,"bytes_used":48472064,"bytes_avail":773045641216,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":41,"id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"a","status":"up:active","gid":5183},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5475},{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5542}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2020-11-16T22:17:19.103466+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:17:20.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 msgr2=0x7f80d4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 0x7f80d4057550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f80fc008000 tx=0x7f80fc016000).stop 2020-11-16T22:17:20.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 msgr2=0x7f810414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f80f800c730 tx=0x7f80f8007a70).stop 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 shutdown_connections 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f80d4055090 0x7f80d4057550 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f81040680a0 0x7f810414ac70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81040b1fa0 0x7f81040ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 --2- 172.21.15.185:0/3934918223 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f81040b0f70 0x7f81040ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:20.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 >> 172.21.15.185:0/3934918223 conn(0x7f81040657c0 msgr2=0x7f8104147740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:20.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 shutdown_connections 2020-11-16T22:17:20.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:20.899+0000 7f8103fff700 1 -- 172.21.15.185:0/3934918223 wait complete. 2020-11-16T22:17:20.894 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:17:20.894 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:17:20.895 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:20.932 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:20.948 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:20.964 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:17:20.965 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:17:20.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:20.965 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:17:21.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:17:21.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:21.058 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:17:21.077 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:17:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.231+0000 7f7950bba700 1 Processor -- start 2020-11-16T22:17:21.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.231+0000 7f7950bba700 1 -- start start 2020-11-16T22:17:21.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f794c0b38a0 0x7f794c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f794c05bc50 0x7f794c144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c0ad350 con 0x7f794c05bc50 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c1451d0 con 0x7f794c0b38a0 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0b2690 con 0x7f794c144d60 2020-11-16T22:17:21.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f794c0b38a0 0x7f794c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f794c0b38a0 0x7f794c0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59468/0 (socket says 172.21.15.185:59468) 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c13fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52916/0 (socket says 172.21.15.185:52916) 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 -- 172.21.15.185:0/1658905295 learned_addr learned my addr 172.21.15.185:0/1658905295 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794affd700 1 --2- 172.21.15.185:0/1658905295 >> v2:172.21.15.96:3300/0 conn(0x7f794c05bc50 0x7f794c144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 -- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 msgr2=0x7f794c13fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:21.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 --2- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c13fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 -- 172.21.15.185:0/1658905295 >> v2:172.21.15.96:3300/0 conn(0x7f794c05bc50 msgr2=0x7f794c144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:21.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 --2- 172.21.15.185:0/1658905295 >> v2:172.21.15.96:3300/0 conn(0x7f794c05bc50 0x7f794c144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 -- 172.21.15.185:0/1658905295 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0b2930 con 0x7f794c0b38a0 2020-11-16T22:17:21.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794b7fe700 1 --2- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c0b38a0 0x7f794c0b2060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f793c006fd0 tx=0x7f793c006eb0).ready entity=mon.1 client_cookie=2d6844b0fb4b200a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794a7fc700 1 -- 172.21.15.185:0/1658905295 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0096e0 con 0x7f794c0b38a0 2020-11-16T22:17:21.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794a7fc700 1 -- 172.21.15.185:0/1658905295 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c009840 con 0x7f794c0b38a0 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f794a7fc700 1 -- 172.21.15.185:0/1658905295 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c009bc0 con 0x7f794c0b38a0 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c0b38a0 msgr2=0x7f794c0b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c0b38a0 0x7f794c0b2060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f793c006fd0 tx=0x7f793c006eb0).stop 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- 172.21.15.185:0/1658905295 shutdown_connections 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- 172.21.15.185:0/1658905295 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c0b38a0 0x7f794c0b2060 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 --2- 172.21.15.185:0/1658905295 >> v2:172.21.15.96:3300/0 conn(0x7f794c05bc50 0x7f794c144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- 172.21.15.185:0/1658905295 >> 172.21.15.185:0/1658905295 conn(0x7f794c063df0 msgr2=0x7f794c0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- 172.21.15.185:0/1658905295 shutdown_connections 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.235+0000 7f7950bba700 1 -- 172.21.15.185:0/1658905295 wait complete. 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 Processor -- start 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- start start 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 0x7f794c13b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c066c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c139c50 con 0x7f794c0b38a0 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1399b0 con 0x7f794c05bc50 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c156de0 con 0x7f794c144d60 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59474/0 (socket says 172.21.15.185:59474) 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 -- 172.21.15.185:0/2928026142 learned_addr learned my addr 172.21.15.185:0/2928026142 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794affd700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 0x7f794c13b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 -- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 msgr2=0x7f794c066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 -- 172.21.15.185:0/2928026142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 msgr2=0x7f794c13b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 0x7f794c13b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 -- 172.21.15.185:0/2928026142 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c157080 con 0x7f794c05bc50 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794affd700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 0x7f794c13b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f794b7fe700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f793c009810 tx=0x7f793c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c011bc0 con 0x7f794c05bc50 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c011d20 con 0x7f794c05bc50 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c141a80 con 0x7f794c05bc50 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c01e670 con 0x7f794c05bc50 2020-11-16T22:17:21.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c13b970 con 0x7f794c05bc50 2020-11-16T22:17:21.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.239+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c13bb90 con 0x7f794c05bc50 2020-11-16T22:17:21.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.243+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f793c025030 con 0x7f794c05bc50 2020-11-16T22:17:21.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.243+0000 7f792b7fe700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 0x7f7924057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.243+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f793c06f400 con 0x7f794c05bc50 2020-11-16T22:17:21.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.243+0000 7f794affd700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 0x7f7924057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.243+0000 7f794affd700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 0x7f7924057520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f794c1414a0 tx=0x7f7940008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.251+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f793c045710 con 0x7f794c05bc50 2020-11-16T22:17:21.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.403+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f794c05d200 con 0x7f794c05bc50 2020-11-16T22:17:21.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f792b7fe700 1 -- 172.21.15.185:0/2928026142 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f793c023020 con 0x7f794c05bc50 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 msgr2=0x7f7924057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 0x7f7924057520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f794c1414a0 tx=0x7f7940008040).stop 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 msgr2=0x7f794c157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f793c009810 tx=0x7f793c015000).stop 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 shutdown_connections 2020-11-16T22:17:21.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7924055060 0x7f7924057520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f794c05bc50 0x7f794c157520 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f794c144d60 0x7f794c066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 --2- 172.21.15.185:0/2928026142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f794c0b38a0 0x7f794c13b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 >> 172.21.15.185:0/2928026142 conn(0x7f794c063df0 msgr2=0x7f794c13a140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 shutdown_connections 2020-11-16T22:17:21.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.411+0000 7f7950bba700 1 -- 172.21.15.185:0/2928026142 wait complete. 2020-11-16T22:17:21.410 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2020-11-16T22:17:21.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 Processor -- start 2020-11-16T22:17:21.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 -- start start 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f740b64b0 0x7f8f740b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f740b6aa0 0x7f8f740acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f74147f90 con 0x7f8f740b72e0 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740b6eb0 con 0x7f8f740b64b0 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f7a529700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740af020 con 0x7f8f740b6aa0 2020-11-16T22:17:21.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f740b6aa0 0x7f8f740acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8f740b6aa0 0x7f8f740acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59486/0 (socket says 172.21.15.185:59486) 2020-11-16T22:17:21.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 -- 172.21.15.185:0/2351555453 learned_addr learned my addr 172.21.15.185:0/2351555453 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f78d26700 1 --2- 172.21.15.185:0/2351555453 >> v2:172.21.15.96:3300/0 conn(0x7f8f740b64b0 0x7f8f740b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 -- 172.21.15.185:0/2351555453 >> v2:172.21.15.96:3300/0 conn(0x7f8f740b64b0 msgr2=0x7f8f740b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 --2- 172.21.15.185:0/2351555453 >> v2:172.21.15.96:3300/0 conn(0x7f8f740b64b0 0x7f8f740b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 -- 172.21.15.185:0/2351555453 >> v1:172.21.15.96:6789/0 conn(0x7f8f740b72e0 legacy=0x7f8f74144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.551+0000 7f8f79d28700 1 -- 172.21.15.185:0/2351555453 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74148460 con 0x7f8f740b6aa0 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f79d28700 1 --2- 172.21.15.185:0/2351555453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b6aa0 0x7f8f740acee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f8f740b5170 tx=0x7f8f6c009290).ready entity=mon.1 client_cookie=fbc95047cc49e138 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f6bfff700 1 -- 172.21.15.185:0/2351555453 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00bbc0 con 0x7f8f740b6aa0 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f6bfff700 1 -- 172.21.15.185:0/2351555453 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00bd20 con 0x7f8f740b6aa0 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f6bfff700 1 -- 172.21.15.185:0/2351555453 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c011610 con 0x7f8f740b6aa0 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- 172.21.15.185:0/2351555453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b6aa0 msgr2=0x7f8f740acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 --2- 172.21.15.185:0/2351555453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b6aa0 0x7f8f740acee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f8f740b5170 tx=0x7f8f6c009290).stop 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- 172.21.15.185:0/2351555453 shutdown_connections 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 --2- 172.21.15.185:0/2351555453 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b6aa0 0x7f8f740acee0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 --2- 172.21.15.185:0/2351555453 >> v2:172.21.15.96:3300/0 conn(0x7f8f740b64b0 0x7f8f740b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- 172.21.15.185:0/2351555453 >> 172.21.15.185:0/2351555453 conn(0x7f8f74067860 msgr2=0x7f8f74067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- 172.21.15.185:0/2351555453 shutdown_connections 2020-11-16T22:17:21.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- 172.21.15.185:0/2351555453 wait complete. 2020-11-16T22:17:21.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 Processor -- start 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.555+0000 7f8f7a529700 1 -- start start 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f740b72e0 0x7f8f740b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740b39e0 con 0x7f8f740b72e0 2020-11-16T22:17:21.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7413f040 con 0x7f8f740b64b0 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7413f2e0 con 0x7f8f740b6aa0 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f78d26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59488/0 (socket says 172.21.15.185:59488) 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f78d26700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52934/0 (socket says 172.21.15.185:52934) 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 -- 172.21.15.185:0/836385787 learned_addr learned my addr 172.21.15.185:0/836385787 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:21.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79d28700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f740b72e0 0x7f8f740b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 -- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 msgr2=0x7f8f740b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 -- 172.21.15.185:0/836385787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f740b72e0 msgr2=0x7f8f740b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f740b72e0 0x7f8f740b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 -- 172.21.15.185:0/836385787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740b0990 con 0x7f8f740b64b0 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f78d26700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:21.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f79527700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f8f6400c690 tx=0x7f8f64007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64007020 con 0x7f8f740b64b0 2020-11-16T22:17:21.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f64007180 con 0x7f8f740b64b0 2020-11-16T22:17:21.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64006540 con 0x7f8f740b64b0 2020-11-16T22:17:21.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740b0540 con 0x7f8f740b64b0 2020-11-16T22:17:21.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f74153da0 con 0x7f8f740b64b0 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7413f4a0 con 0x7f8f740b64b0 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.559+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8f64010070 con 0x7f8f740b64b0 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.563+0000 7f8f697fa700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 0x7f8f4c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.563+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8f6406d0b0 con 0x7f8f740b64b0 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.563+0000 7f8f78d26700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 0x7f8f4c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:21.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.563+0000 7f8f78d26700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 0x7f8f4c057520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f8f740b8d00 tx=0x7f8f70006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:21.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.567+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8f64043310 con 0x7f8f740b64b0 2020-11-16T22:17:21.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:21.779+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) v1 -- 0x7f8f7405d200 con 0x7f8f740b64b0 2020-11-16T22:17:21.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f697fa700 1 -- 172.21.15.185:0/836385787 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v176) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8f64046930 con 0x7f8f740b64b0 2020-11-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 msgr2=0x7f8f4c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 0x7f8f4c057520 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f8f740b8d00 tx=0x7f8f70006df0).stop 2020-11-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 msgr2=0x7f8f740b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f8f6400c690 tx=0x7f8f64007c40).stop 2020-11-16T22:17:22.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 shutdown_connections 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f4c055060 0x7f8f4c057520 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8f740b64b0 0x7f8f740b0050 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f740b72e0 0x7f8f740b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 --2- 172.21.15.185:0/836385787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8f740b6aa0 0x7f8f740b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 >> 172.21.15.185:0/836385787 conn(0x7f8f74067860 msgr2=0x7f8f741446e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 shutdown_connections 2020-11-16T22:17:22.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.015+0000 7f8f7a529700 1 -- 172.21.15.185:0/836385787 wait complete. 2020-11-16T22:17:22.017 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:17:22.018 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:17:22.018 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2020-11-16T22:17:22.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 Processor -- start 2020-11-16T22:17:22.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 -- start start 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbdbc0acc10 0x7fbdbc1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0abf90 con 0x7fbdbc0669a0 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0ac230 con 0x7fbdbc061e60 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.159+0000 7fbdbbfff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc0ac4d0 con 0x7fbdbc0acc10 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbdbc0acc10 0x7fbdbc1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:22.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbdbc0acc10 0x7fbdbc1432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59498/0 (socket says 172.21.15.185:59498) 2020-11-16T22:17:22.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbb7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbdbc061e60 0x7fbdbc1437d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52938/0 (socket says 172.21.15.185:52938) 2020-11-16T22:17:22.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 -- 172.21.15.185:0/3800100420 learned_addr learned my addr 172.21.15.185:0/3800100420 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:22.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 -- 172.21.15.185:0/3800100420 >> v1:172.21.15.185:6789/0 conn(0x7fbdbc061e60 legacy=0x7fbdbc1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 -- 172.21.15.185:0/3800100420 >> v1:172.21.15.96:6789/0 conn(0x7fbdbc0669a0 legacy=0x7fbdbc066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 -- 172.21.15.185:0/3800100420 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0ac770 con 0x7fbdbc0acc10 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdba7fc700 1 --2- 172.21.15.185:0/3800100420 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc0acc10 0x7fbdbc1432e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fbdac009d50 tx=0x7fbdac009290).ready entity=mon.1 client_cookie=9035fb354f295d6f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdb9ffb700 1 -- 172.21.15.185:0/3800100420 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00bc80 con 0x7fbdbc0acc10 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdb9ffb700 1 -- 172.21.15.185:0/3800100420 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00bde0 con 0x7fbdbc0acc10 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdb9ffb700 1 -- 172.21.15.185:0/3800100420 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac011610 con 0x7fbdbc0acc10 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 -- 172.21.15.185:0/3800100420 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc0acc10 msgr2=0x7fbdbc1432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/3800100420 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc0acc10 0x7fbdbc1432e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fbdac009d50 tx=0x7fbdac009290).stop 2020-11-16T22:17:22.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 -- 172.21.15.185:0/3800100420 shutdown_connections 2020-11-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/3800100420 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc0acc10 0x7fbdbc1432e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 -- 172.21.15.185:0/3800100420 >> 172.21.15.185:0/3800100420 conn(0x7fbdbc060130 msgr2=0x7fbdbc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 -- 172.21.15.185:0/3800100420 shutdown_connections 2020-11-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.163+0000 7fbdbbfff700 1 -- 172.21.15.185:0/3800100420 wait complete. 2020-11-16T22:17:22.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 Processor -- start 2020-11-16T22:17:22.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 -- start start 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 0x7fbdbc1464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbdbc146e70 0x7fbdbc0ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0ae9a0 con 0x7fbdbc146e70 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc141f40 con 0x7fbdbc061e60 2020-11-16T22:17:22.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbbfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b0300 con 0x7fbdbc0669a0 2020-11-16T22:17:22.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:22.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59502/0 (socket says 172.21.15.185:59502) 2020-11-16T22:17:22.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 -- 172.21.15.185:0/392427975 learned_addr learned my addr 172.21.15.185:0/392427975 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:22.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdba7fc700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 0x7fbdbc1464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:22.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbb7fe700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbdbc146e70 0x7fbdbc0ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:22.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 -- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 msgr2=0x7fbdbc1464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 0x7fbdbc1464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 -- 172.21.15.185:0/392427975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbdbc146e70 msgr2=0x7fbdbc0ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbdbc146e70 0x7fbdbc0ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 -- 172.21.15.185:0/392427975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0b05a0 con 0x7fbdbc061e60 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdba7fc700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 0x7fbdbc1464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdbaffd700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fbdb400c6e0 tx=0x7fbdb4007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:22.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4007020 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.167+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4007180 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc0b0840 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40064f0 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc1469e0 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc05d200 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbdb4010070 con 0x7fbdbc061e60 2020-11-16T22:17:22.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdb2ffd700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 0x7fbd8c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:22.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbdb406d040 con 0x7fbdbc061e60 2020-11-16T22:17:22.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdba7fc700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 0x7fbd8c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:22.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.171+0000 7fbdba7fc700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 0x7fbd8c057520 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fbdbc1462e0 tx=0x7fbdac016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:22.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.179+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbdb40432a0 con 0x7fbdbc061e60 2020-11-16T22:17:22.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:22.383+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7fbdbc0b3970 con 0x7fbdbc061e60 2020-11-16T22:17:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.015+0000 7fbdb2ffd700 1 -- 172.21.15.185:0/392427975 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v177) v1 ==== 145+0+0 (secure 0 0 0) 0x7fbdb40468c0 con 0x7fbdbc061e60 2020-11-16T22:17:23.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 msgr2=0x7fbd8c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 0x7fbd8c057520 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fbdbc1462e0 tx=0x7fbdac016000).stop 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 msgr2=0x7fbdbc0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fbdb400c6e0 tx=0x7fbdb4007c50).stop 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 shutdown_connections 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd8c055060 0x7fbd8c057520 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbdbc061e60 0x7fbdbc0afbf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbdbc146e70 0x7fbdbc0ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 --2- 172.21.15.185:0/392427975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbdbc0669a0 0x7fbdbc1464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 >> 172.21.15.185:0/392427975 conn(0x7fbdbc060130 msgr2=0x7fbdbc1430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 shutdown_connections 2020-11-16T22:17:23.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.019+0000 7fbdbbfff700 1 -- 172.21.15.185:0/392427975 wait complete. 2020-11-16T22:17:23.020 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:17:23.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 Processor -- start 2020-11-16T22:17:23.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- start start 2020-11-16T22:17:23.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2540b1a20 0x7fc2541411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc254147fb0 0x7fc2541416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc25405ba80 0x7fc2541443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540b2f30 con 0x7fc25405ba80 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc254143f90 con 0x7fc254147fb0 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc254060fe0 con 0x7fc2540b1a20 2020-11-16T22:17:23.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc254147fb0 0x7fc2541416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc254147fb0 0x7fc2541416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59510/0 (socket says 172.21.15.185:59510) 2020-11-16T22:17:23.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 -- 172.21.15.185:0/1502310212 learned_addr learned my addr 172.21.15.185:0/1502310212 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25259c700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2540b1a20 0x7fc2541411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc252d9d700 1 --2- 172.21.15.185:0/1502310212 >> v2:172.21.15.96:3300/0 conn(0x7fc25405ba80 0x7fc2541443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 -- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2540b1a20 msgr2=0x7fc2541411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:23.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2540b1a20 0x7fc2541411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 -- 172.21.15.185:0/1502310212 >> v2:172.21.15.96:3300/0 conn(0x7fc25405ba80 msgr2=0x7fc2541443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 --2- 172.21.15.185:0/1502310212 >> v2:172.21.15.96:3300/0 conn(0x7fc25405ba80 0x7fc2541443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 -- 172.21.15.185:0/1502310212 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254061280 con 0x7fc254147fb0 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc251d9b700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc254147fb0 0x7fc2541416a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fc2540659f0 tx=0x7fc23c006eb0).ready entity=mon.1 client_cookie=5c39682108e261bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25159a700 1 -- 172.21.15.185:0/1502310212 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0097b0 con 0x7fc254147fb0 2020-11-16T22:17:23.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25159a700 1 -- 172.21.15.185:0/1502310212 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c009910 con 0x7fc254147fb0 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25159a700 1 -- 172.21.15.185:0/1502310212 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c009c90 con 0x7fc254147fb0 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc254147fb0 msgr2=0x7fc2541416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc254147fb0 0x7fc2541416a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fc2540659f0 tx=0x7fc23c006eb0).stop 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- 172.21.15.185:0/1502310212 shutdown_connections 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- 172.21.15.185:0/1502310212 >> v2:172.21.15.96:3300/0 conn(0x7fc25405ba80 0x7fc2541443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2540b1a20 0x7fc2541411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 --2- 172.21.15.185:0/1502310212 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc254147fb0 0x7fc2541416a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- 172.21.15.185:0/1502310212 >> 172.21.15.185:0/1502310212 conn(0x7fc254063df0 msgr2=0x7fc2540641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.175+0000 7fc25359e700 1 -- 172.21.15.185:0/1502310212 shutdown_connections 2020-11-16T22:17:23.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 -- 172.21.15.185:0/1502310212 wait complete. 2020-11-16T22:17:23.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 Processor -- start 2020-11-16T22:17:23.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 -- start start 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 0x7fc254145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc254147fb0 0x7fc254145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc254140870 con 0x7fc2540b1a20 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25413fd40 con 0x7fc25405ba80 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25413ffe0 con 0x7fc254147fb0 2020-11-16T22:17:23.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25259c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59516/0 (socket says 172.21.15.185:59516) 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25259c700 1 -- 172.21.15.185:0/995337478 learned_addr learned my addr 172.21.15.185:0/995337478 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc252d9d700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc254147fb0 0x7fc254145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc251d9b700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 0x7fc254145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25259c700 1 -- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc254147fb0 msgr2=0x7fc254145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.179+0000 7fc25259c700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc254147fb0 0x7fc254145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25259c700 1 -- 172.21.15.185:0/995337478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 msgr2=0x7fc254145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25259c700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 0x7fc254145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25259c700 1 -- 172.21.15.185:0/995337478 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254140280 con 0x7fc25405ba80 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc251d9b700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 0x7fc254145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25259c700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fc24400ef80 tx=0x7fc2440091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24400cd40 con 0x7fc25405ba80 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24400cea0 con 0x7fc25405ba80 2020-11-16T22:17:23.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244012600 con 0x7fc25405ba80 2020-11-16T22:17:23.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc254147180 con 0x7fc25405ba80 2020-11-16T22:17:23.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2541462d0 con 0x7fc25405ba80 2020-11-16T22:17:23.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25405d200 con 0x7fc25405ba80 2020-11-16T22:17:23.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc244022070 con 0x7fc25405ba80 2020-11-16T22:17:23.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 0x7fc224057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc251d9b700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 0x7fc224057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.183+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc24406e5c0 con 0x7fc25405ba80 2020-11-16T22:17:23.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.187+0000 7fc251d9b700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 0x7fc224057520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fc2540659f0 tx=0x7fc23c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.191+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc244044820 con 0x7fc25405ba80 2020-11-16T22:17:23.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.347+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fc2540b40d0 con 0x7fc25405ba80 2020-11-16T22:17:23.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.355+0000 7fc24a7fc700 1 -- 172.21.15.185:0/995337478 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc244016020 con 0x7fc25405ba80 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 msgr2=0x7fc224057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 0x7fc224057520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fc2540659f0 tx=0x7fc23c015000).stop 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 msgr2=0x7fc25413de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fc24400ef80 tx=0x7fc2440091d0).stop 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 shutdown_connections 2020-11-16T22:17:23.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc224055060 0x7fc224057520 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc25405ba80 0x7fc25413de60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc254147fb0 0x7fc254145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 --2- 172.21.15.185:0/995337478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2540b1a20 0x7fc254145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 >> 172.21.15.185:0/995337478 conn(0x7fc254063df0 msgr2=0x7fc2540b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 shutdown_connections 2020-11-16T22:17:23.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.359+0000 7fc25359e700 1 -- 172.21.15.185:0/995337478 wait complete. 2020-11-16T22:17:23.356 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:17:23.357 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:23.395 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:23.412 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:17:23.427 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:17:23.428 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:17:23.428 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 Processor -- start 2020-11-16T22:17:23.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- start start 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fbc0b0f90 0x7f8fbc144300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc066f80 con 0x7f8fbc148550 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc1478d0 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc147b70 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc23ca700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fbc0b0f90 0x7f8fbc144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fbc0b0f90 0x7f8fbc144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59526/0 (socket says 172.21.15.185:59526) 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc23ca700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52968/0 (socket says 172.21.15.185:52968) 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 -- 172.21.15.185:0/3232278124 learned_addr learned my addr 172.21.15.185:0/3232278124 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc1bc9700 1 --1- 172.21.15.185:0/3232278124 >> v1:172.21.15.185:6789/0 conn(0x7f8fbc148550 0x7f8fbc141470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52964/0 (socket says 172.21.15.185:52964) 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 -- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 msgr2=0x7f8fbc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:23.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 --2- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 -- 172.21.15.185:0/3232278124 >> v1:172.21.15.185:6789/0 conn(0x7f8fbc148550 legacy=0x7f8fbc141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 -- 172.21.15.185:0/3232278124 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc147e10 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc2bcb700 1 --2- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc144300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f8fb8009d50 tx=0x7f8fb8009290).ready entity=mon.1 client_cookie=d8dc66c9785241ae server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc13c8700 1 -- 172.21.15.185:0/3232278124 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800bbc0 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc13c8700 1 -- 172.21.15.185:0/3232278124 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800bd20 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc13c8700 1 -- 172.21.15.185:0/3232278124 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8011610 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 msgr2=0x7f8fbc144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 --2- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc144300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f8fb8009d50 tx=0x7f8fb8009290).stop 2020-11-16T22:17:23.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- 172.21.15.185:0/3232278124 shutdown_connections 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 --2- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc144300 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 --2- 172.21.15.185:0/3232278124 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- 172.21.15.185:0/3232278124 >> 172.21.15.185:0/3232278124 conn(0x7f8fbc0604a0 msgr2=0x7f8fbc0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- 172.21.15.185:0/3232278124 shutdown_connections 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.615+0000 7f8fc462e700 1 -- 172.21.15.185:0/3232278124 wait complete. 2020-11-16T22:17:23.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 Processor -- start 2020-11-16T22:17:23.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- start start 2020-11-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 0x7f8fbc157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc149440 con 0x7f8fbc148550 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc1496a0 con 0x7f8fbc0b0f90 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc149940 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc23ca700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc23ca700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59530/0 (socket says 172.21.15.185:59530) 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52976/0 (socket says 172.21.15.185:52976) 2020-11-16T22:17:23.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc23ca700 1 -- 172.21.15.185:0/2460910141 learned_addr learned my addr 172.21.15.185:0/2460910141 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc2bcb700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 0x7f8fbc157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 -- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 msgr2=0x7f8fbc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 -- 172.21.15.185:0/2460910141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 msgr2=0x7f8fbc157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 0x7f8fbc157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 -- 172.21.15.185:0/2460910141 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc157d40 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc23ca700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc2bcb700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 0x7f8fbc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc1bc9700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8fbc0b4120 tx=0x7f8fac007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0072e0 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac005530 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0b2560 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0058b0 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.619+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc099c90 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc141ac0 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8fac010070 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8faa7fc700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 0x7f8f94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8fac06d3d0 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8fc23ca700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 0x7f8f94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.623+0000 7f8fc23ca700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 0x7f8f94057520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f8fb4006a90 tx=0x7f8fb4006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.627+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8fac043630 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.823+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fbc149b10 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.823+0000 7f8faa7fc700 1 -- 172.21.15.185:0/2460910141 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f8fac007cc0 con 0x7f8fbc0b37e0 2020-11-16T22:17:23.809 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:23.809 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 msgr2=0x7f8f94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 0x7f8f94057520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f8fb4006a90 tx=0x7f8fb4006df0).stop 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 msgr2=0x7f8fbc099580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8fbc0b4120 tx=0x7f8fac007b80).stop 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 shutdown_connections 2020-11-16T22:17:23.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f94055060 0x7f8f94057520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fbc0b0f90 0x7f8fbc149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc148550 0x7f8fbc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 --2- 172.21.15.185:0/2460910141 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fbc0b37e0 0x7f8fbc099580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 >> 172.21.15.185:0/2460910141 conn(0x7f8fbc0604a0 msgr2=0x7f8fbc141200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 shutdown_connections 2020-11-16T22:17:23.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.827+0000 7f8fc462e700 1 -- 172.21.15.185:0/2460910141 wait complete. 2020-11-16T22:17:23.814 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:23.829 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:23.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 Processor -- start 2020-11-16T22:17:23.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 -- start start 2020-11-16T22:17:23.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec640accc0 0x7fec640ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fec641454c0 0x7fec641422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec64145c40 con 0x7fec640abef0 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640b4c40 con 0x7fec641454c0 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec6a613700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64145ee0 con 0x7fec640accc0 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec637fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec640accc0 0x7fec640ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec637fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fec640accc0 0x7fec640ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59540/0 (socket says 172.21.15.185:59540) 2020-11-16T22:17:23.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec637fe700 1 -- 172.21.15.185:0/3776401790 learned_addr learned my addr 172.21.15.185:0/3776401790 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec63fff700 1 --1- 172.21.15.185:0/3776401790 >> v1:172.21.15.185:6789/0 conn(0x7fec640abef0 0x7fec640b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52976/0 (socket says 172.21.15.185:52976) 2020-11-16T22:17:23.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.963+0000 7fec68bb0700 1 --2- 172.21.15.185:0/3776401790 >> v2:172.21.15.96:3300/0 conn(0x7fec641454c0 0x7fec641422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec637fe700 1 -- 172.21.15.185:0/3776401790 >> v2:172.21.15.96:3300/0 conn(0x7fec641454c0 msgr2=0x7fec641422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:23.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec637fe700 1 --2- 172.21.15.185:0/3776401790 >> v2:172.21.15.96:3300/0 conn(0x7fec641454c0 0x7fec641422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec637fe700 1 -- 172.21.15.185:0/3776401790 >> v1:172.21.15.185:6789/0 conn(0x7fec640abef0 legacy=0x7fec640b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec62ffd700 1 -- 172.21.15.185:0/3776401790 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728733071 0 0) 0x7fec64145c40 con 0x7fec640abef0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec637fe700 1 -- 172.21.15.185:0/3776401790 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64146180 con 0x7fec640accc0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec637fe700 1 --2- 172.21.15.185:0/3776401790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec640ac2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fec4c009d50 tx=0x7fec4c009290).ready entity=mon.1 client_cookie=acbd622557fa4db8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec62ffd700 1 -- 172.21.15.185:0/3776401790 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00bb80 con 0x7fec640accc0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec62ffd700 1 -- 172.21.15.185:0/3776401790 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c00bce0 con 0x7fec640accc0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec62ffd700 1 -- 172.21.15.185:0/3776401790 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011610 con 0x7fec640accc0 2020-11-16T22:17:23.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- 172.21.15.185:0/3776401790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 msgr2=0x7fec640ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 --2- 172.21.15.185:0/3776401790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec640ac2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fec4c009d50 tx=0x7fec4c009290).stop 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- 172.21.15.185:0/3776401790 shutdown_connections 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 --2- 172.21.15.185:0/3776401790 >> v2:172.21.15.96:3300/0 conn(0x7fec641454c0 0x7fec641422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 --2- 172.21.15.185:0/3776401790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec640ac2d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- 172.21.15.185:0/3776401790 >> 172.21.15.185:0/3776401790 conn(0x7fec64060130 msgr2=0x7fec64060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- 172.21.15.185:0/3776401790 shutdown_connections 2020-11-16T22:17:23.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- 172.21.15.185:0/3776401790 wait complete. 2020-11-16T22:17:23.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 Processor -- start 2020-11-16T22:17:23.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.967+0000 7fec6a613700 1 -- start start 2020-11-16T22:17:23.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec640abef0 0x7fec64140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 0x7fec6413b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6413ddb0 con 0x7fec640abef0 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64151e50 con 0x7fec640accc0 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6413dfe0 con 0x7fec641454c0 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec637fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec637fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59544/0 (socket says 172.21.15.185:59544) 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 0x7fec6413b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec637fe700 1 -- 172.21.15.185:0/2156147122 learned_addr learned my addr 172.21.15.185:0/2156147122 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec63fff700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec640abef0 0x7fec64140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 -- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 msgr2=0x7fec6413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 -- 172.21.15.185:0/2156147122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec640abef0 msgr2=0x7fec64140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec640abef0 0x7fec64140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:23.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 -- 172.21.15.185:0/2156147122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6413e280 con 0x7fec641454c0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec637fe700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec68bb0700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 0x7fec6413b7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fec5c00e950 tx=0x7fec5c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00c6b0 con 0x7fec641454c0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00c810 con 0x7fec641454c0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00cbd0 con 0x7fec641454c0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64151a90 con 0x7fec641454c0 2020-11-16T22:17:23.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.971+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64141500 con 0x7fec641454c0 2020-11-16T22:17:23.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64151cb0 con 0x7fec641454c0 2020-11-16T22:17:23.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fec5c01a030 con 0x7fec641454c0 2020-11-16T22:17:23.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec5bfff700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 0x7fec34057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:23.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec5c06d8e0 con 0x7fec641454c0 2020-11-16T22:17:23.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec63fff700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 0x7fec34057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:23.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.975+0000 7fec63fff700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 0x7fec34057520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fec540011a0 tx=0x7fec54007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:23.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:23.983+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fec64151cb0 con 0x7fec641454c0 2020-11-16T22:17:24.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.139+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec6413e410 con 0x7fec641454c0 2020-11-16T22:17:24.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.139+0000 7fec5bfff700 1 -- 172.21.15.185:0/2156147122 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fec5c043b40 con 0x7fec641454c0 2020-11-16T22:17:24.123 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:24.124 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":44,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:20.401788+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:24.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 msgr2=0x7fec34057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 0x7fec34057520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fec540011a0 tx=0x7fec54007a20).stop 2020-11-16T22:17:24.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 msgr2=0x7fec6413b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 0x7fec6413b7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fec5c00e950 tx=0x7fec5c006df0).stop 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 shutdown_connections 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec34055060 0x7fec34057520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fec641454c0 0x7fec6413b7b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec640abef0 0x7fec64140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 --2- 172.21.15.185:0/2156147122 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fec640accc0 0x7fec6413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 >> 172.21.15.185:0/2156147122 conn(0x7fec64060130 msgr2=0x7fec640b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 shutdown_connections 2020-11-16T22:17:24.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.143+0000 7fec6a613700 1 -- 172.21.15.185:0/2156147122 wait complete. 2020-11-16T22:17:24.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:16:12.044444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:16:17.522251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:17:24.143 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:17:24.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 Processor -- start 2020-11-16T22:17:24.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 -- start start 2020-11-16T22:17:24.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 0x7f28980ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28980ad490 con 0x7f28980afc60 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28980ad730 con 0x7f2898144860 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289ce98700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980ad9d0 con 0x7f28980b7440 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f2896d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 0x7f28980ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f2895d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f28980afc60 0x7f289813dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:52992/0 (socket says 172.21.15.185:52992) 2020-11-16T22:17:24.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f2896d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 0x7f28980ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59556/0 (socket says 172.21.15.185:59556) 2020-11-16T22:17:24.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:52996/0 (socket says 172.21.15.185:52996) 2020-11-16T22:17:24.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f2895d9b700 1 -- 172.21.15.185:0/2232657227 learned_addr learned my addr 172.21.15.185:0/2232657227 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:24.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 -- 172.21.15.185:0/2232657227 >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 msgr2=0x7f28980ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:24.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 --2- 172.21.15.185:0/2232657227 >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 0x7f28980ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 -- 172.21.15.185:0/2232657227 >> v1:172.21.15.185:6789/0 conn(0x7f28980afc60 legacy=0x7f289813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289559a700 1 -- 172.21.15.185:0/2232657227 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186971526 0 0) 0x7f28980ad490 con 0x7f28980afc60 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 -- 172.21.15.185:0/2232657227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289813e5b0 con 0x7f28980b7440 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289659c700 1 --2- 172.21.15.185:0/2232657227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f288800b9a0 tx=0x7f2888006df0).ready entity=mon.2 client_cookie=d5f897568f506e79 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289559a700 1 -- 172.21.15.185:0/2232657227 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28880097b0 con 0x7f28980b7440 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289559a700 1 -- 172.21.15.185:0/2232657227 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888009910 con 0x7f28980b7440 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.295+0000 7f289559a700 1 -- 172.21.15.185:0/2232657227 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888009c90 con 0x7f28980b7440 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- 172.21.15.185:0/2232657227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 msgr2=0x7f28980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 --2- 172.21.15.185:0/2232657227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f288800b9a0 tx=0x7f2888006df0).stop 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- 172.21.15.185:0/2232657227 shutdown_connections 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 --2- 172.21.15.185:0/2232657227 >> v2:172.21.15.185:3300/0 conn(0x7f2898144860 0x7f28980ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 --2- 172.21.15.185:0/2232657227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f28980b7820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- 172.21.15.185:0/2232657227 >> 172.21.15.185:0/2232657227 conn(0x7f2898067860 msgr2=0x7f2898067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- 172.21.15.185:0/2232657227 shutdown_connections 2020-11-16T22:17:24.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- 172.21.15.185:0/2232657227 wait complete. 2020-11-16T22:17:24.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 Processor -- start 2020-11-16T22:17:24.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.299+0000 7f289ce98700 1 -- start start 2020-11-16T22:17:24.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f289813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 0x7f289813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980b4220 con 0x7f2898144860 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980b4420 con 0x7f28980afc60 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980b46c0 con 0x7f28980b7440 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59558/0 (socket says 172.21.15.185:59558) 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 -- 172.21.15.185:0/2199720342 learned_addr learned my addr 172.21.15.185:0/2199720342 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:24.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f2895d9b700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f289813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f2896d9d700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 0x7f289813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 -- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 msgr2=0x7f289813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f289813c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 -- 172.21.15.185:0/2199720342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 msgr2=0x7f289813c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 0x7f289813c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 -- 172.21.15.185:0/2199720342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980b4960 con 0x7f28980afc60 2020-11-16T22:17:24.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f2896d9d700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 0x7f289813c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f2895d9b700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f289813c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289659c700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f2888006e30 tx=0x7f2888015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888011e50 con 0x7f28980afc60 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288801e460 con 0x7f28980afc60 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288801e7e0 con 0x7f28980afc60 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28980ba270 con 0x7f28980afc60 2020-11-16T22:17:24.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28980b32e0 con 0x7f28980afc60 2020-11-16T22:17:24.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2888025030 con 0x7f28980afc60 2020-11-16T22:17:24.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289805d4e0 con 0x7f28980afc60 2020-11-16T22:17:24.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 0x7f286c057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f288806f600 con 0x7f28980afc60 2020-11-16T22:17:24.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.303+0000 7f2895d9b700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 0x7f286c057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.307+0000 7f2895d9b700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 0x7f286c057570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f2898080c90 tx=0x7f288c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.307+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f28880458c0 con 0x7f28980afc60 2020-11-16T22:17:24.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.459+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f28980ae530 con 0x7f28980afc60 2020-11-16T22:17:24.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f28867fc700 1 -- 172.21.15.185:0/2199720342 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2888023070 con 0x7f28980afc60 2020-11-16T22:17:24.445 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:17:24.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 msgr2=0x7f286c057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 0x7f286c057570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f2898080c90 tx=0x7f288c009040).stop 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 msgr2=0x7f28980b9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f2888006e30 tx=0x7f2888015000).stop 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 shutdown_connections 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f286c0550b0 0x7f286c057570 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f28980afc60 0x7f28980b9d80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2898144860 0x7f289813c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 --2- 172.21.15.185:0/2199720342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f28980b7440 0x7f289813c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 >> 172.21.15.185:0/2199720342 conn(0x7f2898067860 msgr2=0x7f28980b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 shutdown_connections 2020-11-16T22:17:24.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.463+0000 7f289ce98700 1 -- 172.21.15.185:0/2199720342 wait complete. 2020-11-16T22:17:24.459 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:17:24.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 Processor -- start 2020-11-16T22:17:24.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 -- start start 2020-11-16T22:17:24.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb6240ba800 0x7fb624144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb62413dd30 0x7fb6240b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6241444f0 con 0x7fb62413dd30 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6241446e0 con 0x7fb6240ba800 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6291ae700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624144980 con 0x7fb6240b5c40 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6237fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb6240ba800 0x7fb624144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53010/0 (socket says 172.21.15.185:53010) 2020-11-16T22:17:24.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 -- 172.21.15.185:0/3856533477 learned_addr learned my addr 172.21.15.185:0/3856533477 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:24.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb6289ad700 1 --2- 172.21.15.185:0/3856533477 >> v2:172.21.15.96:3300/0 conn(0x7fb62413dd30 0x7fb6240b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 -- 172.21.15.185:0/3856533477 >> v2:172.21.15.185:3300/0 conn(0x7fb6240ba800 msgr2=0x7fb624144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:24.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 --2- 172.21.15.185:0/3856533477 >> v2:172.21.15.185:3300/0 conn(0x7fb6240ba800 0x7fb624144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 -- 172.21.15.185:0/3856533477 >> v2:172.21.15.96:3300/0 conn(0x7fb62413dd30 msgr2=0x7fb6240b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 --2- 172.21.15.185:0/3856533477 >> v2:172.21.15.96:3300/0 conn(0x7fb62413dd30 0x7fb6240b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 -- 172.21.15.185:0/3856533477 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624144c20 con 0x7fb6240b5c40 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb623fff700 1 --2- 172.21.15.185:0/3856533477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb62405c690 tx=0x7fb614006eb0).ready entity=mon.2 client_cookie=1b1f8591f876afad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb622ffd700 1 -- 172.21.15.185:0/3856533477 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009720 con 0x7fb6240b5c40 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb622ffd700 1 -- 172.21.15.185:0/3856533477 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614009880 con 0x7fb6240b5c40 2020-11-16T22:17:24.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.599+0000 7fb622ffd700 1 -- 172.21.15.185:0/3856533477 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009c00 con 0x7fb6240b5c40 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/3856533477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 msgr2=0x7fb6240b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- 172.21.15.185:0/3856533477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb62405c690 tx=0x7fb614006eb0).stop 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/3856533477 shutdown_connections 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- 172.21.15.185:0/3856533477 >> v2:172.21.15.96:3300/0 conn(0x7fb62413dd30 0x7fb6240b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- 172.21.15.185:0/3856533477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240b5c40 0x7fb6240b6020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- 172.21.15.185:0/3856533477 >> v2:172.21.15.185:3300/0 conn(0x7fb6240ba800 0x7fb624144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/3856533477 >> 172.21.15.185:0/3856533477 conn(0x7fb62405ba80 msgr2=0x7fb62405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/3856533477 shutdown_connections 2020-11-16T22:17:24.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/3856533477 wait complete. 2020-11-16T22:17:24.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 Processor -- start 2020-11-16T22:17:24.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- start start 2020-11-16T22:17:24.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6240b5c40 0x7fb6240b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 0x7fb6240ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62414a3e0 con 0x7fb6240b5c40 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241540d0 con 0x7fb62413dd30 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624145d00 con 0x7fb6240ba800 2020-11-16T22:17:24.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 0x7fb6240ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59574/0 (socket says 172.21.15.185:59574) 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 -- 172.21.15.185:0/381609570 learned_addr learned my addr 172.21.15.185:0/381609570 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 0x7fb6240ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53016/0 (socket says 172.21.15.185:53016) 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb623fff700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6240b5c40 0x7fb6240b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 -- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 msgr2=0x7fb6240ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 0x7fb6240ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 -- 172.21.15.185:0/381609570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6240b5c40 msgr2=0x7fb6240b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6240b5c40 0x7fb6240b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 -- 172.21.15.185:0/381609570 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624145fa0 con 0x7fb62413dd30 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6289ad700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fb61c00ef50 tx=0x7fb61c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009690 con 0x7fb62413dd30 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00cb40 con 0x7fb62413dd30 2020-11-16T22:17:24.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0123f0 con 0x7fb62413dd30 2020-11-16T22:17:24.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624146240 con 0x7fb62413dd30 2020-11-16T22:17:24.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.603+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240af3e0 con 0x7fb62413dd30 2020-11-16T22:17:24.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62405d200 con 0x7fb62413dd30 2020-11-16T22:17:24.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb61c022070 con 0x7fb62413dd30 2020-11-16T22:17:24.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb603fff700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 0x7fb5fc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb623fff700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 0x7fb5fc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb61c06e520 con 0x7fb62413dd30 2020-11-16T22:17:24.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.607+0000 7fb623fff700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 0x7fb5fc057520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb61400b1b0 tx=0x7fb614015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.615+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb61c044780 con 0x7fb62413dd30 2020-11-16T22:17:24.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.763+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb62405c720 con 0x7fb62413dd30 2020-11-16T22:17:24.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb603fff700 1 -- 172.21.15.185:0/381609570 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb61c016070 con 0x7fb62413dd30 2020-11-16T22:17:24.749 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 msgr2=0x7fb5fc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 0x7fb5fc057520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb61400b1b0 tx=0x7fb614015000).stop 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 msgr2=0x7fb624145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fb61c00ef50 tx=0x7fb61c0091d0).stop 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 shutdown_connections 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5fc055060 0x7fb5fc057520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb62413dd30 0x7fb624145380 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6240b5c40 0x7fb6240b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 --2- 172.21.15.185:0/381609570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb6240ba800 0x7fb6240ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 >> 172.21.15.185:0/381609570 conn(0x7fb62405ba80 msgr2=0x7fb6240b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 shutdown_connections 2020-11-16T22:17:24.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.767+0000 7fb6291ae700 1 -- 172.21.15.185:0/381609570 wait complete. 2020-11-16T22:17:24.767 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:17:24.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 Processor -- start 2020-11-16T22:17:24.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 -- start start 2020-11-16T22:17:24.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11940b3920 0x7f11940b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f11940b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940b1310 con 0x7f11940b2e80 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119413e400 con 0x7f11940b3920 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f119a065700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940ac070 con 0x7f1194143ef0 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f1199864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f11940b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f1199063700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f11940b2e80 0x7f119413dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53018/0 (socket says 172.21.15.185:53018) 2020-11-16T22:17:24.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f1199063700 1 -- 172.21.15.185:0/2565724186 learned_addr learned my addr 172.21.15.185:0/2565724186 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.907+0000 7f1198862700 1 --2- 172.21.15.185:0/2565724186 >> v2:172.21.15.96:3300/0 conn(0x7f11940b3920 0x7f11940b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1193fff700 1 -- 172.21.15.185:0/2565724186 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112252912 0 0) 0x7f11940b1310 con 0x7f11940b2e80 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1193fff700 1 -- 172.21.15.185:0/2565724186 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1174003240 con 0x7f11940b2e80 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1199864700 1 -- 172.21.15.185:0/2565724186 >> v2:172.21.15.96:3300/0 conn(0x7f11940b3920 msgr2=0x7f11940b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1199864700 1 --2- 172.21.15.185:0/2565724186 >> v2:172.21.15.96:3300/0 conn(0x7f11940b3920 0x7f11940b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1199864700 1 -- 172.21.15.185:0/2565724186 >> v1:172.21.15.185:6789/0 conn(0x7f11940b2e80 legacy=0x7f119413dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1199864700 1 -- 172.21.15.185:0/2565724186 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940ac310 con 0x7f1194143ef0 2020-11-16T22:17:24.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1199864700 1 --2- 172.21.15.185:0/2565724186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f11940b16b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f117c00b9a0 tx=0x7f117c006df0).ready entity=mon.2 client_cookie=8ae355037a39b785 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1193fff700 1 -- 172.21.15.185:0/2565724186 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c009680 con 0x7f1194143ef0 2020-11-16T22:17:24.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1193fff700 1 -- 172.21.15.185:0/2565724186 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0097e0 con 0x7f1194143ef0 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f1193fff700 1 -- 172.21.15.185:0/2565724186 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c009b60 con 0x7f1194143ef0 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- 172.21.15.185:0/2565724186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 msgr2=0x7f11940b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 --2- 172.21.15.185:0/2565724186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f11940b16b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f117c00b9a0 tx=0x7f117c006df0).stop 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- 172.21.15.185:0/2565724186 shutdown_connections 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 --2- 172.21.15.185:0/2565724186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f11940b16b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 --2- 172.21.15.185:0/2565724186 >> v2:172.21.15.96:3300/0 conn(0x7f11940b3920 0x7f11940b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- 172.21.15.185:0/2565724186 >> 172.21.15.185:0/2565724186 conn(0x7f1194060130 msgr2=0x7f1194060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- 172.21.15.185:0/2565724186 shutdown_connections 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- 172.21.15.185:0/2565724186 wait complete. 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 Processor -- start 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.911+0000 7f119a065700 1 -- start start 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11940b3920 0x7f11941537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f1194153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119413ea10 con 0x7f11940b3920 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119413ecb0 con 0x7f11940b2e80 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119413ef50 con 0x7f1194143ef0 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f1194153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59586/0 (socket says 172.21.15.185:59586) 2020-11-16T22:17:24.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 -- 172.21.15.185:0/437438290 learned_addr learned my addr 172.21.15.185:0/437438290 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:24.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1198862700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11940b3920 0x7f11941537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 -- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 msgr2=0x7f1194153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f1194153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 -- 172.21.15.185:0/437438290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11940b3920 msgr2=0x7f11941537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:24.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11940b3920 0x7f11941537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:24.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 -- 172.21.15.185:0/437438290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194153340 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f1199063700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f118c00c6e0 tx=0x7f118c007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0072e0 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c007440 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c005800 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119413bf60 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11941417c0 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119405d200 con 0x7f11940b2e80 2020-11-16T22:17:24.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.915+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f118c010070 con 0x7f11940b2e80 2020-11-16T22:17:24.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.919+0000 7f11917fa700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 0x7f1170057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:24.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.919+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f118c06d430 con 0x7f11940b2e80 2020-11-16T22:17:24.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.919+0000 7f1198862700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 0x7f1170057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:24.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.919+0000 7f1198862700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 0x7f1170057520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f11940affb0 tx=0x7f11840091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:24.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:24.923+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f118c043690 con 0x7f11940b2e80 2020-11-16T22:17:25.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11940609f0 con 0x7f11940b2e80 2020-11-16T22:17:25.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f11917fa700 1 -- 172.21.15.185:0/437438290 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f118c046cb0 con 0x7f11940b2e80 2020-11-16T22:17:25.063 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:25.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 msgr2=0x7f1170057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 0x7f1170057520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f11940affb0 tx=0x7f11840091d0).stop 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 msgr2=0x7f119413ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f118c00c6e0 tx=0x7f118c007ad0).stop 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 shutdown_connections 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1170055060 0x7f1170057520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11940b2e80 0x7f119413ba70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1194143ef0 0x7f1194153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 --2- 172.21.15.185:0/437438290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11940b3920 0x7f11941537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 >> 172.21.15.185:0/437438290 conn(0x7f1194060130 msgr2=0x7f11940b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 shutdown_connections 2020-11-16T22:17:25.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.079+0000 7f119a065700 1 -- 172.21.15.185:0/437438290 wait complete. 2020-11-16T22:17:25.077 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:17:25.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.215+0000 7f8c66f0f700 1 Processor -- start 2020-11-16T22:17:25.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.215+0000 7f8c66f0f700 1 -- start start 2020-11-16T22:17:25.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c600ba800 0x7f8c600b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c60143470 0x7f8c601438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c600b7290 con 0x7f8c600b5d70 2020-11-16T22:17:25.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c600b4960 con 0x7f8c600ba800 2020-11-16T22:17:25.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c600b4bf0 con 0x7f8c60143470 2020-11-16T22:17:25.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c60143470 0x7f8c601438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8c60143470 0x7f8c601438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59598/0 (socket says 172.21.15.185:59598) 2020-11-16T22:17:25.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6570c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c600ba800 0x7f8c600b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 -- 172.21.15.185:0/1714000753 learned_addr learned my addr 172.21.15.185:0/1714000753 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:25.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 -- 172.21.15.185:0/1714000753 >> v2:172.21.15.96:3300/0 conn(0x7f8c600ba800 msgr2=0x7f8c600b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:25.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 --2- 172.21.15.185:0/1714000753 >> v2:172.21.15.96:3300/0 conn(0x7f8c600ba800 0x7f8c600b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 -- 172.21.15.185:0/1714000753 >> v1:172.21.15.96:6789/0 conn(0x7f8c600b5d70 legacy=0x7f8c600b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 -- 172.21.15.185:0/1714000753 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600b74c0 con 0x7f8c60143470 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c6670e700 1 --2- 172.21.15.185:0/1714000753 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c60143470 0x7f8c601438e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8c60081070 tx=0x7f8c50006eb0).ready entity=mon.1 client_cookie=b76f38e6f047711f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c64f0b700 1 -- 172.21.15.185:0/1714000753 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50009810 con 0x7f8c60143470 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c64f0b700 1 -- 172.21.15.185:0/1714000753 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c50009970 con 0x7f8c60143470 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c64f0b700 1 -- 172.21.15.185:0/1714000753 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50009cf0 con 0x7f8c60143470 2020-11-16T22:17:25.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1714000753 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c60143470 msgr2=0x7f8c601438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1714000753 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c60143470 0x7f8c601438e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8c60081070 tx=0x7f8c50006eb0).stop 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1714000753 shutdown_connections 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1714000753 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c60143470 0x7f8c601438e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1714000753 >> v2:172.21.15.96:3300/0 conn(0x7f8c600ba800 0x7f8c600b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1714000753 >> 172.21.15.185:0/1714000753 conn(0x7f8c6005ba80 msgr2=0x7f8c6005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1714000753 shutdown_connections 2020-11-16T22:17:25.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.219+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1714000753 wait complete. 2020-11-16T22:17:25.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 Processor -- start 2020-11-16T22:17:25.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- start start 2020-11-16T22:17:25.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c600ba800 0x7f8c6014ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60143470 0x7f8c60156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600b0120 con 0x7f8c60143470 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c60153440 con 0x7f8c600ba800 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600afeb0 con 0x7f8c600b5d70 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53044/0 (socket says 172.21.15.185:53044) 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 -- 172.21.15.185:0/1362836188 learned_addr learned my addr 172.21.15.185:0/1362836188 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:25.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c6570c700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c600ba800 0x7f8c6014ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c6670e700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60143470 0x7f8c60156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 -- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c600ba800 msgr2=0x7f8c6014ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c600ba800 0x7f8c6014ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 -- 172.21.15.185:0/1362836188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60143470 msgr2=0x7f8c60156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60143470 0x7f8c60156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 -- 172.21.15.185:0/1362836188 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60156630 con 0x7f8c600b5d70 2020-11-16T22:17:25.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c65f0d700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f8c5800ca90 tx=0x7f8c58007f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5800a6d0 con 0x7f8c600b5d70 2020-11-16T22:17:25.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5800a830 con 0x7f8c600b5d70 2020-11-16T22:17:25.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6014b280 con 0x7f8c600b5d70 2020-11-16T22:17:25.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5800abd0 con 0x7f8c600b5d70 2020-11-16T22:17:25.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.223+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c600afb00 con 0x7f8c600b5d70 2020-11-16T22:17:25.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c600afcf0 con 0x7f8c600b5d70 2020-11-16T22:17:25.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8c58019030 con 0x7f8c600b5d70 2020-11-16T22:17:25.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c4dffb700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 0x7f8c38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c5806d650 con 0x7f8c600b5d70 2020-11-16T22:17:25.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c6570c700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 0x7f8c38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.227+0000 7f8c6570c700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 0x7f8c38057520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f8c6014b810 tx=0x7f8c5c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.235+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8c580438b0 con 0x7f8c600b5d70 2020-11-16T22:17:25.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.387+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c6005d200 con 0x7f8c600b5d70 2020-11-16T22:17:25.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.387+0000 7f8c4dffb700 1 -- 172.21.15.185:0/1362836188 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8c58017020 con 0x7f8c600b5d70 2020-11-16T22:17:25.371 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:25.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 msgr2=0x7f8c38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 0x7f8c38057520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f8c6014b810 tx=0x7f8c5c006df0).stop 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 msgr2=0x7f8c60152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f8c5800ca90 tx=0x7f8c58007f00).stop 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 shutdown_connections 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8c38055060 0x7f8c38057520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60143470 0x7f8c60156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8c600b5d70 0x7f8c60152f50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 --2- 172.21.15.185:0/1362836188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8c600ba800 0x7f8c6014ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 >> 172.21.15.185:0/1362836188 conn(0x7f8c6005ba80 msgr2=0x7f8c601471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 shutdown_connections 2020-11-16T22:17:25.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.391+0000 7f8c66f0f700 1 -- 172.21.15.185:0/1362836188 wait complete. 2020-11-16T22:17:25.386 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:25.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 Processor -- start 2020-11-16T22:17:25.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 -- start start 2020-11-16T22:17:25.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f75d409c900 0x7f75d409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f75d409b270 0x7f75d40b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d409d410 con 0x7f75d41476e0 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d40b6b20 con 0x7f75d409b270 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75dc417700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d409b010 con 0x7f75d409c900 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f75d409c900 0x7f75d409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f75d409c900 0x7f75d409cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59608/0 (socket says 172.21.15.185:59608) 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 -- 172.21.15.185:0/361574662 learned_addr learned my addr 172.21.15.185:0/361574662 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:25.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75d99b2700 1 --2- 172.21.15.185:0/361574662 >> v2:172.21.15.96:3300/0 conn(0x7f75d409b270 0x7f75d40b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 -- 172.21.15.185:0/361574662 >> v2:172.21.15.96:3300/0 conn(0x7f75d409b270 msgr2=0x7f75d40b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 --2- 172.21.15.185:0/361574662 >> v2:172.21.15.96:3300/0 conn(0x7f75d409b270 0x7f75d40b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 -- 172.21.15.185:0/361574662 >> v1:172.21.15.96:6789/0 conn(0x7f75d41476e0 legacy=0x7f75d4147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 -- 172.21.15.185:0/361574662 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d409a630 con 0x7f75d409c900 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75da1b3700 1 --2- 172.21.15.185:0/361574662 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d409cce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f75d40bb700 tx=0x7f75d0009290).ready entity=mon.1 client_cookie=88bd4a8d63e04d35 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75d91b1700 1 -- 172.21.15.185:0/361574662 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000bbc0 con 0x7f75d409c900 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75d91b1700 1 -- 172.21.15.185:0/361574662 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000bd20 con 0x7f75d409c900 2020-11-16T22:17:25.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.523+0000 7f75d91b1700 1 -- 172.21.15.185:0/361574662 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011610 con 0x7f75d409c900 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- 172.21.15.185:0/361574662 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 msgr2=0x7f75d409cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- 172.21.15.185:0/361574662 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d409cce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f75d40bb700 tx=0x7f75d0009290).stop 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- 172.21.15.185:0/361574662 shutdown_connections 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- 172.21.15.185:0/361574662 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d409cce0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- 172.21.15.185:0/361574662 >> v2:172.21.15.96:3300/0 conn(0x7f75d409b270 0x7f75d40b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- 172.21.15.185:0/361574662 >> 172.21.15.185:0/361574662 conn(0x7f75d4068330 msgr2=0x7f75d40ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- 172.21.15.185:0/361574662 shutdown_connections 2020-11-16T22:17:25.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- 172.21.15.185:0/361574662 wait complete. 2020-11-16T22:17:25.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 Processor -- start 2020-11-16T22:17:25.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- start start 2020-11-16T22:17:25.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75d409b270 0x7f75d40b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d41476e0 0x7f75d4157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40621b0 con 0x7f75d41476e0 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40b1070 con 0x7f75d409c900 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75dc417700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40b1310 con 0x7f75d409b270 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75da1b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75d409b270 0x7f75d40b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59618/0 (socket says 172.21.15.185:59618) 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 -- 172.21.15.185:0/3053758301 learned_addr learned my addr 172.21.15.185:0/3053758301 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75da9b4700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d41476e0 0x7f75d4157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 -- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75d409b270 msgr2=0x7f75d40b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75d409b270 0x7f75d40b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 -- 172.21.15.185:0/3053758301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d41476e0 msgr2=0x7f75d4157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d41476e0 0x7f75d4157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.527+0000 7f75d99b2700 1 -- 172.21.15.185:0/3053758301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4156c70 con 0x7f75d409c900 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75d99b2700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f75c400eba0 tx=0x7f75c4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400c8e0 con 0x7f75d409c900 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c400ca40 con 0x7f75d409c900 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d4061c10 con 0x7f75d409c900 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400cdc0 con 0x7f75d409c900 2020-11-16T22:17:25.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d409ee50 con 0x7f75d409c900 2020-11-16T22:17:25.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f75c4012600 con 0x7f75d409c900 2020-11-16T22:17:25.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 0x7f75ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75c406e0e0 con 0x7f75d409c900 2020-11-16T22:17:25.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d40b14d0 con 0x7f75d409c900 2020-11-16T22:17:25.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75da1b3700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 0x7f75ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.531+0000 7f75da1b3700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 0x7f75ac057520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f75d0009e50 tx=0x7f75d0016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.539+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f75c40442e0 con 0x7f75d409c900 2020-11-16T22:17:25.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75d405d4e0 con 0x7f75d409c900 2020-11-16T22:17:25.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75c27fc700 1 -- 172.21.15.185:0/3053758301 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2525 (secure 0 0 0) 0x7f75c4047900 con 0x7f75d409c900 2020-11-16T22:17:25.731 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:25.731 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:16:18.387140+0000","modified":"2020-11-16T22:17:07.182027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5542,"mds_1":5475,"mds_2":5183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5183":{"gid":5183,"name":"a","rank":2,"incarnation":35,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/2473550168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2473550168},{"type":"v1","addr":"172.21.15.185:6835","nonce":2473550168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_5475":{"gid":5475,"name":"c","rank":1,"incarnation":33,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6835/4236989230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":4236989230},{"type":"v1","addr":"172.21.15.96:6835","nonce":4236989230}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_5542":{"gid":5542,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6834/653581747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":653581747},{"type":"v1","addr":"172.21.15.96:6834","nonce":653581747}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-16T22:17:25.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 msgr2=0x7f75ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 0x7f75ac057520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f75d0009e50 tx=0x7f75d0016000).stop 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 msgr2=0x7f75d4061720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f75c400eba0 tx=0x7f75c4006df0).stop 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 shutdown_connections 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f75ac055060 0x7f75ac057520 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d41476e0 0x7f75d4157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f75d409b270 0x7f75d40b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 --2- 172.21.15.185:0/3053758301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75d409c900 0x7f75d4061720 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.747+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 >> 172.21.15.185:0/3053758301 conn(0x7f75d4068330 msgr2=0x7f75d409cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.751+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 shutdown_connections 2020-11-16T22:17:25.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.751+0000 7f75dc417700 1 -- 172.21.15.185:0/3053758301 wait complete. 2020-11-16T22:17:25.735 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 41 2020-11-16T22:17:25.745 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:25.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 Processor -- start 2020-11-16T22:17:25.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- start start 2020-11-16T22:17:25.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6f340b17e0 0x7f6f34067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f34098cc0 con 0x7f6f34066240 2020-11-16T22:17:25.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f340b3950 con 0x7f6f340b2160 2020-11-16T22:17:25.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3414c4f0 con 0x7f6f340b17e0 2020-11-16T22:17:25.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3259c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6f340b17e0 0x7f6f34067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f31d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6f340b2160 0x7f6f3414bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53062/0 (socket says 172.21.15.185:53062) 2020-11-16T22:17:25.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f31d9b700 1 -- 172.21.15.185:0/3601390007 learned_addr learned my addr 172.21.15.185:0/3601390007 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:25.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3259c700 1 -- 172.21.15.185:0/3601390007 >> v1:172.21.15.185:6789/0 conn(0x7f6f340b2160 legacy=0x7f6f3414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3259c700 1 -- 172.21.15.185:0/3601390007 >> v1:172.21.15.96:6789/0 conn(0x7f6f34066240 legacy=0x7f6f34065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3259c700 1 -- 172.21.15.185:0/3601390007 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340b3bf0 con 0x7f6f340b17e0 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3159a700 1 -- 172.21.15.185:0/3601390007 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970673459 0 0) 0x7f6f340b3950 con 0x7f6f340b2160 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3259c700 1 --2- 172.21.15.185:0/3601390007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f34067fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f6f24009c90 tx=0x7f6f240091d0).ready entity=mon.1 client_cookie=1dd113201b398922 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3159a700 1 -- 172.21.15.185:0/3601390007 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bbc0 con 0x7f6f340b17e0 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3159a700 1 -- 172.21.15.185:0/3601390007 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400bd20 con 0x7f6f340b17e0 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f3159a700 1 -- 172.21.15.185:0/3601390007 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24011610 con 0x7f6f340b17e0 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- 172.21.15.185:0/3601390007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 msgr2=0x7f6f34067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/3601390007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f34067fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f6f24009c90 tx=0x7f6f240091d0).stop 2020-11-16T22:17:25.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- 172.21.15.185:0/3601390007 shutdown_connections 2020-11-16T22:17:25.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/3601390007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f34067fb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- 172.21.15.185:0/3601390007 >> 172.21.15.185:0/3601390007 conn(0x7f6f340604a0 msgr2=0x7f6f340608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:25.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- 172.21.15.185:0/3601390007 shutdown_connections 2020-11-16T22:17:25.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.883+0000 7f6f38a9d700 1 -- 172.21.15.185:0/3601390007 wait complete. 2020-11-16T22:17:25.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 Processor -- start 2020-11-16T22:17:25.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- start start 2020-11-16T22:17:25.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f34066240 0x7f6f3414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f3408f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 0x7f6f3408f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ae440 con 0x7f6f34066240 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ae670 con 0x7f6f340b17e0 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f34158710 con 0x7f6f340b2160 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f32d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 0x7f6f3408f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f3408f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f32d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 0x7f6f3408f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53074/0 (socket says 172.21.15.185:53074) 2020-11-16T22:17:25.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f32d9d700 1 -- 172.21.15.185:0/2174849385 learned_addr learned my addr 172.21.15.185:0/2174849385 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f3259c700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f34066240 0x7f6f3414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 -- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 msgr2=0x7f6f3408f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 0x7f6f3408f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 -- 172.21.15.185:0/2174849385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f34066240 msgr2=0x7f6f3414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f34066240 0x7f6f3414bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 -- 172.21.15.185:0/2174849385 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340acf00 con 0x7f6f340b17e0 2020-11-16T22:17:25.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f31d9b700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f3408f040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f6f1c000bc0 tx=0x7f6f1c0079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0072e0 con 0x7f6f340b17e0 2020-11-16T22:17:25.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c007460 con 0x7f6f340b17e0 2020-11-16T22:17:25.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3408fa20 con 0x7f6f340b17e0 2020-11-16T22:17:25.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c005840 con 0x7f6f340b17e0 2020-11-16T22:17:25.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.887+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f340ad920 con 0x7f6f340b17e0 2020-11-16T22:17:25.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f340adb10 con 0x7f6f340b17e0 2020-11-16T22:17:25.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6f1c010070 con 0x7f6f340b17e0 2020-11-16T22:17:25.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f2a7fc700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 0x7f6f10057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:25.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f1c06d400 con 0x7f6f340b17e0 2020-11-16T22:17:25.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f3259c700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 0x7f6f10057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:25.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.891+0000 7f6f3259c700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 0x7f6f10057520 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f6f24009fd0 tx=0x7f6f24016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:25.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:25.899+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6f1c043550 con 0x7f6f340b17e0 2020-11-16T22:17:26.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.051+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f34158540 con 0x7f6f340b17e0 2020-11-16T22:17:26.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.051+0000 7f6f2a7fc700 1 -- 172.21.15.185:0/2174849385 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f6f1c046b70 con 0x7f6f340b17e0 2020-11-16T22:17:26.036 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:26.036 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":44,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:16:20.401788+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:16:12.044444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-16T22:16:17.522251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:26.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 msgr2=0x7f6f10057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 0x7f6f10057520 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f6f24009fd0 tx=0x7f6f24016000).stop 2020-11-16T22:17:26.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 msgr2=0x7f6f3408f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f3408f040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f6f1c000bc0 tx=0x7f6f1c0079e0).stop 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 shutdown_connections 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6f10055060 0x7f6f10057520 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6f340b2160 0x7f6f3408f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f34066240 0x7f6f3414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 --2- 172.21.15.185:0/2174849385 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6f340b17e0 0x7f6f3408f040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 >> 172.21.15.185:0/2174849385 conn(0x7f6f340604a0 msgr2=0x7f6f34062c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:26.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 shutdown_connections 2020-11-16T22:17:26.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.055+0000 7f6f38a9d700 1 -- 172.21.15.185:0/2174849385 wait complete. 2020-11-16T22:17:26.050 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:17:26.050 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:17:26.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.183+0000 7f43e2f8c700 1 Processor -- start 2020-11-16T22:17:26.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.183+0000 7f43e2f8c700 1 -- start start 2020-11-16T22:17:26.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f43dc0b8820 0x7f43dc0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc0af130 con 0x7f43dc141910 2020-11-16T22:17:26.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc141d80 con 0x7f43dc0b8820 2020-11-16T22:17:26.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0af360 con 0x7f43dc0b30e0 2020-11-16T22:17:26.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53080/0 (socket says 172.21.15.185:53080) 2020-11-16T22:17:26.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 -- 172.21.15.185:0/2044534195 learned_addr learned my addr 172.21.15.185:0/2044534195 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:26.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1f8a700 1 --2- 172.21.15.185:0/2044534195 >> v2:172.21.15.185:3300/0 conn(0x7f43dc0b8820 0x7f43dc0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 -- 172.21.15.185:0/2044534195 >> v2:172.21.15.185:3300/0 conn(0x7f43dc0b8820 msgr2=0x7f43dc0af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 --2- 172.21.15.185:0/2044534195 >> v2:172.21.15.185:3300/0 conn(0x7f43dc0b8820 0x7f43dc0af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 -- 172.21.15.185:0/2044534195 >> v1:172.21.15.96:6789/0 conn(0x7f43dc141910 legacy=0x7f43dc1454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e0f88700 1 -- 172.21.15.185:0/2044534195 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189312084 0 0) 0x7f43dc0af130 con 0x7f43dc141910 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 -- 172.21.15.185:0/2044534195 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0af600 con 0x7f43dc0b30e0 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e1789700 1 --2- 172.21.15.185:0/2044534195 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f43d8006fd0 tx=0x7f43d8006eb0).ready entity=mon.2 client_cookie=1fbe918d3191dd9e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:26.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e0f88700 1 -- 172.21.15.185:0/2044534195 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8009810 con 0x7f43dc0b30e0 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e0f88700 1 -- 172.21.15.185:0/2044534195 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8009970 con 0x7f43dc0b30e0 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e0f88700 1 -- 172.21.15.185:0/2044534195 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8009cf0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- 172.21.15.185:0/2044534195 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 msgr2=0x7f43dc141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/2044534195 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f43d8006fd0 tx=0x7f43d8006eb0).stop 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- 172.21.15.185:0/2044534195 shutdown_connections 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/2044534195 >> v2:172.21.15.185:3300/0 conn(0x7f43dc0b8820 0x7f43dc0af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/2044534195 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc141420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- 172.21.15.185:0/2044534195 >> 172.21.15.185:0/2044534195 conn(0x7f43dc0645d0 msgr2=0x7f43dc0649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:26.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- 172.21.15.185:0/2044534195 shutdown_connections 2020-11-16T22:17:26.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.187+0000 7f43e2f8c700 1 -- 172.21.15.185:0/2044534195 wait complete. 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 Processor -- start 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- start start 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43dc141910 0x7f43dc157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc156a70 con 0x7f43dc141910 2020-11-16T22:17:26.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc061d30 con 0x7f43dc0b8820 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0624e0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1789700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53086/0 (socket says 172.21.15.185:53086) 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 -- 172.21.15.185:0/3426706751 learned_addr learned my addr 172.21.15.185:0/3426706751 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1789700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59644/0 (socket says 172.21.15.185:59644) 2020-11-16T22:17:26.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e278b700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43dc141910 0x7f43dc157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 -- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 msgr2=0x7f43dc1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 -- 172.21.15.185:0/3426706751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43dc141910 msgr2=0x7f43dc157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43dc141910 0x7f43dc157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 -- 172.21.15.185:0/3426706751 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc062780 con 0x7f43dc0b30e0 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1789700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:26.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e1f8a700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f43d000eba0 tx=0x7f43d0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:26.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d000b3a0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d000c8e0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc1405d0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d000cca0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.191+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc13ffa0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc061f00 con 0x7f43dc0b30e0 2020-11-16T22:17:26.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f43d0012600 con 0x7f43dc0b30e0 2020-11-16T22:17:26.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43cdffb700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 0x7f43b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:26.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f43d006ecb0 con 0x7f43dc0b30e0 2020-11-16T22:17:26.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43e1789700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 0x7f43b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:26.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.195+0000 7f43e1789700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 0x7f43b4057520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f43d800b1b0 tx=0x7f43d8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:26.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.199+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f43d0044e20 con 0x7f43dc0b30e0 2020-11-16T22:17:26.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:26.399+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f43dc140190 con 0x7f43dc0b30e0 2020-11-16T22:17:27.017 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:17:27.038+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:17:27.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.051+0000 7f43cdffb700 1 -- 172.21.15.185:0/3426706751 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f43d0048440 con 0x7f43dc0b30e0 2020-11-16T22:17:27.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 msgr2=0x7f43b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:27.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 0x7f43b4057520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f43d800b1b0 tx=0x7f43d8015000).stop 2020-11-16T22:17:27.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 msgr2=0x7f43dc13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:27.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f43d000eba0 tx=0x7f43d0006df0).stop 2020-11-16T22:17:27.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 shutdown_connections 2020-11-16T22:17:27.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f43b4055060 0x7f43b4057520 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:27.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43dc141910 0x7f43dc157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f43dc0b30e0 0x7f43dc13fab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:28.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 --2- 172.21.15.185:0/3426706751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f43dc0b8820 0x7f43dc1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:28.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 >> 172.21.15.185:0/3426706751 conn(0x7f43dc0645d0 msgr2=0x7f43dc13deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:28.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 shutdown_connections 2020-11-16T22:17:28.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:27.055+0000 7f43e2f8c700 1 -- 172.21.15.185:0/3426706751 wait complete. 2020-11-16T22:17:28.672 INFO:tasks.ceph.mds.b.smithi096.stderr: -7922> 2020-11-16T22:16:38.978+0000 7fbbc74dd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:17:28.874 INFO:tasks.ceph.mds.b.smithi096.stderr: -2069> 2020-11-16T22:17:04.026+0000 7fbbc3cd6700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v21 ap=1 f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1) n(v4 rc2020-11-16T22:16:39.383300+0000 6=0+6)/n(v3 rc2020-11-16T22:16:39.383300+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@5},l=5297 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x564599fc8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.347300+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:16:39.383300+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.078 INFO:tasks.ceph.mds.b.smithi096.stderr: -1968> 2020-11-16T22:17:04.034+0000 7fbbc9ce2700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v21 ap=2 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v3 rc2020-11-16T22:16:39.383300+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={5297=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564599fc8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:16:39.383300+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.282 INFO:tasks.ceph.mds.c.smithi096.stderr: -1079> 2020-11-16T22:17:04.030+0000 7fa378334700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v2 rc2020-11-16T22:16:39.383300+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@4},l=5297 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a8e907e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:39.383300+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:16:39.383300+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.282 INFO:tasks.ceph.mds.c.smithi096.stderr: -975> 2020-11-16T22:17:04.034+0000 7fa37e340700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1) n(v1 rc2020-11-16T22:16:39.383300+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={5297=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a8e907ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.351300+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:16:39.383300+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.282 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:17:29.283 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:17:29.283 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:17:29.283 INFO:tasks.ceph.mds.a.smithi185.stderr: -937> 2020-11-16T22:17:04.027+0000 7fe58c150700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1) n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/pAsLsXsFsx@4},l=5297 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5589c51b3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:16:36.355300+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:16:36.355300+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.283 INFO:tasks.ceph.mds.a.smithi185.stderr: -923> 2020-11-16T22:17:04.027+0000 7fe58c150700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={5297=pAsLsXsFsx/AsLsXsFsx@2},l=5297 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5589c51b3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v21>,<0x10000000001/d3 v21>,<0x10000000000/d2 v23>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:17:29.284 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:17:29.284 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:17:29.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.435+0000 7f9be7101700 1 Processor -- start 2020-11-16T22:17:29.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.435+0000 7f9be7101700 1 -- start start 2020-11-16T22:17:29.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9be00b2e80 0x7f9be00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be00ac070 con 0x7f9be013dcf0 2020-11-16T22:17:29.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be00ac310 con 0x7f9be00b2e80 2020-11-16T22:17:29.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00ac5b0 con 0x7f9be00b3920 2020-11-16T22:17:29.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9be00b2e80 0x7f9be00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be58fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9be00b2e80 0x7f9be00b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59672/0 (socket says 172.21.15.185:59672) 2020-11-16T22:17:29.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 -- 172.21.15.185:0/20702758 learned_addr learned my addr 172.21.15.185:0/20702758 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:29.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 -- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 msgr2=0x7f9be0143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:29.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 -- 172.21.15.185:0/20702758 >> v1:172.21.15.96:6789/0 conn(0x7f9be013dcf0 legacy=0x7f9be00b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 -- 172.21.15.185:0/20702758 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00ac850 con 0x7f9be00b2e80 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be58fe700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be60ff700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be00b2e80 0x7f9be00b3260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9bdc006fd0 tx=0x7f9bdc006eb0).ready entity=mon.1 client_cookie=58b80ba1d328a46e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be50fd700 1 -- 172.21.15.185:0/20702758 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc009810 con 0x7f9be00b2e80 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be50fd700 1 -- 172.21.15.185:0/20702758 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc009970 con 0x7f9be00b2e80 2020-11-16T22:17:29.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be50fd700 1 -- 172.21.15.185:0/20702758 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc009cf0 con 0x7f9be00b2e80 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be00b2e80 msgr2=0x7f9be00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be00b2e80 0x7f9be00b3260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9bdc006fd0 tx=0x7f9bdc006eb0).stop 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- 172.21.15.185:0/20702758 shutdown_connections 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be00b2e80 0x7f9be00b3260 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 --2- 172.21.15.185:0/20702758 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- 172.21.15.185:0/20702758 >> 172.21.15.185:0/20702758 conn(0x7f9be0060130 msgr2=0x7f9be0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:29.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- 172.21.15.185:0/20702758 shutdown_connections 2020-11-16T22:17:29.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- 172.21.15.185:0/20702758 wait complete. 2020-11-16T22:17:29.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 Processor -- start 2020-11-16T22:17:29.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.439+0000 7f9be7101700 1 -- start start 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9be00b2e80 0x7f9be0148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be0154200 con 0x7f9be00b2e80 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00af4d0 con 0x7f9be013dcf0 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00af770 con 0x7f9be00b3920 2020-11-16T22:17:29.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be58fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59682/0 (socket says 172.21.15.185:59682) 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 -- 172.21.15.185:0/1278362379 learned_addr learned my addr 172.21.15.185:0/1278362379 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be60ff700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9be00b2e80 0x7f9be0148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 -- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 msgr2=0x7f9be0153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:29.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 -- 172.21.15.185:0/1278362379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9be00b2e80 msgr2=0x7f9be0148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9be00b2e80 0x7f9be0148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 -- 172.21.15.185:0/1278362379 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00afa10 con 0x7f9be013dcf0 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be6900700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f9bd800ca60 tx=0x7f9bd8007e70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800a730 con 0x7f9be013dcf0 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd800a890 con 0x7f9be013dcf0 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be0153380 con 0x7f9be013dcf0 2020-11-16T22:17:29.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800ac30 con 0x7f9be013dcf0 2020-11-16T22:17:29.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.443+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be0148000 con 0x7f9be013dcf0 2020-11-16T22:17:29.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be005d200 con 0x7f9be013dcf0 2020-11-16T22:17:29.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9bd8019030 con 0x7f9be013dcf0 2020-11-16T22:17:29.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9bd27fc700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 0x7f9bb8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:29.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f9bd806d5a0 con 0x7f9be013dcf0 2020-11-16T22:17:29.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9be60ff700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 0x7f9bb8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:29.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.447+0000 7f9be60ff700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 0x7f9bb8057520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9be0141b10 tx=0x7f9bdc015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:29.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.455+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9bd80437b0 con 0x7f9be013dcf0 2020-11-16T22:17:29.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:29.659+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9be00b2c20 con 0x7f9be013dcf0 2020-11-16T22:17:30.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.055+0000 7f9bd27fc700 1 -- 172.21.15.185:0/1278362379 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9bd8046dd0 con 0x7f9be013dcf0 2020-11-16T22:17:30.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 msgr2=0x7f9bb8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:30.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 0x7f9bb8057520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9be0141b10 tx=0x7f9bdc015000).stop 2020-11-16T22:17:30.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 msgr2=0x7f9be0153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:30.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f9bd800ca60 tx=0x7f9bd8007e70).stop 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 shutdown_connections 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9bb8055060 0x7f9bb8057520 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9be013dcf0 0x7f9be0153d10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9be00b2e80 0x7f9be0148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 --2- 172.21.15.185:0/1278362379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9be00b3920 0x7f9be0153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 >> 172.21.15.185:0/1278362379 conn(0x7f9be0060130 msgr2=0x7f9be00b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:30.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 shutdown_connections 2020-11-16T22:17:30.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.059+0000 7f9be7101700 1 -- 172.21.15.185:0/1278362379 wait complete. 2020-11-16T22:17:30.059 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:17:30.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.207+0000 7fb8b49d5700 1 Processor -- start 2020-11-16T22:17:30.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- start start 2020-11-16T22:17:30.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8b01440f0 0x7fb8b0144570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 0x7fb8b0147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b00679b0 con 0x7fb8b01440f0 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b0067c50 con 0x7fb8b0144a60 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0067ef0 con 0x7fb8b00b7610 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8af59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 0x7fb8b0147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53132/0 (socket says 172.21.15.185:53132) 2020-11-16T22:17:30.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8af59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 0x7fb8b0147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59690/0 (socket says 172.21.15.185:59690) 2020-11-16T22:17:30.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2943721549 learned_addr learned my addr 172.21.15.185:0/2943721549 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:30.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8ae59c700 1 --2- 172.21.15.185:0/2943721549 >> v2:172.21.15.96:3300/0 conn(0x7fb8b01440f0 0x7fb8b0144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2943721549 >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 msgr2=0x7fb8b0147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:30.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2943721549 >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 0x7fb8b0147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2943721549 >> v2:172.21.15.96:3300/0 conn(0x7fb8b01440f0 msgr2=0x7fb8b0144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2943721549 >> v2:172.21.15.96:3300/0 conn(0x7fb8b01440f0 0x7fb8b0144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2943721549 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0068190 con 0x7fb8b00b7610 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2943721549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb8b0061260 tx=0x7fb8a0009290).ready entity=mon.2 client_cookie=55e057991fadfbe9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8add9b700 1 -- 172.21.15.185:0/2943721549 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000bb10 con 0x7fb8b00b7610 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8add9b700 1 -- 172.21.15.185:0/2943721549 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000bc70 con 0x7fb8b00b7610 2020-11-16T22:17:30.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8add9b700 1 -- 172.21.15.185:0/2943721549 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0011400 con 0x7fb8b00b7610 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2943721549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 msgr2=0x7fb8b00b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2943721549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb8b0061260 tx=0x7fb8a0009290).stop 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2943721549 shutdown_connections 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2943721549 >> v2:172.21.15.185:3300/0 conn(0x7fb8b0144a60 0x7fb8b0147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2943721549 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b00b7610 0x7fb8b00b2f80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2943721549 >> v2:172.21.15.96:3300/0 conn(0x7fb8b01440f0 0x7fb8b0144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2943721549 >> 172.21.15.185:0/2943721549 conn(0x7fb8b0063780 msgr2=0x7fb8b00b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:30.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2943721549 shutdown_connections 2020-11-16T22:17:30.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.211+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2943721549 wait complete. 2020-11-16T22:17:30.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 Processor -- start 2020-11-16T22:17:30.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- start start 2020-11-16T22:17:30.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 0x7fb8b01479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 0x7fb8b00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0153a60 con 0x7fb8b01440f0 2020-11-16T22:17:30.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0153cf0 con 0x7fb8b00b7610 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0153f90 con 0x7fb8b0144a60 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59692/0 (socket says 172.21.15.185:59692) 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2518888269 learned_addr learned my addr 172.21.15.185:0/2518888269 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8af59e700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 0x7fb8b00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8ae59c700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 0x7fb8b01479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 msgr2=0x7fb8b00ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 0x7fb8b00ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2518888269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 msgr2=0x7fb8b01479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 0x7fb8b01479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 -- 172.21.15.185:0/2518888269 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00acf60 con 0x7fb8b00b7610 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8ae59c700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 0x7fb8b01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8af59e700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 0x7fb8b00ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:30.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8aed9d700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fb8b0068030 tx=0x7fb8a0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a001c680 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a001c7e0 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b0147eb0 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a001cb80 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.215+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b0143a50 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb8a0017070 con 0x7fb8b00b7610 2020-11-16T22:17:30.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b005d200 con 0x7fb8b00b7610 2020-11-16T22:17:30.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8a6ffd700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 0x7fb880057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:30.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fb8a006fe80 con 0x7fb8b00b7610 2020-11-16T22:17:30.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8ae59c700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 0x7fb880057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:30.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.219+0000 7fb8ae59c700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 0x7fb880057520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb8b0066e50 tx=0x7fb898008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:30.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.231+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb8a0045fe0 con 0x7fb8b00b7610 2020-11-16T22:17:30.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:30.379+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8b005c6d0 con 0x7fb8b00b7610 2020-11-16T22:17:31.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.067+0000 7fb8a6ffd700 1 -- 172.21.15.185:0/2518888269 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb8a0015020 con 0x7fb8b00b7610 2020-11-16T22:17:31.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 msgr2=0x7fb880057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 0x7fb880057520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fb8b0066e50 tx=0x7fb898008040).stop 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 msgr2=0x7fb8b0153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fb8b0068030 tx=0x7fb8a0016000).stop 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 shutdown_connections 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb880055060 0x7fb880057520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8b00b7610 0x7fb8b0153570 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8b0144a60 0x7fb8b00ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 --2- 172.21.15.185:0/2518888269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b01440f0 0x7fb8b01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 >> 172.21.15.185:0/2518888269 conn(0x7fb8b0063780 msgr2=0x7fb8b00b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:31.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 shutdown_connections 2020-11-16T22:17:31.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.071+0000 7fb8b49d5700 1 -- 172.21.15.185:0/2518888269 wait complete. 2020-11-16T22:17:31.056 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:17:31.070 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 Processor -- start 2020-11-16T22:17:31.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 -- start start 2020-11-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5650068220 0x7f56500ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 0x7f565013d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500ac5a0 con 0x7f56500b7610 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56500ac830 con 0x7f5650068220 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f56555f2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b1810 con 0x7f5650143a20 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f564ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f56500b7610 0x7f56500b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53138/0 (socket says 172.21.15.185:53138) 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f564ffff700 1 -- 172.21.15.185:0/905508456 learned_addr learned my addr 172.21.15.185:0/905508456 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f5654df1700 1 --2- 172.21.15.185:0/905508456 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 0x7f565013d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f564f7fe700 1 --2- 172.21.15.185:0/905508456 >> v2:172.21.15.185:3300/0 conn(0x7f5650068220 0x7f56500ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f564effd700 1 -- 172.21.15.185:0/905508456 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155443150 0 0) 0x7f56500ac5a0 con 0x7f56500b7610 2020-11-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f564effd700 1 -- 172.21.15.185:0/905508456 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c003240 con 0x7f56500b7610 2020-11-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f5654df1700 1 -- 172.21.15.185:0/905508456 >> v2:172.21.15.185:3300/0 conn(0x7f5650068220 msgr2=0x7f56500ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f5654df1700 1 --2- 172.21.15.185:0/905508456 >> v2:172.21.15.185:3300/0 conn(0x7f5650068220 0x7f56500ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f5654df1700 1 -- 172.21.15.185:0/905508456 >> v1:172.21.15.185:6789/0 conn(0x7f56500b7610 legacy=0x7f56500b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.219+0000 7f5654df1700 1 -- 172.21.15.185:0/905508456 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500b1ab0 con 0x7f5650143a20 2020-11-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f5654df1700 1 --2- 172.21.15.185:0/905508456 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 0x7f565013d220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f5648009c90 tx=0x7f56480091d0).ready entity=mon.2 client_cookie=225e270f9d34df88 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f564effd700 1 -- 172.21.15.185:0/905508456 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800b9c0 con 0x7f5650143a20 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f564effd700 1 -- 172.21.15.185:0/905508456 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800bb20 con 0x7f5650143a20 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f564effd700 1 -- 172.21.15.185:0/905508456 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800bea0 con 0x7f5650143a20 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- 172.21.15.185:0/905508456 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 msgr2=0x7f565013d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 --2- 172.21.15.185:0/905508456 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 0x7f565013d220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f5648009c90 tx=0x7f56480091d0).stop 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- 172.21.15.185:0/905508456 shutdown_connections 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 --2- 172.21.15.185:0/905508456 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650143a20 0x7f565013d220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 --2- 172.21.15.185:0/905508456 >> v2:172.21.15.185:3300/0 conn(0x7f5650068220 0x7f56500ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- 172.21.15.185:0/905508456 >> 172.21.15.185:0/905508456 conn(0x7f5650063780 msgr2=0x7f56500b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- 172.21.15.185:0/905508456 shutdown_connections 2020-11-16T22:17:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- 172.21.15.185:0/905508456 wait complete. 2020-11-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 Processor -- start 2020-11-16T22:17:31.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.223+0000 7f56555f2700 1 -- start start 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 0x7f5650115590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 0x7f5650115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650115f70 con 0x7f56500b7610 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650113fb0 con 0x7f5650143a20 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5650111ce0 con 0x7f5650068220 2020-11-16T22:17:31.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f564ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 0x7f5650115590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59706/0 (socket says 172.21.15.185:59706) 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 -- 172.21.15.185:0/2292938210 learned_addr learned my addr 172.21.15.185:0/2292938210 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f564f7fe700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 0x7f5650115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 -- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 msgr2=0x7f5650115590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 0x7f5650115590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 -- 172.21.15.185:0/2292938210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 msgr2=0x7f5650115a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 0x7f5650115a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 -- 172.21.15.185:0/2292938210 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650112530 con 0x7f5650143a20 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f564f7fe700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 0x7f5650115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f564ffff700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 0x7f5650115590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f5654df1700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f56500b1950 tx=0x7f5648016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564801c5f0 con 0x7f5650143a20 2020-11-16T22:17:31.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564801c750 con 0x7f5650143a20 2020-11-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501127d0 con 0x7f5650143a20 2020-11-16T22:17:31.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564801cb10 con 0x7f5650143a20 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5650113060 con 0x7f5650143a20 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5650112960 con 0x7f5650143a20 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5648024030 con 0x7f5650143a20 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 0x7f5628057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.227+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f564806fa80 con 0x7f5650143a20 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.231+0000 7f564ffff700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 0x7f5628057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:31.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.231+0000 7f564ffff700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 0x7f5628057550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f56400055a0 tx=0x7f5640009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:31.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.235+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5648045d90 con 0x7f5650143a20 2020-11-16T22:17:31.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:31.391+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f565005d200 con 0x7f5650143a20 2020-11-16T22:17:32.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.079+0000 7f562ffff700 1 -- 172.21.15.185:0/2292938210 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5648022020 con 0x7f5650143a20 2020-11-16T22:17:32.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 msgr2=0x7f5628057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 0x7f5628057550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f56400055a0 tx=0x7f5640009040).stop 2020-11-16T22:17:32.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 msgr2=0x7f565013b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f56500b1950 tx=0x7f5648016000).stop 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 shutdown_connections 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5628055090 0x7f5628057550 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5650143a20 0x7f565013b940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5650068220 0x7f5650115590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 --2- 172.21.15.185:0/2292938210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56500b7610 0x7f5650115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 >> 172.21.15.185:0/2292938210 conn(0x7f5650063780 msgr2=0x7f5650140480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:32.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 shutdown_connections 2020-11-16T22:17:32.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.083+0000 7f56555f2700 1 -- 172.21.15.185:0/2292938210 wait complete. 2020-11-16T22:17:32.070 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:17:32.084 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:17:32.084 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:17:32.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:17:32.085 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:17:32.085 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:17:32.085 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:17:32.086 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:17:32.086 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:17:32.086 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:17:32.086 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:17:32.087 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:17:32.087 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:17:32.094 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:17:32.095 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:17:32.097 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:17:32.097 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:17:32.101 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:17:32.101 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:17:32.138 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:17:32.138 DEBUG:teuthology.parallel:result is None 2020-11-16T22:17:32.144 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:17:32.144 DEBUG:teuthology.parallel:result is None 2020-11-16T22:17:32.145 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:17:32.146 DEBUG:teuthology.parallel:result is None 2020-11-16T22:17:32.146 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:17:32.217 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:17:32.221 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:17:32.222 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 Processor -- start 2020-11-16T22:17:32.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 -- start start 2020-11-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f46b00b36d0 0x7f46b00b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 0x7f46b00b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b00b1fa0 con 0x7f46b00b4030 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b00b7b80 con 0x7f46b00b36d0 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46af59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00b2240 con 0x7f46b0066190 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46a7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 0x7f46b00b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46aed9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f46b00b4030 0x7f46b00b44e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53168/0 (socket says 172.21.15.185:53168) 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46aed9d700 1 -- 172.21.15.185:0/4292175534 learned_addr learned my addr 172.21.15.185:0/4292175534 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:32.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.287+0000 7f46ae59c700 1 --2- 172.21.15.185:0/4292175534 >> v2:172.21.15.96:3300/0 conn(0x7f46b00b36d0 0x7f46b00b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46add9b700 1 -- 172.21.15.185:0/4292175534 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183362512 0 0) 0x7f46b00b1fa0 con 0x7f46b00b4030 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46add9b700 1 -- 172.21.15.185:0/4292175534 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4690003240 con 0x7f46b00b4030 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46a7fff700 1 -- 172.21.15.185:0/4292175534 >> v2:172.21.15.96:3300/0 conn(0x7f46b00b36d0 msgr2=0x7f46b00b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46a7fff700 1 --2- 172.21.15.185:0/4292175534 >> v2:172.21.15.96:3300/0 conn(0x7f46b00b36d0 0x7f46b00b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46a7fff700 1 -- 172.21.15.185:0/4292175534 >> v1:172.21.15.185:6789/0 conn(0x7f46b00b4030 legacy=0x7f46b00b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46a7fff700 1 -- 172.21.15.185:0/4292175534 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0148160 con 0x7f46b0066190 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46a7fff700 1 --2- 172.21.15.185:0/4292175534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 0x7f46b00b7f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f469800b990 tx=0x7f4698006d90).ready entity=mon.2 client_cookie=16250ea2dd33fc7c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46add9b700 1 -- 172.21.15.185:0/4292175534 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698006030 con 0x7f46b0066190 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46add9b700 1 -- 172.21.15.185:0/4292175534 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698006190 con 0x7f46b0066190 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46add9b700 1 -- 172.21.15.185:0/4292175534 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698006510 con 0x7f46b0066190 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- 172.21.15.185:0/4292175534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 msgr2=0x7f46b00b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 --2- 172.21.15.185:0/4292175534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 0x7f46b00b7f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f469800b990 tx=0x7f4698006d90).stop 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- 172.21.15.185:0/4292175534 shutdown_connections 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 --2- 172.21.15.185:0/4292175534 >> v2:172.21.15.96:3300/0 conn(0x7f46b00b36d0 0x7f46b00b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 --2- 172.21.15.185:0/4292175534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b0066190 0x7f46b00b7f70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- 172.21.15.185:0/4292175534 >> 172.21.15.185:0/4292175534 conn(0x7f46b00666d0 msgr2=0x7f46b0066ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- 172.21.15.185:0/4292175534 shutdown_connections 2020-11-16T22:17:32.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- 172.21.15.185:0/4292175534 wait complete. 2020-11-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 Processor -- start 2020-11-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.291+0000 7f46af59e700 1 -- start start 2020-11-16T22:17:32.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46b00b36d0 0x7f46b00ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 0x7f46b0114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00b0ba0 con 0x7f46b00b36d0 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00ae3b0 con 0x7f46b0066190 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b0154230 con 0x7f46b00b4030 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 0x7f46b0114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59732/0 (socket says 172.21.15.185:59732) 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 -- 172.21.15.185:0/40572040 learned_addr learned my addr 172.21.15.185:0/40572040 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a7fff700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46b00b36d0 0x7f46b00ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 -- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 msgr2=0x7f46b00acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 -- 172.21.15.185:0/40572040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46b00b36d0 msgr2=0x7f46b00ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:32.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46b00b36d0 0x7f46b00ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 -- 172.21.15.185:0/40572040 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b01448a0 con 0x7f46b00b4030 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46aed9d700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 0x7f46b0114f10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f46a800ca90 tx=0x7f46a8007ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a800ce60 con 0x7f46b00b4030 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8005b80 con 0x7f46b00b4030 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b00ae600 con 0x7f46b00b4030 2020-11-16T22:17:32.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8007110 con 0x7f46b00b4030 2020-11-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b00b0820 con 0x7f46b00b4030 2020-11-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b005d200 con 0x7f46b00b4030 2020-11-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f46a8010030 con 0x7f46b00b4030 2020-11-16T22:17:32.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 0x7f468c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f46a806d7f0 con 0x7f46b00b4030 2020-11-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 0x7f468c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:32.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.295+0000 7f46ae59c700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 0x7f468c057520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f46a0006a90 tx=0x7f46a0006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:32.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.307+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f46a8043c80 con 0x7f46b00b4030 2020-11-16T22:17:32.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:32.471+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f46b01545d0 con 0x7f46b00b4030 2020-11-16T22:17:33.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.283+0000 7f46a67fc700 1 -- 172.21.15.185:0/40572040 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46a80472a0 con 0x7f46b00b4030 2020-11-16T22:17:33.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 msgr2=0x7f468c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 0x7f468c057520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f46a0006a90 tx=0x7f46a0006df0).stop 2020-11-16T22:17:33.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 msgr2=0x7f46b0114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 0x7f46b0114f10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f46a800ca90 tx=0x7f46a8007ca0).stop 2020-11-16T22:17:33.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 shutdown_connections 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f468c055060 0x7f468c057520 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f46b0066190 0x7f46b00acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f46b00b4030 0x7f46b0114f10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 --2- 172.21.15.185:0/40572040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46b00b36d0 0x7f46b00ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 >> 172.21.15.185:0/40572040 conn(0x7f46b00666d0 msgr2=0x7f46b00b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 shutdown_connections 2020-11-16T22:17:33.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.287+0000 7f46af59e700 1 -- 172.21.15.185:0/40572040 wait complete. 2020-11-16T22:17:33.272 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:17:33.287 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:17:33.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.427+0000 7fde8f40d700 1 Processor -- start 2020-11-16T22:17:33.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.427+0000 7fde8f40d700 1 -- start start 2020-11-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde8813dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde88141150 con 0x7fde88067260 2020-11-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde881413f0 con 0x7fde88062e30 2020-11-16T22:17:33.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fde88062e30 0x7fde8813e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53184/0 (socket says 172.21.15.185:53184) 2020-11-16T22:17:33.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 -- 172.21.15.185:0/3510935492 learned_addr learned my addr 172.21.15.185:0/3510935492 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:33.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde88140f60 con 0x7fde88064170 2020-11-16T22:17:33.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8c9a8700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde8813dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838731049 0 0) 0x7fde881413f0 con 0x7fde88062e30 2020-11-16T22:17:33.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde70003240 con 0x7fde88062e30 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849841843 0 0) 0x7fde88141150 con 0x7fde88067260 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde881413f0 con 0x7fde88067260 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4197704267 0 0) 0x7fde70003240 con 0x7fde88062e30 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 msgr2=0x7fde8813dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde8813dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 >> v1:172.21.15.96:6789/0 conn(0x7fde88067260 legacy=0x7fde88062720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88141690 con 0x7fde88062e30 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fde800014e0 con 0x7fde88062e30 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 >> v1:172.21.15.185:6789/0 conn(0x7fde88062e30 legacy=0x7fde8813e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde700038c0 0x7fde70003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde700044e0 0x7fde70004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde70004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70005340 con 0x7fde700044e0 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7000d940 con 0x7fde700038c0 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7000b730 con 0x7fde88064170 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d1a9700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde700038c0 0x7fde70003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde70004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8c9a8700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde700044e0 0x7fde70004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 -- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde700038c0 msgr2=0x7fde70003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:33.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde700038c0 0x7fde70003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 -- 172.21.15.185:0/3510935492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde700044e0 msgr2=0x7fde70004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:33.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde700044e0 0x7fde70004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 -- 172.21.15.185:0/3510935492 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70003240 con 0x7fde88064170 2020-11-16T22:17:33.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d1a9700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde700038c0 0x7fde70003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:33.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde70004e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fde8000c440 tx=0x7fde880ae820).ready entity=mon.2 client_cookie=dbac2992190d6dab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:33.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800070d0 con 0x7fde88064170 2020-11-16T22:17:33.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80007230 con 0x7fde88064170 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde77fff700 1 -- 172.21.15.185:0/3510935492 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800055f0 con 0x7fde88064170 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 msgr2=0x7fde70004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8f40d700 1 --2- 172.21.15.185:0/3510935492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88064170 0x7fde70004e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fde8000c440 tx=0x7fde880ae820).stop 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.431+0000 7fde8d9aa700 1 -- 172.21.15.185:0/3510935492 reap_dead start 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 shutdown_connections 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 >> 172.21.15.185:0/3510935492 conn(0x7fde880b43e0 msgr2=0x7fde88060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:33.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 shutdown_connections 2020-11-16T22:17:33.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- 172.21.15.185:0/3510935492 wait complete. 2020-11-16T22:17:33.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 Processor -- start 2020-11-16T22:17:33.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- start start 2020-11-16T22:17:33.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde8813fd40 0x7fde88143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde8813b120 0x7fde8813b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:33.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880af710 con 0x7fde8813b120 2020-11-16T22:17:33.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde88143d90 con 0x7fde8813fd40 2020-11-16T22:17:33.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8f40d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde88143850 con 0x7fde88144fe0 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.435+0000 7fde8c9a8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8d1a9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde8813fd40 0x7fde88143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53200/0 (socket says 172.21.15.185:53200) 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 -- 172.21.15.185:0/750331320 learned_addr learned my addr 172.21.15.185:0/750331320 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8d9aa700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde8813b120 0x7fde8813b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:33.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 -- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde8813fd40 msgr2=0x7fde88143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde8813fd40 0x7fde88143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 -- 172.21.15.185:0/750331320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde8813b120 msgr2=0x7fde8813b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:33.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde8813b120 0x7fde8813b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:33.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 -- 172.21.15.185:0/750331320 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88143af0 con 0x7fde88144fe0 2020-11-16T22:17:33.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8c9a8700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fde78011c90 tx=0x7fde78006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78018070 con 0x7fde88144fe0 2020-11-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800f980 con 0x7fde88144fe0 2020-11-16T22:17:34.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde880b1e10 con 0x7fde88144fe0 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800fd00 con 0x7fde88144fe0 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde881406e0 con 0x7fde88144fe0 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8805d4e0 con 0x7fde88144fe0 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fde78014650 con 0x7fde88144fe0 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 0x7fde60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:34.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fde7806e0b0 con 0x7fde88144fe0 2020-11-16T22:17:34.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8d1a9700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 0x7fde60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:34.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.439+0000 7fde8d1a9700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 0x7fde60057520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fde840015d0 tx=0x7fde84007f10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:34.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.447+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fde78044640 con 0x7fde88144fe0 2020-11-16T22:17:34.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.647+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde88063e50 con 0x7fde88144fe0 2020-11-16T22:17:34.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.647+0000 7fde757fa700 1 -- 172.21.15.185:0/750331320 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4274 (secure 0 0 0) 0x7fde78047c60 con 0x7fde88144fe0 2020-11-16T22:17:34.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 msgr2=0x7fde60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:34.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 0x7fde60057520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fde840015d0 tx=0x7fde84007f10).stop 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 msgr2=0x7fde881444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fde78011c90 tx=0x7fde78006df0).stop 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 shutdown_connections 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fde60055060 0x7fde60057520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fde8813fd40 0x7fde88143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde8813b120 0x7fde8813b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:34.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 --2- 172.21.15.185:0/750331320 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fde88144fe0 0x7fde881444d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 >> 172.21.15.185:0/750331320 conn(0x7fde880b43e0 msgr2=0x7fde88060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 shutdown_connections 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:33.651+0000 7fde8f40d700 1 -- 172.21.15.185:0/750331320 wait complete. 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:17:34.244 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:17:34.245 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.245 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:17:34.245 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:17:34.446 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.446 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:17:34.446 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.446 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:17:34.447 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.448 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.449 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:17:34.450 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.451 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.452 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:17:34.452 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:17:34.452 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.452 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.654 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.654 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.654 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.654 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.655 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.656 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.657 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.658 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.659 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.660 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.660 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.660 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.660 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:17:34.862 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:17:34.862 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.862 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.862 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:17:34.863 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.864 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow r", 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:17:34.865 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:17:34.866 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:17:34.867 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:17:34.868 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:34.868 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:17:35.070 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:17:35.070 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:17:35.070 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:35.071 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:35.072 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:35.073 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:35.074 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:17:35.074 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:17:35.074 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:35.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 Processor -- start 2020-11-16T22:17:35.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 -- start start 2020-11-16T22:17:35.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7fe40b72e0 0x7f7fe40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe4147f90 con 0x7f7fe40b6aa0 2020-11-16T22:17:35.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe4148460 con 0x7f7fe40b64b0 2020-11-16T22:17:35.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe3fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe40b6eb0 con 0x7f7fe40b72e0 2020-11-16T22:17:35.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7fe40b72e0 0x7f7fe40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7fe40b72e0 0x7f7fe40b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59764/0 (socket says 172.21.15.185:59764) 2020-11-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 -- 172.21.15.185:0/940303372 learned_addr learned my addr 172.21.15.185:0/940303372 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:35.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe27fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7fe40b64b0 0x7f7fe4144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53204/0 (socket says 172.21.15.185:53204) 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 -- 172.21.15.185:0/940303372 >> v1:172.21.15.185:6789/0 conn(0x7f7fe40b64b0 legacy=0x7f7fe4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 -- 172.21.15.185:0/940303372 >> v1:172.21.15.96:6789/0 conn(0x7f7fe40b6aa0 legacy=0x7f7fe413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe1ffb700 1 -- 172.21.15.185:0/940303372 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415172765 0 0) 0x7f7fe4148460 con 0x7f7fe40b64b0 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 -- 172.21.15.185:0/940303372 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe40b4810 con 0x7f7fe40b72e0 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.223+0000 7f7fe2ffd700 1 --2- 172.21.15.185:0/940303372 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe40b72e0 0x7f7fe40b76c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7fcc006fd0 tx=0x7f7fcc006eb0).ready entity=mon.1 client_cookie=2b3f7c9f7d205444 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe1ffb700 1 -- 172.21.15.185:0/940303372 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc009770 con 0x7f7fe40b72e0 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe1ffb700 1 -- 172.21.15.185:0/940303372 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0098d0 con 0x7f7fe40b72e0 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe1ffb700 1 -- 172.21.15.185:0/940303372 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc009c50 con 0x7f7fe40b72e0 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- 172.21.15.185:0/940303372 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe40b72e0 msgr2=0x7f7fe40b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/940303372 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe40b72e0 0x7f7fe40b76c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7fcc006fd0 tx=0x7f7fcc006eb0).stop 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- 172.21.15.185:0/940303372 shutdown_connections 2020-11-16T22:17:35.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/940303372 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe40b72e0 0x7f7fe40b76c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- 172.21.15.185:0/940303372 >> 172.21.15.185:0/940303372 conn(0x7f7fe4067860 msgr2=0x7f7fe4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- 172.21.15.185:0/940303372 shutdown_connections 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- 172.21.15.185:0/940303372 wait complete. 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 Processor -- start 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- start start 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 0x7f7fe4115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fe40b6aa0 0x7f7fe40b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40b38a0 con 0x7f7fe40b6aa0 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4116f70 con 0x7f7fe4113ae0 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4113fd0 con 0x7f7fe40b64b0 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 0x7f7fe4115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59772/0 (socket says 172.21.15.185:59772) 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 0x7f7fe4115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53212/0 (socket says 172.21.15.185:53212) 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe37fe700 1 -- 172.21.15.185:0/3447851280 learned_addr learned my addr 172.21.15.185:0/3447851280 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe27fc700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fe40b6aa0 0x7f7fe40b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe37fe700 1 -- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 msgr2=0x7f7fe4115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.227+0000 7f7fe37fe700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 0x7f7fe4115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe37fe700 1 -- 172.21.15.185:0/3447851280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fe40b6aa0 msgr2=0x7f7fe40b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe37fe700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fe40b6aa0 0x7f7fe40b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe37fe700 1 -- 172.21.15.185:0/3447851280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4114450 con 0x7f7fe4113ae0 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe37fe700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f7fdc00cb50 tx=0x7f7fdc007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00a880 con 0x7f7fe4113ae0 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc00aa00 con 0x7f7fe4113ae0 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe4115ad0 con 0x7f7fe4113ae0 2020-11-16T22:17:35.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00adc0 con 0x7f7fe4113ae0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe40b2160 con 0x7f7fe4113ae0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7fdc010070 con 0x7f7fe4113ae0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe405d200 con 0x7f7fe4113ae0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 0x7f7fc0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7fdc06da70 con 0x7f7fe4113ae0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe2ffd700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 0x7f7fc0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.231+0000 7f7fe2ffd700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 0x7f7fc0057520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f7fe405d850 tx=0x7f7fcc015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.239+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7fdc043ef0 con 0x7f7fe4113ae0 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.391+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fe40603a0 con 0x7f7fe4113ae0 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fdaffd700 1 -- 172.21.15.185:0/3447851280 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13675 (secure 0 0 0) 0x7f7fdc047510 con 0x7f7fe4113ae0 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 msgr2=0x7f7fc0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 0x7f7fc0057520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f7fe405d850 tx=0x7f7fcc015000).stop 2020-11-16T22:17:35.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 msgr2=0x7f7fe4141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f7fdc00cb50 tx=0x7f7fdc007e80).stop 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 shutdown_connections 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7fc0055060 0x7f7fc0057520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7fe4113ae0 0x7f7fe4141330 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7fe40b64b0 0x7f7fe4115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 --2- 172.21.15.185:0/3447851280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fe40b6aa0 0x7f7fe40b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 >> 172.21.15.185:0/3447851280 conn(0x7f7fe4067860 msgr2=0x7f7fe40b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 shutdown_connections 2020-11-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.395+0000 7f7fe3fff700 1 -- 172.21.15.185:0/3447851280 wait complete. 2020-11-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:35.424 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":49,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:33.277791+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:35.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:17:35.627 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 Processor -- start 2020-11-16T22:17:35.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 -- start start 2020-11-16T22:17:35.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f27ac0b3920 0x7f27ac147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac14c6f0 con 0x7f27ac0b3920 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0ac120 con 0x7f27ac0b13d0 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27ab59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0ac3c0 con 0x7f27ac0b3080 2020-11-16T22:17:35.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27aa59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59776/0 (socket says 172.21.15.185:59776) 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27aa59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53222/0 (socket says 172.21.15.185:53222) 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1587324445 learned_addr learned my addr 172.21.15.185:0/1587324445 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27aad9d700 1 --2- 172.21.15.185:0/1587324445 >> v2:172.21.15.96:3300/0 conn(0x7f27ac0b3920 0x7f27ac147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 msgr2=0x7f27ac0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:35.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1587324445 >> v2:172.21.15.96:3300/0 conn(0x7f27ac0b3920 msgr2=0x7f27ac147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1587324445 >> v2:172.21.15.96:3300/0 conn(0x7f27ac0b3920 0x7f27ac147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1587324445 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0acd50 con 0x7f27ac0b13d0 2020-11-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f27ac061ab0 tx=0x7f279c006eb0).ready entity=mon.1 client_cookie=5cce1d3ecc8d1701 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a959a700 1 -- 172.21.15.185:0/1587324445 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c009720 con 0x7f27ac0b13d0 2020-11-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a959a700 1 -- 172.21.15.185:0/1587324445 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c009880 con 0x7f27ac0b13d0 2020-11-16T22:17:35.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.783+0000 7f27a959a700 1 -- 172.21.15.185:0/1587324445 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c009c00 con 0x7f27ac0b13d0 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b13d0 msgr2=0x7f27ac0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f27ac061ab0 tx=0x7f279c006eb0).stop 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- 172.21.15.185:0/1587324445 shutdown_connections 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1587324445 >> v2:172.21.15.96:3300/0 conn(0x7f27ac0b3920 0x7f27ac147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1587324445 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0b1bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- 172.21.15.185:0/1587324445 >> 172.21.15.185:0/1587324445 conn(0x7f27ac060130 msgr2=0x7f27ac060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- 172.21.15.185:0/1587324445 shutdown_connections 2020-11-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- 172.21.15.185:0/1587324445 wait complete. 2020-11-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 Processor -- start 2020-11-16T22:17:35.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- start start 2020-11-16T22:17:35.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac10fc30 con 0x7f27ac0b13d0 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac10fe10 con 0x7f27ac0b3920 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27ab59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac1100b0 con 0x7f27ac0b3080 2020-11-16T22:17:35.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27aad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53226/0 (socket says 172.21.15.185:53226) 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1915189061 learned_addr learned my addr 172.21.15.185:0/1915189061 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27aad9d700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59786/0 (socket says 172.21.15.185:59786) 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27aa59c700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 msgr2=0x7f27ac157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1915189061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac0b13d0 msgr2=0x7f27ac0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.787+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a9d9b700 1 -- 172.21.15.185:0/1915189061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac110350 con 0x7f27ac0b3080 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27aad9d700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a9d9b700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f27ac068950 tx=0x7f279c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c011bc0 con 0x7f27ac0b3080 2020-11-16T22:17:35.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c011d20 con 0x7f27ac0b3080 2020-11-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac1125b0 con 0x7f27ac0b3080 2020-11-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c01e690 con 0x7f27ac0b3080 2020-11-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0af7e0 con 0x7f27ac0b3080 2020-11-16T22:17:35.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac1104e0 con 0x7f27ac0b3080 2020-11-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f279c025030 con 0x7f27ac0b3080 2020-11-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 0x7f277c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f279c06ee80 con 0x7f27ac0b3080 2020-11-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27aa59c700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 0x7f277c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:35.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.791+0000 7f27aa59c700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 0x7f277c057520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f27a4009980 tx=0x7f27a4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:35.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.799+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f279c0454c0 con 0x7f27ac0b3080 2020-11-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.955+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f27ac05d200 con 0x7f27ac0b3080 2020-11-16T22:17:35.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.955+0000 7f27a27fc700 1 -- 172.21.15.185:0/1915189061 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v49) v1 ==== 110+0+10 (secure 0 0 0) 0x7f279c023070 con 0x7f27ac0b3080 2020-11-16T22:17:35.939 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.955+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 msgr2=0x7f277c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.955+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 0x7f277c057520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f27a4009980 tx=0x7f27a4008040).stop 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.955+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 msgr2=0x7f27ac0ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f27ac068950 tx=0x7f279c015000).stop 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 shutdown_connections 2020-11-16T22:17:35.941 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f277c055060 0x7f277c057520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27ac0b3920 0x7f27ac157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac0b13d0 0x7f27ac0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 --2- 172.21.15.185:0/1915189061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27ac0b3080 0x7f27ac0ae2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 >> 172.21.15.185:0/1915189061 conn(0x7f27ac060130 msgr2=0x7f27ac060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 shutdown_connections 2020-11-16T22:17:35.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:35.959+0000 7f27ab59e700 1 -- 172.21.15.185:0/1915189061 wait complete. 2020-11-16T22:17:35.954 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.095+0000 7fe15eab4700 1 Processor -- start 2020-11-16T22:17:36.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.095+0000 7fe15eab4700 1 -- start start 2020-11-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe158143ef0 0x7fe1580b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580b1310 con 0x7fe1580b3920 2020-11-16T22:17:36.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15813e400 con 0x7fe158143ef0 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580ac070 con 0x7fe1580b2e80 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53234/0 (socket says 172.21.15.185:53234) 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 -- 172.21.15.185:0/795876877 learned_addr learned my addr 172.21.15.185:0/795876877 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15e2b3700 1 --2- 172.21.15.185:0/795876877 >> v2:172.21.15.96:3300/0 conn(0x7fe158143ef0 0x7fe1580b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 -- 172.21.15.185:0/795876877 >> v2:172.21.15.96:3300/0 conn(0x7fe158143ef0 msgr2=0x7fe1580b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 --2- 172.21.15.185:0/795876877 >> v2:172.21.15.96:3300/0 conn(0x7fe158143ef0 0x7fe1580b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 -- 172.21.15.185:0/795876877 >> v1:172.21.15.96:6789/0 conn(0x7fe1580b3920 legacy=0x7fe15813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 -- 172.21.15.185:0/795876877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580ac310 con 0x7fe1580b2e80 2020-11-16T22:17:36.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15dab2700 1 --2- 172.21.15.185:0/795876877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fe148006fd0 tx=0x7fe148006eb0).ready entity=mon.2 client_cookie=f392dc3e0f2b2fe7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15cab0700 1 -- 172.21.15.185:0/795876877 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480098d0 con 0x7fe1580b2e80 2020-11-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15cab0700 1 -- 172.21.15.185:0/795876877 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148009a30 con 0x7fe1580b2e80 2020-11-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15cab0700 1 -- 172.21.15.185:0/795876877 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148009db0 con 0x7fe1580b2e80 2020-11-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- 172.21.15.185:0/795876877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 msgr2=0x7fe1580b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 --2- 172.21.15.185:0/795876877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fe148006fd0 tx=0x7fe148006eb0).stop 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- 172.21.15.185:0/795876877 shutdown_connections 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 --2- 172.21.15.185:0/795876877 >> v2:172.21.15.96:3300/0 conn(0x7fe158143ef0 0x7fe1580b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 --2- 172.21.15.185:0/795876877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b2e80 0x7fe1580b3260 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- 172.21.15.185:0/795876877 >> 172.21.15.185:0/795876877 conn(0x7fe158060130 msgr2=0x7fe158060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- 172.21.15.185:0/795876877 shutdown_connections 2020-11-16T22:17:36.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.099+0000 7fe15eab4700 1 -- 172.21.15.185:0/795876877 wait complete. 2020-11-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 Processor -- start 2020-11-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- start start 2020-11-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b3920 0x7fe158157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 0x7fe158147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580ad1a0 con 0x7fe158143ef0 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158157830 con 0x7fe1580b2e80 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe158143680 con 0x7fe1580b3920 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59796/0 (socket says 172.21.15.185:59796) 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 -- 172.21.15.185:0/3354264632 learned_addr learned my addr 172.21.15.185:0/3354264632 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:36.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15d2b1700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b3920 0x7fe158157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15e2b3700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 0x7fe158147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 -- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b3920 msgr2=0x7fe158157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b3920 0x7fe158157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 -- 172.21.15.185:0/3354264632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 msgr2=0x7fe158147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 0x7fe158147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 -- 172.21.15.185:0/3354264632 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158143920 con 0x7fe1580b2e80 2020-11-16T22:17:36.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15e2b3700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 0x7fe158147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15dab2700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fe1580ac1b0 tx=0x7fe148015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148011dd0 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe158143c20 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14801e460 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14801e800 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.103+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe158153380 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1581479b0 con 0x7fe1580b2e80 2020-11-16T22:17:36.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe148025030 con 0x7fe1580b2e80 2020-11-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe155ffb700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 0x7fe130057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe15d2b1700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 0x7fe130057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe14806f1c0 con 0x7fe1580b2e80 2020-11-16T22:17:36.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.107+0000 7fe15d2b1700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 0x7fe130057520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fe1581483f0 tx=0x7fe14c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.115+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe148045800 con 0x7fe1580b2e80 2020-11-16T22:17:36.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.263+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe15805d200 con 0x7fe1580b2e80 2020-11-16T22:17:36.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.267+0000 7fe155ffb700 1 -- 172.21.15.185:0/3354264632 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe148023070 con 0x7fe1580b2e80 2020-11-16T22:17:36.252 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:36.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 msgr2=0x7fe130057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 0x7fe130057520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fe1581483f0 tx=0x7fe14c009040).stop 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 msgr2=0x7fe1580ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fe1580ac1b0 tx=0x7fe148015000).stop 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 shutdown_connections 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe130055060 0x7fe130057520 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe1580b2e80 0x7fe1580ad600 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe158143ef0 0x7fe158147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 --2- 172.21.15.185:0/3354264632 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe1580b3920 0x7fe158157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 >> 172.21.15.185:0/3354264632 conn(0x7fe158060130 msgr2=0x7fe158067180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:36.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 shutdown_connections 2020-11-16T22:17:36.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.271+0000 7fe15eab4700 1 -- 172.21.15.185:0/3354264632 wait complete. 2020-11-16T22:17:36.270 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:17:36.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 Processor -- start 2020-11-16T22:17:36.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- start start 2020-11-16T22:17:36.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb0098cc0 con 0x7f0bb00b2160 2020-11-16T22:17:36.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00b3950 con 0x7f0bb0066240 2020-11-16T22:17:36.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb014c4f0 con 0x7f0bb00b17e0 2020-11-16T22:17:36.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb5aac700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f0bb0066240 0x7f0bb0065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53246/0 (socket says 172.21.15.185:53246) 2020-11-16T22:17:36.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53248/0 (socket says 172.21.15.185:53248) 2020-11-16T22:17:36.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb5aac700 1 -- 172.21.15.185:0/574579315 learned_addr learned my addr 172.21.15.185:0/574579315 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0ba7fff700 1 -- 172.21.15.185:0/574579315 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427809204 0 0) 0x7f0bb00b3950 con 0x7f0bb0066240 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0ba7fff700 1 -- 172.21.15.185:0/574579315 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b9c003240 con 0x7f0bb0066240 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 -- 172.21.15.185:0/574579315 >> v1:172.21.15.185:6789/0 conn(0x7f0bb0066240 legacy=0x7f0bb0065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 -- 172.21.15.185:0/574579315 >> v1:172.21.15.96:6789/0 conn(0x7f0bb00b2160 legacy=0x7f0bb014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 -- 172.21.15.185:0/574579315 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00b3bf0 con 0x7f0bb00b17e0 2020-11-16T22:17:36.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb52ab700 1 --2- 172.21.15.185:0/574579315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f0ba0009cc0 tx=0x7f0ba00091d0).ready entity=mon.2 client_cookie=c88233f6163d4545 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0ba7fff700 1 -- 172.21.15.185:0/574579315 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000bad0 con 0x7f0bb00b17e0 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0ba7fff700 1 -- 172.21.15.185:0/574579315 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba000bc30 con 0x7f0bb00b17e0 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0ba7fff700 1 -- 172.21.15.185:0/574579315 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0011400 con 0x7f0bb00b17e0 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- 172.21.15.185:0/574579315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 msgr2=0x7f0bb0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 --2- 172.21.15.185:0/574579315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f0ba0009cc0 tx=0x7f0ba00091d0).stop 2020-11-16T22:17:36.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- 172.21.15.185:0/574579315 shutdown_connections 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 --2- 172.21.15.185:0/574579315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b17e0 0x7f0bb0067fb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- 172.21.15.185:0/574579315 >> 172.21.15.185:0/574579315 conn(0x7f0bb00604a0 msgr2=0x7f0bb00608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- 172.21.15.185:0/574579315 shutdown_connections 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.407+0000 7f0bb750f700 1 -- 172.21.15.185:0/574579315 wait complete. 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 Processor -- start 2020-11-16T22:17:36.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- start start 2020-11-16T22:17:36.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0066240 0x7f0bb00ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 0x7f0bb0158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0062cd0 con 0x7f0bb0066240 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0062f00 con 0x7f0bb00b17e0 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00631a0 con 0x7f0bb00b2160 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb4aaa700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 0x7f0bb0158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb4aaa700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 0x7f0bb0158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59810/0 (socket says 172.21.15.185:59810) 2020-11-16T22:17:36.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53258/0 (socket says 172.21.15.185:53258) 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb52ab700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0066240 0x7f0bb00ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb4aaa700 1 -- 172.21.15.185:0/4039681467 learned_addr learned my addr 172.21.15.185:0/4039681467 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 -- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 msgr2=0x7f0bb0158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 0x7f0bb0158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 -- 172.21.15.185:0/4039681467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0066240 msgr2=0x7f0bb00ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0066240 0x7f0bb00ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 -- 172.21.15.185:0/4039681467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0063440 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb5aac700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f0bac00c6e0 tx=0x7f0bac007920).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0072e0 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac007440 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb014c230 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac005800 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb0157b20 con 0x7f0bb00b2160 2020-11-16T22:17:36.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.411+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb005d4e0 con 0x7f0bb00b2160 2020-11-16T22:17:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.415+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0bac010070 con 0x7f0bb00b2160 2020-11-16T22:17:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.415+0000 7f0ba57fa700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 0x7f0b88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.415+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0bac06cff0 con 0x7f0bb00b2160 2020-11-16T22:17:36.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.415+0000 7f0bb52ab700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 0x7f0b88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.415+0000 7f0bb52ab700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 0x7f0b88057520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0ba0009c10 tx=0x7f0ba0016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.419+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0bac043580 con 0x7f0bb00b2160 2020-11-16T22:17:36.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.571+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bb0062a50 con 0x7f0bb00b2160 2020-11-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.571+0000 7f0ba57fa700 1 -- 172.21.15.185:0/4039681467 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0bac046ba0 con 0x7f0bb00b2160 2020-11-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:36.556 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:17:36.557 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:17:36.558 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 msgr2=0x7f0b88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 0x7f0b88057520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0ba0009c10 tx=0x7f0ba0016000).stop 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 msgr2=0x7f0bb00aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f0bac00c6e0 tx=0x7f0bac007920).stop 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 shutdown_connections 2020-11-16T22:17:36.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0b88055060 0x7f0b88057520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0bb00b2160 0x7f0bb00aece0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb0066240 0x7f0bb00ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 --2- 172.21.15.185:0/4039681467 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0bb00b17e0 0x7f0bb0158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 >> 172.21.15.185:0/4039681467 conn(0x7f0bb00604a0 msgr2=0x7f0bb0096310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 shutdown_connections 2020-11-16T22:17:36.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.575+0000 7f0bb750f700 1 -- 172.21.15.185:0/4039681467 wait complete. 2020-11-16T22:17:36.570 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:17:36.570 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:17:36.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 Processor -- start 2020-11-16T22:17:36.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 -- start start 2020-11-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4a70061240 0x7f4a700b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a700b4350 con 0x7f4a700b4c70 2020-11-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a700629a0 con 0x7f4a70063fd0 2020-11-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a76f0f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a700b60a0 con 0x7f4a70061240 2020-11-16T22:17:36.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a75f0d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4a70063fd0 0x7f4a700b4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53256/0 (socket says 172.21.15.185:53256) 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a75f0d700 1 -- 172.21.15.185:0/2097861450 learned_addr learned my addr 172.21.15.185:0/2097861450 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a7670e700 1 --2- 172.21.15.185:0/2097861450 >> v2:172.21.15.185:3300/0 conn(0x7f4a70061240 0x7f4a700b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770577206 0 0) 0x7f4a700629a0 con 0x7f4a70063fd0 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a58003240 con 0x7f4a70063fd0 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938016309 0 0) 0x7f4a700b4350 con 0x7f4a700b4c70 2020-11-16T22:17:36.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a7670e700 1 -- 172.21.15.185:0/2097861450 >> v1:172.21.15.185:6789/0 conn(0x7f4a70063fd0 legacy=0x7f4a700b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a7670e700 1 -- 172.21.15.185:0/2097861450 >> v1:172.21.15.96:6789/0 conn(0x7f4a700b4c70 legacy=0x7f4a70143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a7670e700 1 -- 172.21.15.185:0/2097861450 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70062c40 con 0x7f4a70061240 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.715+0000 7f4a7670e700 1 --2- 172.21.15.185:0/2097861450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a700b3c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f4a6c00b9a0 tx=0x7f4a6c006df0).ready entity=mon.1 client_cookie=e5c0addf82af5ee9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0097b0 con 0x7f4a70061240 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c009910 con 0x7f4a70061240 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a74f0b700 1 -- 172.21.15.185:0/2097861450 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c009c90 con 0x7f4a70061240 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- 172.21.15.185:0/2097861450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 msgr2=0x7f4a700b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/2097861450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a700b3c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f4a6c00b9a0 tx=0x7f4a6c006df0).stop 2020-11-16T22:17:36.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- 172.21.15.185:0/2097861450 shutdown_connections 2020-11-16T22:17:36.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/2097861450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a700b3c40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- 172.21.15.185:0/2097861450 >> 172.21.15.185:0/2097861450 conn(0x7f4a70067940 msgr2=0x7f4a70067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:36.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- 172.21.15.185:0/2097861450 shutdown_connections 2020-11-16T22:17:36.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- 172.21.15.185:0/2097861450 wait complete. 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 Processor -- start 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.719+0000 7f4a76f0f700 1 -- start start 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a70063fd0 0x7f4a7013faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4a700b4c70 0x7f4a70153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a70157660 con 0x7f4a70063fd0 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700afe30 con 0x7f4a70061240 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700b00d0 con 0x7f4a700b4c70 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59824/0 (socket says 172.21.15.185:59824) 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 -- 172.21.15.185:0/3999291434 learned_addr learned my addr 172.21.15.185:0/3999291434 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a7670e700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4a700b4c70 0x7f4a70153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a7570c700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a70063fd0 0x7f4a7013faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 -- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4a700b4c70 msgr2=0x7f4a70153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4a700b4c70 0x7f4a70153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 -- 172.21.15.185:0/3999291434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a70063fd0 msgr2=0x7f4a7013faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a70063fd0 0x7f4a7013faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 -- 172.21.15.185:0/3999291434 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70153240 con 0x7f4a70061240 2020-11-16T22:17:36.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a75f0d700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f4a6800ca60 tx=0x7f4a68007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68005b80 con 0x7f4a70061240 2020-11-16T22:17:36.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68005ce0 con 0x7f4a70061240 2020-11-16T22:17:36.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a701534e0 con 0x7f4a70061240 2020-11-16T22:17:36.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68007340 con 0x7f4a70061240 2020-11-16T22:17:36.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a700af990 con 0x7f4a70061240 2020-11-16T22:17:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a700afc40 con 0x7f4a70061240 2020-11-16T22:17:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.723+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4a68010030 con 0x7f4a70061240 2020-11-16T22:17:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.727+0000 7f4a5dffb700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 0x7f4a48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.727+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4a6806d9b0 con 0x7f4a70061240 2020-11-16T22:17:36.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.727+0000 7f4a7570c700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 0x7f4a48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:36.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.727+0000 7f4a7570c700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 0x7f4a48057520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f4a7014b6f0 tx=0x7f4a60009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:36.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.731+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4a68043f40 con 0x7f4a70061240 2020-11-16T22:17:36.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:36.883+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4a7005d200 con 0x7f4a70061240 2020-11-16T22:17:37.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.291+0000 7f4a5dffb700 1 -- 172.21.15.185:0/3999291434 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4a68047560 con 0x7f4a70061240 2020-11-16T22:17:37.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 msgr2=0x7f4a48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 0x7f4a48057520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f4a7014b6f0 tx=0x7f4a60009040).stop 2020-11-16T22:17:37.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 msgr2=0x7f4a70157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f4a6800ca60 tx=0x7f4a68007e40).stop 2020-11-16T22:17:37.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 shutdown_connections 2020-11-16T22:17:37.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4a48055060 0x7f4a48057520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4a70061240 0x7f4a70157170 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4a700b4c70 0x7f4a70153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 --2- 172.21.15.185:0/3999291434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a70063fd0 0x7f4a7013faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 >> 172.21.15.185:0/3999291434 conn(0x7f4a70067940 msgr2=0x7f4a700b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:37.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 shutdown_connections 2020-11-16T22:17:37.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.295+0000 7f4a76f0f700 1 -- 172.21.15.185:0/3999291434 wait complete. 2020-11-16T22:17:37.280 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:17:37.295 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:17:37.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.443+0000 7f8511a31700 1 Processor -- start 2020-11-16T22:17:37.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.443+0000 7f8511a31700 1 -- start start 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f850c0ac820 0x7f850c143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c060970 con 0x7f850c060130 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0b4610 con 0x7f850c0ac820 2020-11-16T22:17:37.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c060ba0 con 0x7f850c0b3830 2020-11-16T22:17:37.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f850bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f850c0ac820 0x7f850c143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511230700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f850c060130 0x7f850c13f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53272/0 (socket says 172.21.15.185:53272) 2020-11-16T22:17:37.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53276/0 (socket says 172.21.15.185:53276) 2020-11-16T22:17:37.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511230700 1 -- 172.21.15.185:0/1080090671 learned_addr learned my addr 172.21.15.185:0/1080090671 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:37.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f850b7fe700 1 -- 172.21.15.185:0/1080090671 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608413622 0 0) 0x7f850c060970 con 0x7f850c060130 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 -- 172.21.15.185:0/1080090671 >> v2:172.21.15.185:3300/0 conn(0x7f850c0ac820 msgr2=0x7f850c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 --2- 172.21.15.185:0/1080090671 >> v2:172.21.15.185:3300/0 conn(0x7f850c0ac820 0x7f850c143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 -- 172.21.15.185:0/1080090671 >> v1:172.21.15.185:6789/0 conn(0x7f850c060130 legacy=0x7f850c13f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 -- 172.21.15.185:0/1080090671 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c060e40 con 0x7f850c0b3830 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8510a2f700 1 --2- 172.21.15.185:0/1080090671 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f850000b9a0 tx=0x7f8500006df0).ready entity=mon.2 client_cookie=d6c1d6c444f20f14 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:37.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f850b7fe700 1 -- 172.21.15.185:0/1080090671 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000097b0 con 0x7f850c0b3830 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f850b7fe700 1 -- 172.21.15.185:0/1080090671 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500009910 con 0x7f850c0b3830 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f850b7fe700 1 -- 172.21.15.185:0/1080090671 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500009c90 con 0x7f850c0b3830 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- 172.21.15.185:0/1080090671 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 msgr2=0x7f850c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 --2- 172.21.15.185:0/1080090671 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f850000b9a0 tx=0x7f8500006df0).stop 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- 172.21.15.185:0/1080090671 shutdown_connections 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 --2- 172.21.15.185:0/1080090671 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143a40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 --2- 172.21.15.185:0/1080090671 >> v2:172.21.15.185:3300/0 conn(0x7f850c0ac820 0x7f850c143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- 172.21.15.185:0/1080090671 >> 172.21.15.185:0/1080090671 conn(0x7f850c063df0 msgr2=0x7f850c0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:37.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- 172.21.15.185:0/1080090671 shutdown_connections 2020-11-16T22:17:37.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.447+0000 7f8511a31700 1 -- 172.21.15.185:0/1080090671 wait complete. 2020-11-16T22:17:37.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 Processor -- start 2020-11-16T22:17:37.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- start start 2020-11-16T22:17:37.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 0x7f850c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0af3f0 con 0x7f850c060130 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1549c0 con 0x7f850c0ac820 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c154c60 con 0x7f850c0b3830 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f850bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53284/0 (socket says 172.21.15.185:53284) 2020-11-16T22:17:37.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f850bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59840/0 (socket says 172.21.15.185:59840) 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 -- 172.21.15.185:0/4138571862 learned_addr learned my addr 172.21.15.185:0/4138571862 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8510a2f700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 0x7f850c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 -- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 msgr2=0x7f850c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 -- 172.21.15.185:0/4138571862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 msgr2=0x7f850c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:37.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 0x7f850c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 -- 172.21.15.185:0/4138571862 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c154f00 con 0x7f850c0b3830 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8510a2f700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 0x7f850c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f850bfff700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511230700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f850400cb10 tx=0x7f8504007d20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504005b80 con 0x7f850c0b3830 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8504005ce0 con 0x7f850c0b3830 2020-11-16T22:17:37.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c144240 con 0x7f850c0b3830 2020-11-16T22:17:37.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504007320 con 0x7f850c0b3830 2020-11-16T22:17:37.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.451+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c0add20 con 0x7f850c0b3830 2020-11-16T22:17:37.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c05d200 con 0x7f850c0b3830 2020-11-16T22:17:37.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8504010030 con 0x7f850c0b3830 2020-11-16T22:17:37.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8508ff9700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 0x7f84e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:37.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f850406dd20 con 0x7f850c0b3830 2020-11-16T22:17:37.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8510a2f700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 0x7f84e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:37.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.455+0000 7f8510a2f700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 0x7f84e4057520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8500006e30 tx=0x7f8500015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:37.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.463+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8504044040 con 0x7f850c0b3830 2020-11-16T22:17:37.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:37.615+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f850c061e20 con 0x7f850c0b3830 2020-11-16T22:17:38.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.295+0000 7f8508ff9700 1 -- 172.21.15.185:0/4138571862 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8504047660 con 0x7f850c0b3830 2020-11-16T22:17:38.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.295+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 msgr2=0x7f84e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.295+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 0x7f84e4057520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8500006e30 tx=0x7f8500015000).stop 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 msgr2=0x7f850c143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f850400cb10 tx=0x7f8504007d20).stop 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 shutdown_connections 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f84e4055060 0x7f84e4057520 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f850c0b3830 0x7f850c143d50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f850c060130 0x7f850c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 --2- 172.21.15.185:0/4138571862 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f850c0ac820 0x7f850c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 >> 172.21.15.185:0/4138571862 conn(0x7f850c063df0 msgr2=0x7f850c13eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:38.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 shutdown_connections 2020-11-16T22:17:38.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.299+0000 7f8511a31700 1 -- 172.21.15.185:0/4138571862 wait complete. 2020-11-16T22:17:38.487 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:17:38.488 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:17:38.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.643+0000 7f91cecd2700 1 Processor -- start 2020-11-16T22:17:38.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.643+0000 7f91cecd2700 1 -- start start 2020-11-16T22:17:38.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91c80afa00 0x7f91c80b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 0x7f91c80b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c8142270 con 0x7f91c80b8830 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c80b0160 con 0x7f91c80b5c00 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c80b6090 con 0x7f91c80afa00 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91c80afa00 0x7f91c80b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91c80afa00 0x7f91c80b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59848/0 (socket says 172.21.15.185:59848) 2020-11-16T22:17:38.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 -- 172.21.15.185:0/802228865 learned_addr learned my addr 172.21.15.185:0/802228865 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:38.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91ce4d1700 1 --2- 172.21.15.185:0/802228865 >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 0x7f91c80b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 -- 172.21.15.185:0/802228865 >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 msgr2=0x7f91c80b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:38.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 --2- 172.21.15.185:0/802228865 >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 0x7f91c80b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 -- 172.21.15.185:0/802228865 >> v1:172.21.15.96:6789/0 conn(0x7f91c80b8830 legacy=0x7f91c80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 -- 172.21.15.185:0/802228865 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8142510 con 0x7f91c80afa00 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91ce4d1700 1 --2- 172.21.15.185:0/802228865 >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 0x7f91c80b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cd4cf700 1 --2- 172.21.15.185:0/802228865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b18d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f91c8063ba0 tx=0x7f91b8006eb0).ready entity=mon.1 client_cookie=4f56cf305b46ce server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cccce700 1 -- 172.21.15.185:0/802228865 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b80098d0 con 0x7f91c80afa00 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cccce700 1 -- 172.21.15.185:0/802228865 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8009a30 con 0x7f91c80afa00 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cccce700 1 -- 172.21.15.185:0/802228865 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8009db0 con 0x7f91c80afa00 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- 172.21.15.185:0/802228865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 msgr2=0x7f91c80b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 --2- 172.21.15.185:0/802228865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b18d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f91c8063ba0 tx=0x7f91b8006eb0).stop 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- 172.21.15.185:0/802228865 shutdown_connections 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 --2- 172.21.15.185:0/802228865 >> v2:172.21.15.96:3300/0 conn(0x7f91c80b5c00 0x7f91c80b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 --2- 172.21.15.185:0/802228865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b18d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- 172.21.15.185:0/802228865 >> 172.21.15.185:0/802228865 conn(0x7f91c8060130 msgr2=0x7f91c8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- 172.21.15.185:0/802228865 shutdown_connections 2020-11-16T22:17:38.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.647+0000 7f91cecd2700 1 -- 172.21.15.185:0/802228865 wait complete. 2020-11-16T22:17:38.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 Processor -- start 2020-11-16T22:17:38.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 -- start start 2020-11-16T22:17:38.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91c80b8830 0x7f91c8145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8152200 con 0x7f91c80b8830 2020-11-16T22:17:38.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c81524a0 con 0x7f91c80afa00 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cecd2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80ad570 con 0x7f91c80b5c00 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cd4cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cd4cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53298/0 (socket says 172.21.15.185:53298) 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cd4cf700 1 -- 172.21.15.185:0/2602820870 learned_addr learned my addr 172.21.15.185:0/2602820870 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cdcd0700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91ce4d1700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91c80b8830 0x7f91c8145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.651+0000 7f91cd4cf700 1 -- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 msgr2=0x7f91c80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cd4cf700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cd4cf700 1 -- 172.21.15.185:0/2602820870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91c80b8830 msgr2=0x7f91c8145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:38.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cd4cf700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91c80b8830 0x7f91c8145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:38.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cd4cf700 1 -- 172.21.15.185:0/2602820870 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80ad810 con 0x7f91c80b5c00 2020-11-16T22:17:38.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cd4cf700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f91b8006ef0 tx=0x7f91b8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:38.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8011d50 con 0x7f91c80b5c00 2020-11-16T22:17:38.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b801e460 con 0x7f91c80b5c00 2020-11-16T22:17:38.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c80b2030 con 0x7f91c80b5c00 2020-11-16T22:17:38.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b801e800 con 0x7f91c80b5c00 2020-11-16T22:17:38.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c80af790 con 0x7f91c80b5c00 2020-11-16T22:17:38.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c8152660 con 0x7f91c80b5c00 2020-11-16T22:17:38.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f91b8025030 con 0x7f91c80b5c00 2020-11-16T22:17:38.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91b5ffb700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 0x7f91a0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:38.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.655+0000 7f91cdcd0700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 0x7f91a0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:38.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.659+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f91b806f340 con 0x7f91c80b5c00 2020-11-16T22:17:38.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.659+0000 7f91cdcd0700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 0x7f91a0057520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f91c4003300 tx=0x7f91c4007b00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:39.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.663+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f91b80456c0 con 0x7f91c80b5c00 2020-11-16T22:17:39.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.863+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f91c805d200 con 0x7f91c80b5c00 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.875+0000 7f91b5ffb700 1 -- 172.21.15.185:0/2602820870 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v47) v1 ==== 188+0+0 (secure 0 0 0) 0x7f91b8023020 con 0x7f91c80b5c00 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.879+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 msgr2=0x7f91a0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.879+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 0x7f91a0057520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f91c4003300 tx=0x7f91c4007b00).stop 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.879+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 msgr2=0x7f91c8145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.879+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f91b8006ef0 tx=0x7f91b8015000).stop 2020-11-16T22:17:39.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 shutdown_connections 2020-11-16T22:17:39.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91a0055060 0x7f91a0057520 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0x7f91c4003300 tx=0x7f91c4007b00).stop 2020-11-16T22:17:39.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91c80afa00 0x7f91c80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91c80b8830 0x7f91c8145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 --2- 172.21.15.185:0/2602820870 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91c80b5c00 0x7f91c8145650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 >> 172.21.15.185:0/2602820870 conn(0x7f91c8060130 msgr2=0x7f91c8140a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:39.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.883+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 shutdown_connections 2020-11-16T22:17:39.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:38.891+0000 7f91cecd2700 1 -- 172.21.15.185:0/2602820870 wait complete. 2020-11-16T22:17:39.251 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 9 and data pool 10 2020-11-16T22:17:39.251 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:39.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.395+0000 7ff792df0700 1 Processor -- start 2020-11-16T22:17:39.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.395+0000 7ff792df0700 1 -- start start 2020-11-16T22:17:39.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 0x7ff78c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:39.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c061fe0 con 0x7ff78c0b3090 2020-11-16T22:17:39.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c0b0fe0 con 0x7ff78c0ad110 2020-11-16T22:17:39.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0b2b60 con 0x7ff78c0b1d30 2020-11-16T22:17:39.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7915ed700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 0x7ff78c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:39.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7925ef700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff78c0ad110 0x7ff78c1477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53300/0 (socket says 172.21.15.185:53300) 2020-11-16T22:17:39.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7925ef700 1 -- 172.21.15.185:0/681987733 learned_addr learned my addr 172.21.15.185:0/681987733 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:39.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7915ed700 1 -- 172.21.15.185:0/681987733 >> v1:172.21.15.185:6789/0 conn(0x7ff78c0ad110 legacy=0x7ff78c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff790dec700 1 -- 172.21.15.185:0/681987733 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139260740 0 0) 0x7ff78c0b0fe0 con 0x7ff78c0ad110 2020-11-16T22:17:39.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7915ed700 1 -- 172.21.15.185:0/681987733 >> v1:172.21.15.96:6789/0 conn(0x7ff78c0b3090 legacy=0x7ff78c14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7915ed700 1 -- 172.21.15.185:0/681987733 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0b1280 con 0x7ff78c0b1d30 2020-11-16T22:17:39.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff7915ed700 1 --2- 172.21.15.185:0/681987733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 0x7ff78c0b1720 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7ff77c009cc0 tx=0x7ff77c0091d0).ready entity=mon.2 client_cookie=6b102464b54ba89d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:39.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff790dec700 1 -- 172.21.15.185:0/681987733 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00bbc0 con 0x7ff78c0b1d30 2020-11-16T22:17:39.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff790dec700 1 -- 172.21.15.185:0/681987733 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00bd20 con 0x7ff78c0b1d30 2020-11-16T22:17:39.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff790dec700 1 -- 172.21.15.185:0/681987733 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011610 con 0x7ff78c0b1d30 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- 172.21.15.185:0/681987733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 msgr2=0x7ff78c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 --2- 172.21.15.185:0/681987733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 0x7ff78c0b1720 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7ff77c009cc0 tx=0x7ff77c0091d0).stop 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- 172.21.15.185:0/681987733 shutdown_connections 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 --2- 172.21.15.185:0/681987733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0b1d30 0x7ff78c0b1720 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- 172.21.15.185:0/681987733 >> 172.21.15.185:0/681987733 conn(0x7ff78c060130 msgr2=0x7ff78c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:39.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- 172.21.15.185:0/681987733 shutdown_connections 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- 172.21.15.185:0/681987733 wait complete. 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 Processor -- start 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.399+0000 7ff792df0700 1 -- start start 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 0x7ff78c10f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:39.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78c0b3090 0x7ff78c146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:39.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c111e70 con 0x7ff78c0b3090 2020-11-16T22:17:39.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c10fe70 con 0x7ff78c0b1d30 2020-11-16T22:17:39.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0b0080 con 0x7ff78c0ad110 2020-11-16T22:17:39.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59866/0 (socket says 172.21.15.185:59866) 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 -- 172.21.15.185:0/2085034769 learned_addr learned my addr 172.21.15.185:0/2085034769 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff791dee700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 0x7ff78c10f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7925ef700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78c0b3090 0x7ff78c146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 -- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 msgr2=0x7ff78c10f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 0x7ff78c10f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 -- 172.21.15.185:0/2085034769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78c0b3090 msgr2=0x7ff78c146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78c0b3090 0x7ff78c146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 -- 172.21.15.185:0/2085034769 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0b0320 con 0x7ff78c0b1d30 2020-11-16T22:17:39.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff791dee700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 0x7ff78c10f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:39.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff7915ed700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff77c009ea0 tx=0x7ff77c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:39.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c010800 con 0x7ff78c0b1d30 2020-11-16T22:17:39.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c0b05c0 con 0x7ff78c0b1d30 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c010960 con 0x7ff78c0b1d30 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c010d00 con 0x7ff78c0b1d30 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c0af0e0 con 0x7ff78c0b1d30 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff77c024030 con 0x7ff78c0b1d30 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 0x7ff768057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:39.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff77c06f900 con 0x7ff78c0b1d30 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff791dee700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 0x7ff768057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff791dee700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 0x7ff768057520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff7740010d0 tx=0x7ff774009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.403+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c14bf80 con 0x7ff78c0b1d30 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.411+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff77c015030 con 0x7ff78c0b1d30 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.559+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff78c05d200 con 0x7ff78c0b1d30 2020-11-16T22:17:39.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.563+0000 7ff789ffb700 1 -- 172.21.15.185:0/2085034769 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16841 (secure 0 0 0) 0x7ff77c02c030 con 0x7ff78c0b1d30 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 msgr2=0x7ff768057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 0x7ff768057520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff7740010d0 tx=0x7ff774009040).stop 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 msgr2=0x7ff78c0aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ff77c009ea0 tx=0x7ff77c016000).stop 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 shutdown_connections 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff768055060 0x7ff768057520 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78c0b3090 0x7ff78c146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff78c0ad110 0x7ff78c10f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 --2- 172.21.15.185:0/2085034769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff78c0b1d30 0x7ff78c0aea50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 >> 172.21.15.185:0/2085034769 conn(0x7ff78c060130 msgr2=0x7ff78c0b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 shutdown_connections 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:39.567+0000 7ff792df0700 1 -- 172.21.15.185:0/2085034769 wait complete. 2020-11-16T22:17:39.850 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:39.851 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":54,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:38.870002+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:40.059 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:40.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 Processor -- start 2020-11-16T22:17:40.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 -- start start 2020-11-16T22:17:40.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23600b3090 0x7f23600b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23600b1d30 0x7f236014b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23600b2b60 con 0x7f23600b1d30 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2360061fe0 con 0x7f23600b3090 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b0fe0 con 0x7f23600ad110 2020-11-16T22:17:40.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53322/0 (socket says 172.21.15.185:53322) 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235e59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23600b3090 0x7f23600b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 -- 172.21.15.185:0/502589779 learned_addr learned my addr 172.21.15.185:0/502589779 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235dd9b700 1 --2- 172.21.15.185:0/502589779 >> v2:172.21.15.96:3300/0 conn(0x7f23600b1d30 0x7f236014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 -- 172.21.15.185:0/502589779 >> v2:172.21.15.185:3300/0 conn(0x7f23600b3090 msgr2=0x7f23600b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 --2- 172.21.15.185:0/502589779 >> v2:172.21.15.185:3300/0 conn(0x7f23600b3090 0x7f23600b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 -- 172.21.15.185:0/502589779 >> v2:172.21.15.96:3300/0 conn(0x7f23600b1d30 msgr2=0x7f236014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 --2- 172.21.15.185:0/502589779 >> v2:172.21.15.96:3300/0 conn(0x7f23600b1d30 0x7f236014b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.207+0000 7f235ed9d700 1 -- 172.21.15.185:0/502589779 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600b1280 con 0x7f23600ad110 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235ed9d700 1 --2- 172.21.15.185:0/502589779 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f23600b2ec0 tx=0x7f2358009290).ready entity=mon.2 client_cookie=8c04da9bf5476043 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235d59a700 1 -- 172.21.15.185:0/502589779 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800bbc0 con 0x7f23600ad110 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235d59a700 1 -- 172.21.15.185:0/502589779 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235800bd20 con 0x7f23600ad110 2020-11-16T22:17:40.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235d59a700 1 -- 172.21.15.185:0/502589779 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358011610 con 0x7f23600ad110 2020-11-16T22:17:40.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- 172.21.15.185:0/502589779 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 msgr2=0x7f23601477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- 172.21.15.185:0/502589779 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f23600b2ec0 tx=0x7f2358009290).stop 2020-11-16T22:17:40.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- 172.21.15.185:0/502589779 shutdown_connections 2020-11-16T22:17:40.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- 172.21.15.185:0/502589779 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600ad110 0x7f23601477b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- 172.21.15.185:0/502589779 >> v2:172.21.15.185:3300/0 conn(0x7f23600b3090 0x7f23600b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- 172.21.15.185:0/502589779 >> v2:172.21.15.96:3300/0 conn(0x7f23600b1d30 0x7f236014b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- 172.21.15.185:0/502589779 >> 172.21.15.185:0/502589779 conn(0x7f2360060130 msgr2=0x7f2360060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- 172.21.15.185:0/502589779 shutdown_connections 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- 172.21.15.185:0/502589779 wait complete. 2020-11-16T22:17:40.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 Processor -- start 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- start start 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 0x7f236010f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360113800 con 0x7f23600b1d30 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601584d0 con 0x7f23600ad110 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23601132c0 con 0x7f23600b3090 2020-11-16T22:17:40.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.211+0000 7f235ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59880/0 (socket says 172.21.15.185:59880) 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 -- 172.21.15.185:0/2871555702 learned_addr learned my addr 172.21.15.185:0/2871555702 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53328/0 (socket says 172.21.15.185:53328) 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235dd9b700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 0x7f236010f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 -- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 msgr2=0x7f2360149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 -- 172.21.15.185:0/2871555702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 msgr2=0x7f236010f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 0x7f236010f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 -- 172.21.15.185:0/2871555702 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360113560 con 0x7f23600ad110 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235ed9d700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235dd9b700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 0x7f236010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:40.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235e59c700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f235000eba0 tx=0x7f2350006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000c8e0 con 0x7f23600ad110 2020-11-16T22:17:40.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235000ca40 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23601112d0 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000cdc0 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2360158210 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236005d200 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2350012600 con 0x7f23600ad110 2020-11-16T22:17:40.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 0x7f2334057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f235dd9b700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 0x7f2334057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.215+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f235006e080 con 0x7f23600ad110 2020-11-16T22:17:40.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.219+0000 7f235dd9b700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 0x7f2334057520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f2360148eb0 tx=0x7f2354006f90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.223+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2350044280 con 0x7f23600ad110 2020-11-16T22:17:40.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.379+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23600b2ca0 con 0x7f23600ad110 2020-11-16T22:17:40.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f234e7fc700 1 -- 172.21.15.185:0/2871555702 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16828 (secure 0 0 0) 0x7f23500478a0 con 0x7f23600ad110 2020-11-16T22:17:40.366 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:40.366 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:39.874438+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:40.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 msgr2=0x7f2334057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 0x7f2334057520 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f2360148eb0 tx=0x7f2354006f90).stop 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 msgr2=0x7f23601124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f235000eba0 tx=0x7f2350006df0).stop 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 shutdown_connections 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2334055060 0x7f2334057520 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23600ad110 0x7f23601124f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23600b3090 0x7f2360149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 --2- 172.21.15.185:0/2871555702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23600b1d30 0x7f236010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 >> 172.21.15.185:0/2871555702 conn(0x7f2360060130 msgr2=0x7f2360067ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:40.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 shutdown_connections 2020-11-16T22:17:40.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.383+0000 7f235f59e700 1 -- 172.21.15.185:0/2871555702 wait complete. 2020-11-16T22:17:40.380 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:17:40.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 Processor -- start 2020-11-16T22:17:40.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- start start 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa508063ca0 0x7fa5080b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa5080ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50814bc30 con 0x7fa5080b6bf0 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50814cb10 con 0x7fa508063ca0 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50814be90 con 0x7fa508149180 2020-11-16T22:17:40.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa5080ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d038700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa5080b6bf0 0x7fa508147cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53326/0 (socket says 172.21.15.185:53326) 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d038700 1 -- 172.21.15.185:0/2297735859 learned_addr learned my addr 172.21.15.185:0/2297735859 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50c837700 1 --2- 172.21.15.185:0/2297735859 >> v2:172.21.15.185:3300/0 conn(0x7fa508063ca0 0x7fa5080b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa4fffff700 1 -- 172.21.15.185:0/2297735859 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144737955 0 0) 0x7fa50814bc30 con 0x7fa5080b6bf0 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa4fffff700 1 -- 172.21.15.185:0/2297735859 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4ec003240 con 0x7fa5080b6bf0 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 -- 172.21.15.185:0/2297735859 >> v2:172.21.15.185:3300/0 conn(0x7fa508063ca0 msgr2=0x7fa5080b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:40.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 --2- 172.21.15.185:0/2297735859 >> v2:172.21.15.185:3300/0 conn(0x7fa508063ca0 0x7fa5080b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 -- 172.21.15.185:0/2297735859 >> v1:172.21.15.185:6789/0 conn(0x7fa5080b6bf0 legacy=0x7fa508147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 -- 172.21.15.185:0/2297735859 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50814c130 con 0x7fa508149180 2020-11-16T22:17:40.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50d839700 1 --2- 172.21.15.185:0/2297735859 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa5080ac4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fa500009c90 tx=0x7fa5000091d0).ready entity=mon.2 client_cookie=d3413cf2f30f606d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa4fffff700 1 -- 172.21.15.185:0/2297735859 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000b9c0 con 0x7fa508149180 2020-11-16T22:17:40.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa4fffff700 1 -- 172.21.15.185:0/2297735859 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000bb20 con 0x7fa508149180 2020-11-16T22:17:40.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa4fffff700 1 -- 172.21.15.185:0/2297735859 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000bea0 con 0x7fa508149180 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- 172.21.15.185:0/2297735859 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 msgr2=0x7fa5080ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 --2- 172.21.15.185:0/2297735859 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa5080ac4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fa500009c90 tx=0x7fa5000091d0).stop 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- 172.21.15.185:0/2297735859 shutdown_connections 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 --2- 172.21.15.185:0/2297735859 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa5080ac4c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 --2- 172.21.15.185:0/2297735859 >> v2:172.21.15.185:3300/0 conn(0x7fa508063ca0 0x7fa5080b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- 172.21.15.185:0/2297735859 >> 172.21.15.185:0/2297735859 conn(0x7fa5080657b0 msgr2=0x7fa508065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- 172.21.15.185:0/2297735859 shutdown_connections 2020-11-16T22:17:40.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.527+0000 7fa50f29c700 1 -- 172.21.15.185:0/2297735859 wait complete. 2020-11-16T22:17:40.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 Processor -- start 2020-11-16T22:17:40.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 -- start start 2020-11-16T22:17:40.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 0x7fa5080adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 0x7fa508143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508149dc0 con 0x7fa508063ca0 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508149ff0 con 0x7fa5080b6bf0 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50f29c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50814a290 con 0x7fa508149180 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d839700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d839700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53342/0 (socket says 172.21.15.185:53342) 2020-11-16T22:17:40.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d839700 1 -- 172.21.15.185:0/3275222890 learned_addr learned my addr 172.21.15.185:0/3275222890 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 0x7fa508143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d038700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 0x7fa5080adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 -- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 msgr2=0x7fa508143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 -- 172.21.15.185:0/3275222890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 msgr2=0x7fa5080adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 0x7fa5080adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 -- 172.21.15.185:0/3275222890 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50814a530 con 0x7fa5080b6bf0 2020-11-16T22:17:40.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d839700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50d038700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 0x7fa5080adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa50c837700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 0x7fa508143050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fa4f800ebd0 tx=0x7fa4f8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800c8e0 con 0x7fa5080b6bf0 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800ca40 con 0x7fa5080b6bf0 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.531+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800cdc0 con 0x7fa5080b6bf0 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa508143a30 con 0x7fa5080b6bf0 2020-11-16T22:17:40.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5080b0970 con 0x7fa5080b6bf0 2020-11-16T22:17:40.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5080692e0 con 0x7fa5080b6bf0 2020-11-16T22:17:40.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa4f8012600 con 0x7fa5080b6bf0 2020-11-16T22:17:40.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa4fd7fa700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 0x7fa4e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:40.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4f806e890 con 0x7fa5080b6bf0 2020-11-16T22:17:40.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa50d038700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 0x7fa4e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:40.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.535+0000 7fa50d038700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 0x7fa4e0057520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa5040055a0 tx=0x7fa504009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:40.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.543+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa4f8044bc0 con 0x7fa5080b6bf0 2020-11-16T22:17:40.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:40.743+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa5080692e0 con 0x7fa5080b6bf0 2020-11-16T22:17:41.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa4fd7fa700 1 -- 172.21.15.185:0/3275222890 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa5080692e0 con 0x7fa5080b6bf0 2020-11-16T22:17:41.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 msgr2=0x7fa4e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 0x7fa4e0057520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa5040055a0 tx=0x7fa504009040).stop 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 msgr2=0x7fa508143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 0x7fa508143050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fa4f800ebd0 tx=0x7fa4f8006df0).stop 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 shutdown_connections 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4e0055060 0x7fa4e0057520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa508149180 0x7fa508143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508063ca0 0x7fa5080adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 --2- 172.21.15.185:0/3275222890 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5080b6bf0 0x7fa508143050 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.307+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 >> 172.21.15.185:0/3275222890 conn(0x7fa5080657b0 msgr2=0x7fa5080b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:41.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.311+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 shutdown_connections 2020-11-16T22:17:41.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.311+0000 7fa50f29c700 1 -- 172.21.15.185:0/3275222890 wait complete. 2020-11-16T22:17:41.308 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:41.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 Processor -- start 2020-11-16T22:17:41.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 -- start start 2020-11-16T22:17:41.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f535c0669a0 0x7f535c066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0abf90 con 0x7f535c061e60 2020-11-16T22:17:41.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0ac230 con 0x7f535c0669a0 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.455+0000 7f53633d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0ac4d0 con 0x7f535c0acc10 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53350/0 (socket says 172.21.15.185:53350) 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 -- 172.21.15.185:0/2072918708 learned_addr learned my addr 172.21.15.185:0/2072918708 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53623d3700 1 --2- 172.21.15.185:0/2072918708 >> v2:172.21.15.96:3300/0 conn(0x7f535c0669a0 0x7f535c066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 -- 172.21.15.185:0/2072918708 >> v2:172.21.15.96:3300/0 conn(0x7f535c0669a0 msgr2=0x7f535c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:41.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 --2- 172.21.15.185:0/2072918708 >> v2:172.21.15.96:3300/0 conn(0x7f535c0669a0 0x7f535c066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 -- 172.21.15.185:0/2072918708 >> v1:172.21.15.96:6789/0 conn(0x7f535c061e60 legacy=0x7f535c147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 -- 172.21.15.185:0/2072918708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0ac770 con 0x7f535c0acc10 2020-11-16T22:17:41.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f5361bd2700 1 --2- 172.21.15.185:0/2072918708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5350006fd0 tx=0x7f5350006eb0).ready entity=mon.2 client_cookie=6f52b84f823bafea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53613d1700 1 -- 172.21.15.185:0/2072918708 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350009810 con 0x7f535c0acc10 2020-11-16T22:17:41.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53613d1700 1 -- 172.21.15.185:0/2072918708 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350009970 con 0x7f535c0acc10 2020-11-16T22:17:41.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53613d1700 1 -- 172.21.15.185:0/2072918708 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350009cf0 con 0x7f535c0acc10 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- 172.21.15.185:0/2072918708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 msgr2=0x7f535c147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 --2- 172.21.15.185:0/2072918708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f5350006fd0 tx=0x7f5350006eb0).stop 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- 172.21.15.185:0/2072918708 shutdown_connections 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 --2- 172.21.15.185:0/2072918708 >> v2:172.21.15.96:3300/0 conn(0x7f535c0669a0 0x7f535c066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 --2- 172.21.15.185:0/2072918708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c0acc10 0x7f535c147550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- 172.21.15.185:0/2072918708 >> 172.21.15.185:0/2072918708 conn(0x7f535c060130 msgr2=0x7f535c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:41.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- 172.21.15.185:0/2072918708 shutdown_connections 2020-11-16T22:17:41.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- 172.21.15.185:0/2072918708 wait complete. 2020-11-16T22:17:41.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 Processor -- start 2020-11-16T22:17:41.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.459+0000 7f53633d5700 1 -- start start 2020-11-16T22:17:41.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c061e60 0x7f535c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 0x7f535c147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c113ac0 con 0x7f535c0669a0 2020-11-16T22:17:41.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c13fbe0 con 0x7f535c147c20 2020-11-16T22:17:41.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0af2d0 con 0x7f535c061e60 2020-11-16T22:17:41.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59910/0 (socket says 172.21.15.185:59910) 2020-11-16T22:17:41.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 -- 172.21.15.185:0/3621513812 learned_addr learned my addr 172.21.15.185:0/3621513812 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53623d3700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c061e60 0x7f535c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5361bd2700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 0x7f535c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 -- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c061e60 msgr2=0x7f535c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c061e60 0x7f535c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 -- 172.21.15.185:0/3621513812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 msgr2=0x7f535c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 0x7f535c147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 -- 172.21.15.185:0/3621513812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0aea00 con 0x7f535c147c20 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5361bd2700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 0x7f535c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f5362bd4700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f535400ca90 tx=0x7f5354007e70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400a730 con 0x7f535c147c20 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400a890 con 0x7f535c147c20 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c113d50 con 0x7f535c147c20 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400ac30 con 0x7f535c147c20 2020-11-16T22:17:41.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c0b0790 con 0x7f535c147c20 2020-11-16T22:17:41.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.463+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c05d200 con 0x7f535c147c20 2020-11-16T22:17:41.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.467+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5354019030 con 0x7f535c147c20 2020-11-16T22:17:41.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.467+0000 7f535a7fc700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 0x7f5334057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.467+0000 7f53623d3700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 0x7f5334057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.467+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f535406d340 con 0x7f535c147c20 2020-11-16T22:17:41.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.467+0000 7f53623d3700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 0x7f5334057520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f534c006fd0 tx=0x7f534c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.475+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f53540435f0 con 0x7f535c147c20 2020-11-16T22:17:41.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.675+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f535c144f30 con 0x7f535c147c20 2020-11-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.679+0000 7f535a7fc700 1 -- 172.21.15.185:0/3621513812 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f5354017020 con 0x7f535c147c20 2020-11-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:41.661 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":45},{"gid":6054,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:41.297923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 msgr2=0x7f5334057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 0x7f5334057520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f534c006fd0 tx=0x7f534c006df0).stop 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 msgr2=0x7f535c115960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f535400ca90 tx=0x7f5354007e70).stop 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 shutdown_connections 2020-11-16T22:17:41.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5334055060 0x7f5334057520 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f535c147c20 0x7f535c115960 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f535c061e60 0x7f535c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 --2- 172.21.15.185:0/3621513812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f535c0669a0 0x7f535c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 >> 172.21.15.185:0/3621513812 conn(0x7f535c060130 msgr2=0x7f535c1402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 shutdown_connections 2020-11-16T22:17:41.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.683+0000 7f53633d5700 1 -- 172.21.15.185:0/3621513812 wait complete. 2020-11-16T22:17:41.667 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 50 2020-11-16T22:17:41.681 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:41.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 Processor -- start 2020-11-16T22:17:41.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- start start 2020-11-16T22:17:41.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7feaa00b3090 0x7feaa00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 0x7feaa014b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa0061fe0 con 0x7feaa00ad110 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa00b2b60 con 0x7feaa00b3090 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00b0fe0 con 0x7feaa00b1d30 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7feaa00ad110 0x7feaa01477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53358/0 (socket says 172.21.15.185:53358) 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9f7fe700 1 -- 172.21.15.185:0/2290826920 learned_addr learned my addr 172.21.15.185:0/2290826920 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 --2- 172.21.15.185:0/2290826920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 0x7feaa014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9effd700 1 --2- 172.21.15.185:0/2290826920 >> v2:172.21.15.96:3300/0 conn(0x7feaa00b3090 0x7feaa00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9dffb700 1 -- 172.21.15.185:0/2290826920 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194301814 0 0) 0x7feaa0061fe0 con 0x7feaa00ad110 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9dffb700 1 -- 172.21.15.185:0/2290826920 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea8c003240 con 0x7feaa00ad110 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 -- 172.21.15.185:0/2290826920 >> v2:172.21.15.96:3300/0 conn(0x7feaa00b3090 msgr2=0x7feaa00b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 --2- 172.21.15.185:0/2290826920 >> v2:172.21.15.96:3300/0 conn(0x7feaa00b3090 0x7feaa00b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 -- 172.21.15.185:0/2290826920 >> v1:172.21.15.185:6789/0 conn(0x7feaa00ad110 legacy=0x7feaa01477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 -- 172.21.15.185:0/2290826920 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00b1280 con 0x7feaa00b1d30 2020-11-16T22:17:41.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9e7fc700 1 --2- 172.21.15.185:0/2290826920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 0x7feaa014b950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fea9400b9a0 tx=0x7fea94006df0).ready entity=mon.2 client_cookie=d73f907bbf5221e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9dffb700 1 -- 172.21.15.185:0/2290826920 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94009680 con 0x7feaa00b1d30 2020-11-16T22:17:41.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9dffb700 1 -- 172.21.15.185:0/2290826920 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea940097e0 con 0x7feaa00b1d30 2020-11-16T22:17:41.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9dffb700 1 -- 172.21.15.185:0/2290826920 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94009b60 con 0x7feaa00b1d30 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- 172.21.15.185:0/2290826920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 msgr2=0x7feaa014b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 --2- 172.21.15.185:0/2290826920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 0x7feaa014b950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fea9400b9a0 tx=0x7fea94006df0).stop 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- 172.21.15.185:0/2290826920 shutdown_connections 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 --2- 172.21.15.185:0/2290826920 >> v2:172.21.15.96:3300/0 conn(0x7feaa00b3090 0x7feaa00b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 --2- 172.21.15.185:0/2290826920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b1d30 0x7feaa014b950 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- 172.21.15.185:0/2290826920 >> 172.21.15.185:0/2290826920 conn(0x7feaa0060130 msgr2=0x7feaa0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:41.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- 172.21.15.185:0/2290826920 shutdown_connections 2020-11-16T22:17:41.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.819+0000 7fea9ffff700 1 -- 172.21.15.185:0/2290826920 wait complete. 2020-11-16T22:17:41.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.823+0000 7fea9ffff700 1 Processor -- start 2020-11-16T22:17:41.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.823+0000 7fea9ffff700 1 -- start start 2020-11-16T22:17:41.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 0x7feaa0145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 0x7feaa0112090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 0x7feaa0112580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 0x7feaa0145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0110e10 con 0x7feaa00b1d30 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa014bac0 con 0x7feaa00ad110 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa014bcf0 con 0x7feaa00b3090 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 0x7feaa0145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59922/0 (socket says 172.21.15.185:59922) 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9effd700 1 -- 172.21.15.185:0/3057162846 learned_addr learned my addr 172.21.15.185:0/3057162846 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 0x7feaa0112580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9e7fc700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 0x7feaa0112090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 -- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 msgr2=0x7feaa0145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 0x7feaa0145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 -- 172.21.15.185:0/3057162846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 msgr2=0x7feaa0112090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 0x7feaa0112090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 -- 172.21.15.185:0/3057162846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa014bf90 con 0x7feaa00b3090 2020-11-16T22:17:41.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9e7fc700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 0x7feaa0112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9f7fe700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 0x7feaa0112580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fea9800ca90 tx=0x7fea98007ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98005b80 con 0x7feaa00b3090 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98005ce0 con 0x7feaa00b3090 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa0064c40 con 0x7feaa00b3090 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98007320 con 0x7feaa00b3090 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa014a460 con 0x7feaa00b3090 2020-11-16T22:17:41.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa0113820 con 0x7feaa00b3090 2020-11-16T22:17:41.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.827+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fea98010030 con 0x7feaa00b3090 2020-11-16T22:17:41.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.831+0000 7fea86ffd700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 0x7fea7c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:41.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.831+0000 7fea9effd700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 0x7fea7c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:41.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.831+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea9806dcb0 con 0x7feaa00b3090 2020-11-16T22:17:41.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.831+0000 7fea9effd700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 0x7fea7c057520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fea90006a90 tx=0x7fea90006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:41.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.839+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fea98043eb0 con 0x7feaa00b3090 2020-11-16T22:17:41.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaa014c120 con 0x7feaa00b3090 2020-11-16T22:17:41.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea86ffd700 1 -- 172.21.15.185:0/3057162846 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16828 (secure 0 0 0) 0x7fea980474d0 con 0x7feaa00b3090 2020-11-16T22:17:41.971 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:41.971 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:39.874438+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:41.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 msgr2=0x7fea7c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 0x7fea7c057520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fea90006a90 tx=0x7fea90006df0).stop 2020-11-16T22:17:41.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 msgr2=0x7feaa0112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:41.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.987+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 0x7feaa0112580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fea9800ca90 tx=0x7fea98007ca0).stop 2020-11-16T22:17:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 shutdown_connections 2020-11-16T22:17:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea7c055060 0x7fea7c057520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7feaa00ad110 0x7feaa0145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7feaa00b3090 0x7feaa0112580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 --2- 172.21.15.185:0/3057162846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feaa00b1d30 0x7feaa0112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 >> 172.21.15.185:0/3057162846 conn(0x7feaa0060130 msgr2=0x7feaa00b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 shutdown_connections 2020-11-16T22:17:41.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:41.991+0000 7fea9ffff700 1 -- 172.21.15.185:0/3057162846 wait complete. 2020-11-16T22:17:41.985 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:17:42.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 Processor -- start 2020-11-16T22:17:42.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 -- start start 2020-11-16T22:17:42.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f77000afbb0 0x7f77000616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f77000b55e0 0x7f7700144af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000b7310 con 0x7f77001436c0 2020-11-16T22:17:42.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7700144200 con 0x7f77000afbb0 2020-11-16T22:17:42.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f7705643700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77001443f0 con 0x7f77000b55e0 2020-11-16T22:17:42.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f77001436c0 0x7f7700143af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53370/0 (socket says 172.21.15.185:53370) 2020-11-16T22:17:42.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2129744648 learned_addr learned my addr 172.21.15.185:0/2129744648 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:42.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fe7fc700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.185:3300/0 conn(0x7f77000b55e0 0x7f7700144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76feffd700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.96:3300/0 conn(0x7f77000afbb0 0x7f77000616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878338135 0 0) 0x7f77000b7310 con 0x7f77001436c0 2020-11-16T22:17:42.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ec003240 con 0x7f77001436c0 2020-11-16T22:17:42.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2142080139 0 0) 0x7f76ec003240 con 0x7f77001436c0 2020-11-16T22:17:42.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 >> v2:172.21.15.185:3300/0 conn(0x7f77000b55e0 msgr2=0x7f7700144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.185:3300/0 conn(0x7f77000b55e0 0x7f7700144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 >> v2:172.21.15.96:3300/0 conn(0x7f77000afbb0 msgr2=0x7f77000616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.96:3300/0 conn(0x7f77000afbb0 0x7f77000616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000b6fd0 con 0x7f77001436c0 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f76f8001860 con 0x7f77001436c0 2020-11-16T22:17:42.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 >> v1:172.21.15.185:6789/0 conn(0x7f77001436c0 legacy=0x7f7700143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec003730 0x7f76ec003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f76ec0044e0 0x7f76ec004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 0x7f76ec00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001443f0 con 0x7f76ec003730 2020-11-16T22:17:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec004260 con 0x7f76ec0044e0 2020-11-16T22:17:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec00bc20 con 0x7f76ec004e50 2020-11-16T22:17:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fe7fc700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.185:3300/0 conn(0x7f77000b55e0 0x7f7700144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76feffd700 1 --2- 172.21.15.185:0/2129744648 >> v2:172.21.15.96:3300/0 conn(0x7f77000afbb0 0x7f77000616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 0x7f76ec00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76fe7fc700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f76ec0044e0 0x7f76ec004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f76ec0044e0 msgr2=0x7f76ec004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f76ec0044e0 0x7f76ec004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2129744648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec003730 msgr2=0x7f76ec003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec003730 0x7f76ec003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:17:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.135+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2129744648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec003240 con 0x7f76ec004e50 2020-11-16T22:17:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76fe7fc700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f76ec0044e0 0x7f76ec004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:42.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2129744648 reap_dead start 2020-11-16T22:17:42.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 0x7f76ec00b730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f76f800c6a0 tx=0x7f76f8007ad0).ready entity=mon.2 client_cookie=fff5f3ae58ce94a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:42.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8005b80 con 0x7f76ec004e50 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8005ce0 con 0x7f76ec004e50 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76fdffb700 1 -- 172.21.15.185:0/2129744648 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8007360 con 0x7f76ec004e50 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 msgr2=0x7f76ec00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 0x7f76ec00b730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f76f800c6a0 tx=0x7f76f8007ad0).stop 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- 172.21.15.185:0/2129744648 shutdown_connections 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 --2- 172.21.15.185:0/2129744648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f76ec004e50 0x7f76ec00b730 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- 172.21.15.185:0/2129744648 >> 172.21.15.185:0/2129744648 conn(0x7f77000baab0 msgr2=0x7f77000baec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:42.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- 172.21.15.185:0/2129744648 shutdown_connections 2020-11-16T22:17:42.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- 172.21.15.185:0/2129744648 wait complete. 2020-11-16T22:17:42.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 Processor -- start 2020-11-16T22:17:42.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- start start 2020-11-16T22:17:42.535 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:17:42.186+0000 7f4f6a19b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 0x7f77001403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77001568c0 0x7f770013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770009a630 con 0x7f76ec004e50 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7700156d60 con 0x7f7700145260 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f7705643700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7700157000 con 0x7f77001568c0 2020-11-16T22:17:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59946/0 (socket says 172.21.15.185:59946) 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76fe7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77001568c0 0x7f770013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2934994080 learned_addr learned my addr 172.21.15.185:0/2934994080 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76feffd700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 0x7f77001403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77001568c0 msgr2=0x7f770013e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77001568c0 0x7f770013e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2934994080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 msgr2=0x7f77001403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 0x7f77001403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 -- 172.21.15.185:0/2934994080 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77001572a0 con 0x7f7700145260 2020-11-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76feffd700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 0x7f77001403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76ff7fe700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f76f8009000 tx=0x7f76f8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.139+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8001760 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80018c0 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8004ab0 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001408b0 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770009a1d0 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770005d4e0 con 0x7f7700145260 2020-11-16T22:17:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f76f8025030 con 0x7f7700145260 2020-11-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76e6ffd700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 0x7f76dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76feffd700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 0x7f76dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f76f806f1b0 con 0x7f7700145260 2020-11-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.143+0000 7f76feffd700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 0x7f76dc057520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f76f0006e80 tx=0x7f76f0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.151+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f76f80454c0 con 0x7f7700145260 2020-11-16T22:17:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.363+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f7700060130 con 0x7f7700145260 2020-11-16T22:17:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f76e6ffd700 1 -- 172.21.15.185:0/2934994080 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f76f8023070 con 0x7f7700145260 2020-11-16T22:17:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 msgr2=0x7f76dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 0x7f76dc057520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f76f0006e80 tx=0x7f76f0008040).stop 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 msgr2=0x7f770013edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f76f8009000 tx=0x7f76f8016000).stop 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 shutdown_connections 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f76dc055060 0x7f76dc057520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7700145260 0x7f770013edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec004e50 0x7f77001403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 --2- 172.21.15.185:0/2934994080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77001568c0 0x7f770013e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 >> 172.21.15.185:0/2934994080 conn(0x7f77000baab0 msgr2=0x7f77000b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 shutdown_connections 2020-11-16T22:17:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:42.375+0000 7f7705643700 1 -- 172.21.15.185:0/2934994080 wait complete. 2020-11-16T22:17:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:17:43.151 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:43.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 Processor -- start 2020-11-16T22:17:43.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 -- start start 2020-11-16T22:17:43.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 0x7fa55c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c0b82e0 con 0x7fa55c0b4720 2020-11-16T22:17:43.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c0b7650 con 0x7fa55c144b90 2020-11-16T22:17:43.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa56253f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0b78f0 con 0x7fa55c12cca0 2020-11-16T22:17:43.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa561d3e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa561d3e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53398/0 (socket says 172.21.15.185:53398) 2020-11-16T22:17:43.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa561d3e700 1 -- 172.21.15.185:0/2657943962 learned_addr learned my addr 172.21.15.185:0/2657943962 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:43.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.299+0000 7fa560d3c700 1 --2- 172.21.15.185:0/2657943962 >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 0x7fa55c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa561d3e700 1 -- 172.21.15.185:0/2657943962 >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 msgr2=0x7fa55c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:43.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa561d3e700 1 --2- 172.21.15.185:0/2657943962 >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 0x7fa55c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa561d3e700 1 -- 172.21.15.185:0/2657943962 >> v1:172.21.15.96:6789/0 conn(0x7fa55c0b4720 legacy=0x7fa55c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa561d3e700 1 -- 172.21.15.185:0/2657943962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0b1ee0 con 0x7fa55c12cca0 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa560d3c700 1 --2- 172.21.15.185:0/2657943962 >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 0x7fa55c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa561d3e700 1 --2- 172.21.15.185:0/2657943962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa554009d50 tx=0x7fa554009290).ready entity=mon.2 client_cookie=e416cf419aaa2988 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa547fff700 1 -- 172.21.15.185:0/2657943962 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55400bc80 con 0x7fa55c12cca0 2020-11-16T22:17:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa547fff700 1 -- 172.21.15.185:0/2657943962 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55400bde0 con 0x7fa55c12cca0 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa547fff700 1 -- 172.21.15.185:0/2657943962 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554011610 con 0x7fa55c12cca0 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- 172.21.15.185:0/2657943962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 msgr2=0x7fa55c12d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 --2- 172.21.15.185:0/2657943962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa554009d50 tx=0x7fa554009290).stop 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- 172.21.15.185:0/2657943962 shutdown_connections 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 --2- 172.21.15.185:0/2657943962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c12cca0 0x7fa55c12d130 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 --2- 172.21.15.185:0/2657943962 >> v2:172.21.15.96:3300/0 conn(0x7fa55c144b90 0x7fa55c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- 172.21.15.185:0/2657943962 >> 172.21.15.185:0/2657943962 conn(0x7fa55c067860 msgr2=0x7fa55c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- 172.21.15.185:0/2657943962 shutdown_connections 2020-11-16T22:17:43.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- 172.21.15.185:0/2657943962 wait complete. 2020-11-16T22:17:43.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 Processor -- start 2020-11-16T22:17:43.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.303+0000 7fa56253f700 1 -- start start 2020-11-16T22:17:43.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 0x7fa55c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 0x7fa55c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 0x7fa55c156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0af130 con 0x7fa55c12cca0 2020-11-16T22:17:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0af3d0 con 0x7fa55c0b4720 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56253f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0af670 con 0x7fa55c144b90 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa561d3e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 0x7fa55c156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56153d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 0x7fa55c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 0x7fa55c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa561d3e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 0x7fa55c156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53404/0 (socket says 172.21.15.185:53404) 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa56153d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 0x7fa55c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59956/0 (socket says 172.21.15.185:59956) 2020-11-16T22:17:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa561d3e700 1 -- 172.21.15.185:0/2365936474 learned_addr learned my addr 172.21.15.185:0/2365936474 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 -- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 msgr2=0x7fa55c156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 0x7fa55c156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 -- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 msgr2=0x7fa55c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 0x7fa55c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 -- 172.21.15.185:0/2365936474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0af910 con 0x7fa55c12cca0 2020-11-16T22:17:43.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.307+0000 7fa560d3c700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 0x7fa55c156ad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fa55000db60 tx=0x7fa550008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:43.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000de70 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550009790 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0b5d80 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550009b50 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c147ff0 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c05d200 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.311+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa550013480 con 0x7fa55c12cca0 2020-11-16T22:17:43.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.315+0000 7fa54dffb700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 0x7fa530057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:43.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.315+0000 7fa56153d700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 0x7fa530057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:43.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.315+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa55006e060 con 0x7fa55c12cca0 2020-11-16T22:17:43.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.315+0000 7fa56153d700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 0x7fa530057520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fa558003230 tx=0x7fa558000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:43.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.323+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa550044260 con 0x7fa55c12cca0 2020-11-16T22:17:43.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.527+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa55c148540 con 0x7fa55c12cca0 2020-11-16T22:17:43.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.527+0000 7fa54dffb700 1 -- 172.21.15.185:0/2365936474 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fa550006a60 con 0x7fa55c12cca0 2020-11-16T22:17:43.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 msgr2=0x7fa530057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 0x7fa530057520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fa558003230 tx=0x7fa558000b80).stop 2020-11-16T22:17:43.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 msgr2=0x7fa55c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 0x7fa55c156ad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fa55000db60 tx=0x7fa550008040).stop 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 shutdown_connections 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa530055060 0x7fa530057520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa55c0b4720 0x7fa55c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa55c144b90 0x7fa55c156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 --2- 172.21.15.185:0/2365936474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa55c12cca0 0x7fa55c156ad0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:43.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 >> 172.21.15.185:0/2365936474 conn(0x7fa55c067860 msgr2=0x7fa55c0b98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:43.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 shutdown_connections 2020-11-16T22:17:43.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:43.531+0000 7fa56253f700 1 -- 172.21.15.185:0/2365936474 wait complete. 2020-11-16T22:17:43.906 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 50 2020-11-16T22:17:43.906 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:43.907 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":45},{"gid":6054,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:41.297923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:44.209 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:17:38.870443+0000', 'modified': '2020-11-16T22:17:41.297923+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6057': {'gid': 6057, 'name': 'b', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6834/2034361528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2034361528}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 2034361528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:17:44.210 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:17:44.210 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:17:44.352 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6057,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.4910945050000004,"mdsmap_epoch":50,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":12.119231920000001} 2020-11-16T22:17:44.353 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6057, "mdsmap_epoch": 50, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 5.491094505, "state": "up:active", "uptime": 12.11923192, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:17:44.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:17:44.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:17:44.354 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:17:44.354 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:17:44.354 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:17:44.354 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:17:44.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:17:44.355 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:17:44.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.356 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:17:44.374 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:17:44.374 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:17:44.375 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:17:44.376 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.377 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:17:44.378 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:17:44.379 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:17:44.379 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:17:44.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.380 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:17:44.415 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:17:44.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.419 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:17:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:17:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:17:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:17:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:17:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:17:44.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.436 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:17:44.436 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:17:44.437 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:17:44.437 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:17:44.437 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:17:44.438 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:17:44.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.581 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:17:44.595 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:17:44.595 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:17:44.596 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:17:44.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:44.597 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:17:44.625 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:17:44.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 Processor -- start 2020-11-16T22:17:44.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- start start 2020-11-16T22:17:44.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8fa40b64b0 0x7f8fa40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fa40b6aa0 0x7f8fa40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa4147f90 con 0x7f8fa40b72e0 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa40b6eb0 con 0x7f8fa40b64b0 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa40af020 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fa40b6aa0 0x7f8fa40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8fa40b6aa0 0x7f8fa40acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59968/0 (socket says 172.21.15.185:59968) 2020-11-16T22:17:44.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 -- 172.21.15.185:0/2563305651 learned_addr learned my addr 172.21.15.185:0/2563305651 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:44.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fa980d700 1 --2- 172.21.15.185:0/2563305651 >> v2:172.21.15.96:3300/0 conn(0x7f8fa40b64b0 0x7f8fa40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 -- 172.21.15.185:0/2563305651 >> v2:172.21.15.96:3300/0 conn(0x7f8fa40b64b0 msgr2=0x7f8fa40b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:44.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 --2- 172.21.15.185:0/2563305651 >> v2:172.21.15.96:3300/0 conn(0x7f8fa40b64b0 0x7f8fa40b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 -- 172.21.15.185:0/2563305651 >> v1:172.21.15.96:6789/0 conn(0x7f8fa40b72e0 legacy=0x7f8fa4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 -- 172.21.15.185:0/2563305651 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4148460 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8faa80f700 1 --2- 172.21.15.185:0/2563305651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40acee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f8fa40b5170 tx=0x7f8f9c009290).ready entity=mon.1 client_cookie=1ecf8f489e52275c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fa900c700 1 -- 172.21.15.185:0/2563305651 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00bbc0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fa900c700 1 -- 172.21.15.185:0/2563305651 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c00bd20 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fa900c700 1 -- 172.21.15.185:0/2563305651 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c011610 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- 172.21.15.185:0/2563305651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 msgr2=0x7f8fa40acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 --2- 172.21.15.185:0/2563305651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40acee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f8fa40b5170 tx=0x7f8f9c009290).stop 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- 172.21.15.185:0/2563305651 shutdown_connections 2020-11-16T22:17:44.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 --2- 172.21.15.185:0/2563305651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40acee0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 --2- 172.21.15.185:0/2563305651 >> v2:172.21.15.96:3300/0 conn(0x7f8fa40b64b0 0x7f8fa40b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- 172.21.15.185:0/2563305651 >> 172.21.15.185:0/2563305651 conn(0x7f8fa4067860 msgr2=0x7f8fa4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:44.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- 172.21.15.185:0/2563305651 shutdown_connections 2020-11-16T22:17:44.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.771+0000 7f8fab010700 1 -- 172.21.15.185:0/2563305651 wait complete. 2020-11-16T22:17:44.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 Processor -- start 2020-11-16T22:17:44.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- start start 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fa40b64b0 0x7f8fa41430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 0x7f8fa40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa413f1d0 con 0x7f8fa40b72e0 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa413eba0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4140ed0 con 0x7f8fa40b64b0 2020-11-16T22:17:44.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59970/0 (socket says 172.21.15.185:59970) 2020-11-16T22:17:44.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 -- 172.21.15.185:0/4239400230 learned_addr learned my addr 172.21.15.185:0/4239400230 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:44.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8faa00e700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fa40b64b0 0x7f8fa41430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8faa80f700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 0x7f8fa40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 -- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fa40b64b0 msgr2=0x7f8fa41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fa40b64b0 0x7f8fa41430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 -- 172.21.15.185:0/4239400230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 msgr2=0x7f8fa40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 0x7f8fa40b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 -- 172.21.15.185:0/4239400230 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa413ee00 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8faa80f700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 0x7f8fa40b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:44.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fa980d700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f8f9400eba0 tx=0x7f8f94006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:44.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400c9b0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400cb10 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400ce90 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- 172.21.15.185:0/4239400230 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa413e6c0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.775+0000 7f8fab010700 1 -- 172.21.15.185:0/4239400230 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa41435d0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8f94012600 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa40b8cc0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8f927fc700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 0x7f8f7c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:44.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8f9406e200 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8faa00e700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 0x7f8f7c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:44.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.779+0000 7f8faa00e700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 0x7f8f7c057520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f8fa0000bc0 tx=0x7f8fa0007b40).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:44.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.787+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8fa40b8cc0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.935+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fa40b8cc0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.935+0000 7f8f927fc700 1 -- 172.21.15.185:0/4239400230 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16828 (secure 0 0 0) 0x7f8fa40b8cc0 con 0x7f8fa40b6aa0 2020-11-16T22:17:44.920 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:44.921 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:39.874438+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:17:44.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 msgr2=0x7f8f7c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 0x7f8f7c057520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f8fa0000bc0 tx=0x7f8fa0007b40).stop 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 msgr2=0x7f8fa40b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f8f9400eba0 tx=0x7f8f94006df0).stop 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 shutdown_connections 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8f7c055060 0x7f8f7c057520 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa40b72e0 0x7f8fa40b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8fa40b64b0 0x7f8fa41430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 --2- 172.21.15.185:0/4239400230 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8fa40b6aa0 0x7f8fa40b0660 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:44.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 >> 172.21.15.185:0/4239400230 conn(0x7f8fa4067860 msgr2=0x7f8fa40adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:44.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 shutdown_connections 2020-11-16T22:17:44.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:44.939+0000 7f8f83fff700 1 -- 172.21.15.185:0/4239400230 wait complete. 2020-11-16T22:17:44.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:17:36.945221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:17:37.680544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:17:44.935 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:17:45.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.079+0000 7f8a49455700 1 Processor -- start 2020-11-16T22:17:45.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.079+0000 7f8a49455700 1 -- start start 2020-11-16T22:17:45.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8a440b3830 0x7f8a44143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a440ac820 0x7f8a44143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a44060130 0x7f8a4413f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a44060970 con 0x7f8a44060130 2020-11-16T22:17:45.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a44060e40 con 0x7f8a440b3830 2020-11-16T22:17:45.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440b4610 con 0x7f8a440ac820 2020-11-16T22:17:45.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8a440b3830 0x7f8a44143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a3b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a440ac820 0x7f8a44143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8a440b3830 0x7f8a44143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59978/0 (socket says 172.21.15.185:59978) 2020-11-16T22:17:45.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 -- 172.21.15.185:0/2560809583 learned_addr learned my addr 172.21.15.185:0/2560809583 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a48c54700 1 --2- 172.21.15.185:0/2560809583 >> v2:172.21.15.96:3300/0 conn(0x7f8a44060130 0x7f8a4413f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 -- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a440ac820 msgr2=0x7f8a44143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 --2- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a440ac820 0x7f8a44143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 -- 172.21.15.185:0/2560809583 >> v2:172.21.15.96:3300/0 conn(0x7f8a44060130 msgr2=0x7f8a4413f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 --2- 172.21.15.185:0/2560809583 >> v2:172.21.15.96:3300/0 conn(0x7f8a44060130 0x7f8a4413f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 -- 172.21.15.185:0/2560809583 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44060ba0 con 0x7f8a440b3830 2020-11-16T22:17:45.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a43fff700 1 --2- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440b3830 0x7f8a44143a40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f8a440b7d40 tx=0x7f8a34009290).ready entity=mon.1 client_cookie=8c3cd4312a2cc48e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a437fe700 1 -- 172.21.15.185:0/2560809583 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400bb10 con 0x7f8a440b3830 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a437fe700 1 -- 172.21.15.185:0/2560809583 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3400bc70 con 0x7f8a440b3830 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a437fe700 1 -- 172.21.15.185:0/2560809583 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34010410 con 0x7f8a440b3830 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440b3830 msgr2=0x7f8a44143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440b3830 0x7f8a44143a40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f8a440b7d40 tx=0x7f8a34009290).stop 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- 172.21.15.185:0/2560809583 shutdown_connections 2020-11-16T22:17:45.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- 172.21.15.185:0/2560809583 >> v2:172.21.15.96:3300/0 conn(0x7f8a44060130 0x7f8a4413f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440b3830 0x7f8a44143a40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 --2- 172.21.15.185:0/2560809583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a440ac820 0x7f8a44143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- 172.21.15.185:0/2560809583 >> 172.21.15.185:0/2560809583 conn(0x7f8a44063df0 msgr2=0x7f8a440641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- 172.21.15.185:0/2560809583 shutdown_connections 2020-11-16T22:17:45.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.083+0000 7f8a49455700 1 -- 172.21.15.185:0/2560809583 wait complete. 2020-11-16T22:17:45.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.087+0000 7f8a49455700 1 Processor -- start 2020-11-16T22:17:45.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.087+0000 7f8a49455700 1 -- start start 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.087+0000 7f8a49455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 0x7f8a44137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44155a80 con 0x7f8a440b3830 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44110400 con 0x7f8a440ac820 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440b1270 con 0x7f8a44060130 2020-11-16T22:17:45.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a3b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53428/0 (socket says 172.21.15.185:53428) 2020-11-16T22:17:45.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a3b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:59986/0 (socket says 172.21.15.185:59986) 2020-11-16T22:17:45.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 -- 172.21.15.185:0/3144708857 learned_addr learned my addr 172.21.15.185:0/3144708857 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a48c54700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 0x7f8a44137840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 -- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 msgr2=0x7f8a440ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 -- 172.21.15.185:0/3144708857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 msgr2=0x7f8a44137840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 0x7f8a44137840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 -- 172.21.15.185:0/3144708857 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4410ff60 con 0x7f8a44060130 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a48c54700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 0x7f8a44137840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a3b7fe700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:45.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a43fff700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8a3400c010 tx=0x7f8a34017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401a450 con 0x7f8a44060130 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3401e070 con 0x7f8a44060130 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a44144150 con 0x7f8a44060130 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401a5b0 con 0x7f8a44060130 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a440b0ee0 con 0x7f8a44060130 2020-11-16T22:17:45.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a440afa50 con 0x7f8a44060130 2020-11-16T22:17:45.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.091+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8a3402a070 con 0x7f8a44060130 2020-11-16T22:17:45.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.095+0000 7f8a40ff9700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 0x7f8a20057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.095+0000 7f8a3b7fe700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 0x7f8a20057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.095+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8a3406f5c0 con 0x7f8a44060130 2020-11-16T22:17:45.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.095+0000 7f8a3b7fe700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 0x7f8a20057520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8a440b14a0 tx=0x7f8a2c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.099+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8a34045820 con 0x7f8a44060130 2020-11-16T22:17:45.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.247+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8a4405d200 con 0x7f8a44060130 2020-11-16T22:17:45.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.247+0000 7f8a40ff9700 1 -- 172.21.15.185:0/3144708857 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8a34048e40 con 0x7f8a44060130 2020-11-16T22:17:45.231 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:17:45.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 msgr2=0x7f8a20057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 0x7f8a20057520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8a440b14a0 tx=0x7f8a2c006df0).stop 2020-11-16T22:17:45.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 msgr2=0x7f8a441437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f8a3400c010 tx=0x7f8a34017000).stop 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 shutdown_connections 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8a20055060 0x7f8a20057520 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a440b3830 0x7f8a44137840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8a44060130 0x7f8a441437f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 --2- 172.21.15.185:0/3144708857 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8a440ac820 0x7f8a440ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 >> 172.21.15.185:0/3144708857 conn(0x7f8a44063df0 msgr2=0x7f8a440641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 shutdown_connections 2020-11-16T22:17:45.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.251+0000 7f8a49455700 1 -- 172.21.15.185:0/3144708857 wait complete. 2020-11-16T22:17:45.246 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:17:45.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.387+0000 7f2cfffff700 1 Processor -- start 2020-11-16T22:17:45.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.387+0000 7f2cfffff700 1 -- start start 2020-11-16T22:17:45.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2d000b2e80 0x7f2d000b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 0x7f2d00143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d000ac070 con 0x7f2d0013dcf0 2020-11-16T22:17:45.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d000ac310 con 0x7f2d000b2e80 2020-11-16T22:17:45.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000ac5b0 con 0x7f2d000b3920 2020-11-16T22:17:45.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfe7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 0x7f2d00143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfe7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 0x7f2d00143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53438/0 (socket says 172.21.15.185:53438) 2020-11-16T22:17:45.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cff7fe700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2d0013dcf0 0x7f2d000b16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:39498/0 (socket says 172.21.15.185:39498) 2020-11-16T22:17:45.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfe7fc700 1 -- 172.21.15.185:0/725967465 learned_addr learned my addr 172.21.15.185:0/725967465 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/725967465 >> v2:172.21.15.96:3300/0 conn(0x7f2d000b2e80 0x7f2d000b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 -- 172.21.15.185:0/725967465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 msgr2=0x7f2d00143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/725967465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 0x7f2d00143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 -- 172.21.15.185:0/725967465 >> v1:172.21.15.96:6789/0 conn(0x7f2d0013dcf0 legacy=0x7f2d000b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 -- 172.21.15.185:0/725967465 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000ac850 con 0x7f2d000b2e80 2020-11-16T22:17:45.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/725967465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b2e80 0x7f2d000b3260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f2cf000b9a0 tx=0x7f2cf0006df0).ready entity=mon.0 client_cookie=9184a9bef3382f05 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfdffb700 1 -- 172.21.15.185:0/725967465 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00097b0 con 0x7f2d000b2e80 2020-11-16T22:17:45.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfdffb700 1 -- 172.21.15.185:0/725967465 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0009910 con 0x7f2d000b2e80 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfdffb700 1 -- 172.21.15.185:0/725967465 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0009c90 con 0x7f2d000b2e80 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- 172.21.15.185:0/725967465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b2e80 msgr2=0x7f2d000b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 --2- 172.21.15.185:0/725967465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b2e80 0x7f2d000b3260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f2cf000b9a0 tx=0x7f2cf0006df0).stop 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- 172.21.15.185:0/725967465 shutdown_connections 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 --2- 172.21.15.185:0/725967465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b2e80 0x7f2d000b3260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 --2- 172.21.15.185:0/725967465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b3920 0x7f2d00143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- 172.21.15.185:0/725967465 >> 172.21.15.185:0/725967465 conn(0x7f2d00060130 msgr2=0x7f2d00060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- 172.21.15.185:0/725967465 shutdown_connections 2020-11-16T22:17:45.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.391+0000 7f2cfffff700 1 -- 172.21.15.185:0/725967465 wait complete. 2020-11-16T22:17:45.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 Processor -- start 2020-11-16T22:17:45.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- start start 2020-11-16T22:17:45.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 0x7f2d001400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d0013dcf0 0x7f2d00156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0013fc70 con 0x7f2d000b3920 2020-11-16T22:17:45.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d001460a0 con 0x7f2d0013dcf0 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00145b60 con 0x7f2d000b2e80 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d0013dcf0 0x7f2d00156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53442/0 (socket says 172.21.15.185:53442) 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 -- 172.21.15.185:0/3733149693 learned_addr learned my addr 172.21.15.185:0/3733149693 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfe7fc700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 0x7f2d001400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 -- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d0013dcf0 msgr2=0x7f2d00156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d0013dcf0 0x7f2d00156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 -- 172.21.15.185:0/3733149693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 msgr2=0x7f2d001400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 0x7f2d001400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 -- 172.21.15.185:0/3733149693 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00145e00 con 0x7f2d000b2e80 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfe7fc700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 0x7f2d001400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfeffd700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2cf00098e0 tx=0x7f2cf0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0011d50 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d00145370 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf001e460 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf001e7e0 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d001405c0 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2cf001a040 con 0x7f2d000b2e80 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 0x7f2cdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0005d200 con 0x7f2d000b2e80 2020-11-16T22:17:45.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfe7fc700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 0x7f2cdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2cf006f190 con 0x7f2d000b2e80 2020-11-16T22:17:45.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.395+0000 7f2cfe7fc700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 0x7f2cdc057520 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f2cf4006a10 tx=0x7f2cf4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.403+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2cf0048ed0 con 0x7f2d000b2e80 2020-11-16T22:17:45.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2d000b2c20 con 0x7f2d000b2e80 2020-11-16T22:17:45.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2ce6ffd700 1 -- 172.21.15.185:0/3733149693 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2cf0023020 con 0x7f2d000b2e80 2020-11-16T22:17:45.534 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:45.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 msgr2=0x7f2cdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 0x7f2cdc057520 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f2cf4006a10 tx=0x7f2cf4008040).stop 2020-11-16T22:17:45.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 msgr2=0x7f2d00146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f2cf00098e0 tx=0x7f2cf0015000).stop 2020-11-16T22:17:45.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 shutdown_connections 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2cdc055060 0x7f2cdc057520 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2d0013dcf0 0x7f2d00156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2d000b2e80 0x7f2d00146790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 --2- 172.21.15.185:0/3733149693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d000b3920 0x7f2d001400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 >> 172.21.15.185:0/3733149693 conn(0x7f2d00060130 msgr2=0x7f2d00060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.551+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 shutdown_connections 2020-11-16T22:17:45.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.555+0000 7f2cfffff700 1 -- 172.21.15.185:0/3733149693 wait complete. 2020-11-16T22:17:45.548 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:17:45.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.691+0000 7f2cbffff700 1 Processor -- start 2020-11-16T22:17:45.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.691+0000 7f2cbffff700 1 -- start start 2020-11-16T22:17:45.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2cc00670b0 0x7f2cc0067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2cc00b74f0 0x7f2cc00b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc00b4e80 con 0x7f2cc0068030 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc00b3e10 con 0x7f2cc00b74f0 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc0141700 con 0x7f2cc00670b0 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbf7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2cc0068030 0x7f2cc00b4770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53448/0 (socket says 172.21.15.185:53448) 2020-11-16T22:17:45.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbf7fe700 1 -- 172.21.15.185:0/459957845 learned_addr learned my addr 172.21.15.185:0/459957845 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:45.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2cc00670b0 0x7f2cc0067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbe7fc700 1 --2- 172.21.15.185:0/459957845 >> v2:172.21.15.96:3300/0 conn(0x7f2cc00b74f0 0x7f2cc00b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbdffb700 1 -- 172.21.15.185:0/459957845 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953006628 0 0) 0x7f2cc00b4e80 con 0x7f2cc0068030 2020-11-16T22:17:45.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbdffb700 1 -- 172.21.15.185:0/459957845 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca4003240 con 0x7f2cc0068030 2020-11-16T22:17:45.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 -- 172.21.15.185:0/459957845 >> v2:172.21.15.96:3300/0 conn(0x7f2cc00b74f0 msgr2=0x7f2cc00b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 --2- 172.21.15.185:0/459957845 >> v2:172.21.15.96:3300/0 conn(0x7f2cc00b74f0 0x7f2cc00b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 -- 172.21.15.185:0/459957845 >> v1:172.21.15.185:6789/0 conn(0x7f2cc0068030 legacy=0x7f2cc00b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 -- 172.21.15.185:0/459957845 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc01419a0 con 0x7f2cc00670b0 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbeffd700 1 --2- 172.21.15.185:0/459957845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc00670b0 0x7f2cc0067490 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f2cc0065390 tx=0x7f2cb0006df0).ready entity=mon.1 client_cookie=d8f33871ac3415fb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbdffb700 1 -- 172.21.15.185:0/459957845 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0009640 con 0x7f2cc00670b0 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbdffb700 1 -- 172.21.15.185:0/459957845 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00097a0 con 0x7f2cc00670b0 2020-11-16T22:17:45.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbdffb700 1 -- 172.21.15.185:0/459957845 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0009b20 con 0x7f2cc00670b0 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- 172.21.15.185:0/459957845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc00670b0 msgr2=0x7f2cc0067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 --2- 172.21.15.185:0/459957845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc00670b0 0x7f2cc0067490 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f2cc0065390 tx=0x7f2cb0006df0).stop 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- 172.21.15.185:0/459957845 shutdown_connections 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 --2- 172.21.15.185:0/459957845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc00670b0 0x7f2cc0067490 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 --2- 172.21.15.185:0/459957845 >> v2:172.21.15.96:3300/0 conn(0x7f2cc00b74f0 0x7f2cc00b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- 172.21.15.185:0/459957845 >> 172.21.15.185:0/459957845 conn(0x7f2cc0064b90 msgr2=0x7f2cc0063790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- 172.21.15.185:0/459957845 shutdown_connections 2020-11-16T22:17:45.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.695+0000 7f2cbffff700 1 -- 172.21.15.185:0/459957845 wait complete. 2020-11-16T22:17:45.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 Processor -- start 2020-11-16T22:17:45.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- start start 2020-11-16T22:17:45.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cc00670b0 0x7f2cc00af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cc00b74f0 0x7f2cc00b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc01409b0 con 0x7f2cc00b74f0 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00afb00 con 0x7f2cc0068030 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc01540c0 con 0x7f2cc00670b0 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60012/0 (socket says 172.21.15.185:60012) 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 -- 172.21.15.185:0/2317204345 learned_addr learned my addr 172.21.15.185:0/2317204345 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbeffd700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cc00670b0 0x7f2cc00af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbf7fe700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cc00b74f0 0x7f2cc00b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 -- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cc00670b0 msgr2=0x7f2cc00af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cc00670b0 0x7f2cc00af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 -- 172.21.15.185:0/2317204345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cc00b74f0 msgr2=0x7f2cc00b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cc00b74f0 0x7f2cc00b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 -- 172.21.15.185:0/2317204345 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0142b50 con 0x7f2cc0068030 2020-11-16T22:17:45.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbe7fc700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2ca800e410 tx=0x7f2ca80091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8011660 con 0x7f2cc0068030 2020-11-16T22:17:45.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80117c0 con 0x7f2cc0068030 2020-11-16T22:17:45.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc0142df0 con 0x7f2cc0068030 2020-11-16T22:17:45.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8011b40 con 0x7f2cc0068030 2020-11-16T22:17:45.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc0142100 con 0x7f2cc0068030 2020-11-16T22:17:45.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.699+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc0154280 con 0x7f2cc0068030 2020-11-16T22:17:45.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.703+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2ca80193d0 con 0x7f2cc0068030 2020-11-16T22:17:45.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.703+0000 7f2cb6ffd700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 0x7f2c90057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.703+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ca8070a30 con 0x7f2cc0068030 2020-11-16T22:17:45.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.703+0000 7f2cbeffd700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 0x7f2c90057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.703+0000 7f2cbeffd700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 0x7f2c90057520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f2cb0006e30 tx=0x7f2cb0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.711+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ca8046c30 con 0x7f2cc0068030 2020-11-16T22:17:45.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cc00b4eb0 con 0x7f2cc0068030 2020-11-16T22:17:45.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cb6ffd700 1 -- 172.21.15.185:0/2317204345 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2cc00b4eb0 con 0x7f2cc0068030 2020-11-16T22:17:45.850 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:45.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 msgr2=0x7f2c90057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 0x7f2c90057520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f2cb0006e30 tx=0x7f2cb0015000).stop 2020-11-16T22:17:45.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 msgr2=0x7f2cc00af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2ca800e410 tx=0x7f2ca80091d0).stop 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 shutdown_connections 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2c90055060 0x7f2c90057520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cc00b74f0 0x7f2cc00b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cc00670b0 0x7f2cc00af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 --2- 172.21.15.185:0/2317204345 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cc0068030 0x7f2cc00af610 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 >> 172.21.15.185:0/2317204345 conn(0x7f2cc0064b90 msgr2=0x7f2cc013be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 shutdown_connections 2020-11-16T22:17:45.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:45.867+0000 7f2cbffff700 1 -- 172.21.15.185:0/2317204345 wait complete. 2020-11-16T22:17:45.864 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:17:45.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.003+0000 7f348759e700 1 Processor -- start 2020-11-16T22:17:45.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.003+0000 7f348759e700 1 -- start start 2020-11-16T22:17:45.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 0x7f34880b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3488143ef0 0x7f34880b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34880b1310 con 0x7f34880b2e80 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348813e400 con 0x7f34880b3920 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34880ac070 con 0x7f3488143ef0 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3488143ef0 0x7f34880b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3488143ef0 0x7f34880b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60022/0 (socket says 172.21.15.185:60022) 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 -- 172.21.15.185:0/62495569 learned_addr learned my addr 172.21.15.185:0/62495569 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348659c700 1 --1- 172.21.15.185:0/62495569 >> v1:172.21.15.185:6789/0 conn(0x7f34880b2e80 0x7f348813dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53458/0 (socket says 172.21.15.185:53458) 2020-11-16T22:17:45.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3485d9b700 1 --2- 172.21.15.185:0/62495569 >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 0x7f34880b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 -- 172.21.15.185:0/62495569 >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 msgr2=0x7f34880b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 --2- 172.21.15.185:0/62495569 >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 0x7f34880b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 -- 172.21.15.185:0/62495569 >> v1:172.21.15.185:6789/0 conn(0x7f34880b2e80 legacy=0x7f348813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 -- 172.21.15.185:0/62495569 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880ac310 con 0x7f3488143ef0 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3485d9b700 1 --2- 172.21.15.185:0/62495569 >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 0x7f34880b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:45.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f3486d9d700 1 --2- 172.21.15.185:0/62495569 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3488143ef0 0x7f34880b16b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3488064bb0 tx=0x7f3480006eb0).ready entity=mon.1 client_cookie=489927c4dba05d5d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348559a700 1 -- 172.21.15.185:0/62495569 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800098d0 con 0x7f3488143ef0 2020-11-16T22:17:45.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348559a700 1 -- 172.21.15.185:0/62495569 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480009a30 con 0x7f3488143ef0 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348559a700 1 -- 172.21.15.185:0/62495569 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480009db0 con 0x7f3488143ef0 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- 172.21.15.185:0/62495569 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3488143ef0 msgr2=0x7f34880b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 --2- 172.21.15.185:0/62495569 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3488143ef0 0x7f34880b16b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3488064bb0 tx=0x7f3480006eb0).stop 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- 172.21.15.185:0/62495569 shutdown_connections 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 --2- 172.21.15.185:0/62495569 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3488143ef0 0x7f34880b16b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 --2- 172.21.15.185:0/62495569 >> v2:172.21.15.96:3300/0 conn(0x7f34880b3920 0x7f34880b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- 172.21.15.185:0/62495569 >> 172.21.15.185:0/62495569 conn(0x7f3488060130 msgr2=0x7f3488060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:45.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- 172.21.15.185:0/62495569 shutdown_connections 2020-11-16T22:17:45.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.007+0000 7f348759e700 1 -- 172.21.15.185:0/62495569 wait complete. 2020-11-16T22:17:45.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 Processor -- start 2020-11-16T22:17:45.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 -- start start 2020-11-16T22:17:45.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 0x7f3488149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 0x7f348813c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:45.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348813e7d0 con 0x7f34880b2e80 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348813ea70 con 0x7f34880b3920 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348813ed10 con 0x7f3488143ef0 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3485d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 0x7f348813c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53472/0 (socket says 172.21.15.185:53472) 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 -- 172.21.15.185:0/1667399162 learned_addr learned my addr 172.21.15.185:0/1667399162 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:45.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3485d9b700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 0x7f348813c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60026/0 (socket says 172.21.15.185:60026) 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348659c700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 0x7f3488149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 -- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 msgr2=0x7f348813c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 0x7f348813c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 -- 172.21.15.185:0/1667399162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 msgr2=0x7f3488149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 0x7f3488149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:45.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 -- 172.21.15.185:0/1667399162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488149620 con 0x7f3488143ef0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348659c700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 0x7f3488149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f3486d9d700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f34880ac1b0 tx=0x7f3480015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480006b20 con 0x7f3488143ef0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348001d5d0 con 0x7f3488143ef0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34881467e0 con 0x7f3488143ef0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.011+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348001d950 con 0x7f3488143ef0 2020-11-16T22:17:45.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488146be0 con 0x7f3488143ef0 2020-11-16T22:17:46.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348813eed0 con 0x7f3488143ef0 2020-11-16T22:17:46.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3480025030 con 0x7f3488143ef0 2020-11-16T22:17:46.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f347e7fc700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 0x7f3458057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f348006f640 con 0x7f3488143ef0 2020-11-16T22:17:46.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f348659c700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 0x7f3458057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.015+0000 7f348659c700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 0x7f3458057520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f3488140410 tx=0x7f3478007140).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:46.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.023+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f34800459d0 con 0x7f3488143ef0 2020-11-16T22:17:46.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f348805d200 con 0x7f3488143ef0 2020-11-16T22:17:46.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f347e7fc700 1 -- 172.21.15.185:0/1667399162 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3480023070 con 0x7f3488143ef0 2020-11-16T22:17:46.163 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:17:46.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 msgr2=0x7f3458057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 0x7f3458057520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f3488140410 tx=0x7f3478007140).stop 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 msgr2=0x7f348813c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.179+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f34880ac1b0 tx=0x7f3480015000).stop 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 shutdown_connections 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3458055060 0x7f3458057520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3488143ef0 0x7f348813c7e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34880b2e80 0x7f3488149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 --2- 172.21.15.185:0/1667399162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f34880b3920 0x7f348813c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 >> 172.21.15.185:0/1667399162 conn(0x7f3488060130 msgr2=0x7f3488060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:46.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 shutdown_connections 2020-11-16T22:17:46.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.183+0000 7f348759e700 1 -- 172.21.15.185:0/1667399162 wait complete. 2020-11-16T22:17:46.177 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:17:46.177 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.315+0000 7fa4eaecf700 1 Processor -- start 2020-11-16T22:17:46.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.315+0000 7fa4eaecf700 1 -- start start 2020-11-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4e40b64b0 0x7fa4e40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e4147f90 con 0x7fa4e40b72e0 2020-11-16T22:17:46.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40af020 con 0x7fa4e40b64b0 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40b4810 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53476/0 (socket says 172.21.15.185:53476) 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4e9ecd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa4e40b72e0 0x7fa4e4144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53474/0 (socket says 172.21.15.185:53474) 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 -- 172.21.15.185:0/4043148313 learned_addr learned my addr 172.21.15.185:0/4043148313 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:46.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4e96cc700 1 --2- 172.21.15.185:0/4043148313 >> v2:172.21.15.96:3300/0 conn(0x7fa4e40b64b0 0x7fa4e40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 -- 172.21.15.185:0/4043148313 >> v2:172.21.15.96:3300/0 conn(0x7fa4e40b64b0 msgr2=0x7fa4e40b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 --2- 172.21.15.185:0/4043148313 >> v2:172.21.15.96:3300/0 conn(0x7fa4e40b64b0 0x7fa4e40b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 -- 172.21.15.185:0/4043148313 >> v1:172.21.15.185:6789/0 conn(0x7fa4e40b72e0 legacy=0x7fa4e4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 -- 172.21.15.185:0/4043148313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4148460 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4ea6ce700 1 --2- 172.21.15.185:0/4043148313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fa4dc009d50 tx=0x7fa4dc009290).ready entity=mon.2 client_cookie=74886c0ffebd2c67 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:46.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4e8ecb700 1 -- 172.21.15.185:0/4043148313 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00bbc0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4e8ecb700 1 -- 172.21.15.185:0/4043148313 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00bd20 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4e8ecb700 1 -- 172.21.15.185:0/4043148313 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc011610 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- 172.21.15.185:0/4043148313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 msgr2=0x7fa4e40acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/4043148313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fa4dc009d50 tx=0x7fa4dc009290).stop 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- 172.21.15.185:0/4043148313 shutdown_connections 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/4043148313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e40acee0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/4043148313 >> v2:172.21.15.96:3300/0 conn(0x7fa4e40b64b0 0x7fa4e40b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- 172.21.15.185:0/4043148313 >> 172.21.15.185:0/4043148313 conn(0x7fa4e4067860 msgr2=0x7fa4e4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- 172.21.15.185:0/4043148313 shutdown_connections 2020-11-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.319+0000 7fa4eaecf700 1 -- 172.21.15.185:0/4043148313 wait complete. 2020-11-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 Processor -- start 2020-11-16T22:17:46.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 -- start start 2020-11-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 0x7fa4e4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4e40b72e0 0x7fa4e413f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40afe40 con 0x7fa4e40b64b0 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4157710 con 0x7fa4e40b72e0 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4eaecf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40b00a0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4ea6ce700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4e40b72e0 0x7fa4e413f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53484/0 (socket says 172.21.15.185:53484) 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 -- 172.21.15.185:0/1464213361 learned_addr learned my addr 172.21.15.185:0/1464213361 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:46.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e9ecd700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 0x7fa4e4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 -- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4e40b72e0 msgr2=0x7fa4e413f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4e40b72e0 0x7fa4e413f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 -- 172.21.15.185:0/1464213361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 msgr2=0x7fa4e4157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 0x7fa4e4157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 -- 172.21.15.185:0/1464213361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40b0340 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e9ecd700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 0x7fa4e4157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:46.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.323+0000 7fa4e96cc700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fa4d400eba0 tx=0x7fa4d4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:46.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400c9b0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d400cb10 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e40b05e0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400ce90 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e41406c0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40b0770 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa4d4012600 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 0x7fa4bc057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4e9ecd700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 0x7fa4bc057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4d406e1e0 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.327+0000 7fa4e9ecd700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 0x7fa4bc057570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fa4e0000f70 tx=0x7fa4e00070e0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:46.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.335+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa4d4044350 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.535+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e405d200 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.535+0000 7fa4d1ffb700 1 -- 172.21.15.185:0/1464213361 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fa4d4018270 con 0x7fa4e40b6aa0 2020-11-16T22:17:46.520 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:46.520 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":50,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":45},{"gid":6054,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:41.297923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:46.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 msgr2=0x7fa4bc057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 0x7fa4bc057570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fa4e0000f70 tx=0x7fa4e00070e0).stop 2020-11-16T22:17:46.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 msgr2=0x7fa4e413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:46.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fa4d400eba0 tx=0x7fa4d4006df0).stop 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 shutdown_connections 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4bc0550b0 0x7fa4bc057570 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4e40b72e0 0x7fa4e413f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e40b64b0 0x7fa4e4157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 --2- 172.21.15.185:0/1464213361 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4e40b6aa0 0x7fa4e413e860 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 >> 172.21.15.185:0/1464213361 conn(0x7fa4e4067860 msgr2=0x7fa4e40b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 shutdown_connections 2020-11-16T22:17:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:46.539+0000 7fa4eaecf700 1 -- 172.21.15.185:0/1464213361 wait complete. 2020-11-16T22:17:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 50 2020-11-16T22:17:46.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:17:38.870443+0000', 'modified': '2020-11-16T22:17:41.297923+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6057': {'gid': 6057, 'name': 'b', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6834/2034361528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2034361528}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 2034361528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:17:46.838 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:17:46.839 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:17:46.981 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6057,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.1194358920000003,"mdsmap_epoch":50,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":14.747555293} 2020-11-16T22:17:46.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6057, "mdsmap_epoch": 50, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 8.119435892, "state": "up:active", "uptime": 14.747555293, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:17:46.982 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:17:46.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:46.982 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:17:46.999 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:17:47.000 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:17:47.064 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:17:47.064 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:17:47.092 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:17:47.092 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:17:47.099 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:17:47.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:17:47.099 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:17:47.131 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.287+0000 7f6b2f4b2700 1 Processor -- start 2020-11-16T22:17:47.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.287+0000 7f6b2f4b2700 1 -- start start 2020-11-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.287+0000 7f6b2f4b2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b28142950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6b28067490 0x7f6b2813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b28060e40 con 0x7f6b28067030 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b28141f70 con 0x7f6b28067490 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28142210 con 0x7f6b280b76b0 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6b28067490 0x7f6b2813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2dcaf700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6b28067030 0x7f6b28060450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53484/0 (socket says 172.21.15.185:53484) 2020-11-16T22:17:47.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2dcaf700 1 -- 172.21.15.185:0/3719251424 learned_addr learned my addr 172.21.15.185:0/3719251424 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:47.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2e4b0700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b28142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 msgr2=0x7f6b28142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b28142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/3719251424 >> v1:172.21.15.185:6789/0 conn(0x7f6b28067030 legacy=0x7f6b28060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:47.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/3719251424 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b281424b0 con 0x7f6b28067490 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2d4ae700 1 -- 172.21.15.185:0/3719251424 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806125458 0 0) 0x7f6b28060e40 con 0x7f6b28067030 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2ecb1700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b2813ba20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f6b20009c90 tx=0x7f6b200091d0).ready entity=mon.1 client_cookie=11cc297d9e3f8aaf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2d4ae700 1 -- 172.21.15.185:0/3719251424 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000ba70 con 0x7f6b28067490 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2d4ae700 1 -- 172.21.15.185:0/3719251424 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000bbd0 con 0x7f6b28067490 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2d4ae700 1 -- 172.21.15.185:0/3719251424 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20011400 con 0x7f6b28067490 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 msgr2=0x7f6b2813ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:47.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b2813ba20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f6b20009c90 tx=0x7f6b200091d0).stop 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/3719251424 shutdown_connections 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b2813ba20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/3719251424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b28142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/3719251424 >> 172.21.15.185:0/3719251424 conn(0x7f6b28064b90 msgr2=0x7f6b28063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/3719251424 shutdown_connections 2020-11-16T22:17:47.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.291+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/3719251424 wait complete. 2020-11-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 Processor -- start 2020-11-16T22:17:47.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 -- start start 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b28067030 0x7f6b280aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b281431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281512f0 con 0x7f6b28067030 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280af530 con 0x7f6b28067490 2020-11-16T22:17:47.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2f4b2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281514e0 con 0x7f6b280b76b0 2020-11-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2ecb1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2ecb1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53500/0 (socket says 172.21.15.185:53500) 2020-11-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/572559695 learned_addr learned my addr 172.21.15.185:0/572559695 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2dcaf700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b281431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.295+0000 7f6b2e4b0700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b28067030 0x7f6b280aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 msgr2=0x7f6b281431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b281431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/572559695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b28067030 msgr2=0x7f6b280aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b28067030 0x7f6b280aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 -- 172.21.15.185:0/572559695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28151780 con 0x7f6b280b76b0 2020-11-16T22:17:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2ecb1700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f6b20009210 tx=0x7f6b20016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2001c6c0 con 0x7f6b280b76b0 2020-11-16T22:17:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2001c820 con 0x7f6b280b76b0 2020-11-16T22:17:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b28151a20 con 0x7f6b280b76b0 2020-11-16T22:17:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2001cbe0 con 0x7f6b280b76b0 2020-11-16T22:17:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281521e0 con 0x7f6b280b76b0 2020-11-16T22:17:47.488 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b280af2c0 con 0x7f6b280b76b0 2020-11-16T22:17:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6b20024030 con 0x7f6b280b76b0 2020-11-16T22:17:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 0x7f6b00057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b2e4b0700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 0x7f6b00057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:47.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.299+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6b2006f740 con 0x7f6b280b76b0 2020-11-16T22:17:47.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.303+0000 7f6b2e4b0700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 0x7f6b00057520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6b24006fd0 tx=0x7f6b24006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:47.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.311+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6b20045ad0 con 0x7f6b280b76b0 2020-11-16T22:17:47.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:47.515+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f6b28151e90 con 0x7f6b280b76b0 2020-11-16T22:17:47.696 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:47.698 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:17:48.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.407+0000 7f6b167fc700 1 -- 172.21.15.185:0/572559695 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v51) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6b20022020 con 0x7f6b280b76b0 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 msgr2=0x7f6b00057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 0x7f6b00057520 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6b24006fd0 tx=0x7f6b24006df0).stop 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 msgr2=0x7f6b281436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f6b20009210 tx=0x7f6b20016000).stop 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 shutdown_connections 2020-11-16T22:17:48.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6b00055060 0x7f6b00057520 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6b280b76b0 0x7f6b281436c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b28067030 0x7f6b280aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 --2- 172.21.15.185:0/572559695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6b28067490 0x7f6b281431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 >> 172.21.15.185:0/572559695 conn(0x7f6b28064b90 msgr2=0x7f6b280b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 shutdown_connections 2020-11-16T22:17:48.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.411+0000 7f6b2f4b2700 1 -- 172.21.15.185:0/572559695 wait complete. 2020-11-16T22:17:48.417 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:48.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 Processor -- start 2020-11-16T22:17:48.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 -- start start 2020-11-16T22:17:48.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa6dc0b3090 0x7fa6dc0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc060ef0 con 0x7fa6dc1458a0 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc1428e0 con 0x7fa6dc0b3090 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6db59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc062090 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60064/0 (socket says 172.21.15.185:60064) 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/584899204 learned_addr learned my addr 172.21.15.185:0/584899204 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6da59c700 1 --2- 172.21.15.185:0/584899204 >> v2:172.21.15.96:3300/0 conn(0x7fa6dc0b3090 0x7fa6dc0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/584899204 >> v2:172.21.15.96:3300/0 conn(0x7fa6dc0b3090 msgr2=0x7fa6dc0b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 --2- 172.21.15.185:0/584899204 >> v2:172.21.15.96:3300/0 conn(0x7fa6dc0b3090 0x7fa6dc0b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/584899204 >> v1:172.21.15.96:6789/0 conn(0x7fa6dc1458a0 legacy=0x7fa6dc1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/584899204 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0ad640 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d9d9b700 1 --2- 172.21.15.185:0/584899204 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fa6dc0b2a30 tx=0x7fa6c4006eb0).ready entity=mon.1 client_cookie=5fbd72781e582923 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:48.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d959a700 1 -- 172.21.15.185:0/584899204 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4009810 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.551+0000 7fa6d959a700 1 -- 172.21.15.185:0/584899204 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4009970 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6d959a700 1 -- 172.21.15.185:0/584899204 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4009cf0 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- 172.21.15.185:0/584899204 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc0b3b80 msgr2=0x7fa6dc1453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 --2- 172.21.15.185:0/584899204 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fa6dc0b2a30 tx=0x7fa6c4006eb0).stop 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- 172.21.15.185:0/584899204 shutdown_connections 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 --2- 172.21.15.185:0/584899204 >> v2:172.21.15.96:3300/0 conn(0x7fa6dc0b3090 0x7fa6dc0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 --2- 172.21.15.185:0/584899204 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc0b3b80 0x7fa6dc1453b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- 172.21.15.185:0/584899204 >> 172.21.15.185:0/584899204 conn(0x7fa6dc060130 msgr2=0x7fa6dc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:48.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- 172.21.15.185:0/584899204 shutdown_connections 2020-11-16T22:17:48.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- 172.21.15.185:0/584899204 wait complete. 2020-11-16T22:17:48.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 Processor -- start 2020-11-16T22:17:48.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.555+0000 7fa6db59e700 1 -- start start 2020-11-16T22:17:48.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6dc0b3090 0x7fa6dc0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc1458a0 0x7fa6dc13d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0b66d0 con 0x7fa6dc0b3090 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0b70e0 con 0x7fa6dc1458a0 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc141f20 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53512/0 (socket says 172.21.15.185:53512) 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/4153201696 learned_addr learned my addr 172.21.15.185:0/4153201696 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:48.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6dad9d700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc1458a0 0x7fa6dc13d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6da59c700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6dc0b3090 0x7fa6dc0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc1458a0 msgr2=0x7fa6dc13d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc1458a0 0x7fa6dc13d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/4153201696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6dc0b3090 msgr2=0x7fa6dc0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6dc0b3090 0x7fa6dc0b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 -- 172.21.15.185:0/4153201696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0b6b70 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d9d9b700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa6c4010000 tx=0x7fa6c4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011a70 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4011bd0 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc0b6e10 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c401e460 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.559+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1419c0 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc0b69e0 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa6c4025030 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6d27fc700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 0x7fa6b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:48.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa6c406f2e0 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6da59c700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 0x7fa6b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:48.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.563+0000 7fa6da59c700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 0x7fa6b8057520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa6dc148140 tx=0x7fa6cc006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:48.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.567+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa6c4045560 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.771+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6dc05d200 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.771+0000 7fa6d27fc700 1 -- 172.21.15.185:0/4153201696 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2504 (secure 0 0 0) 0x7fa6c4023070 con 0x7fa6dc0b3b80 2020-11-16T22:17:48.755 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:48.755 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":45}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:48.414393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6057,"mds_1":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:creating","state_seq":2,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:48.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.771+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 msgr2=0x7fa6b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.771+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 0x7fa6b8057520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa6dc148140 tx=0x7fa6cc006df0).stop 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 msgr2=0x7fa6dc10f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa6c4010000 tx=0x7fa6c4015000).stop 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 shutdown_connections 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa6b8055060 0x7fa6b8057520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa6dc1458a0 0x7fa6dc13d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6dc0b3090 0x7fa6dc0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 --2- 172.21.15.185:0/4153201696 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa6dc0b3b80 0x7fa6dc10f6b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:48.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 >> 172.21.15.185:0/4153201696 conn(0x7fa6dc060130 msgr2=0x7fa6dc1472c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:48.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 shutdown_connections 2020-11-16T22:17:48.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:48.775+0000 7fa6db59e700 1 -- 172.21.15.185:0/4153201696 wait complete. 2020-11-16T22:17:48.758 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 52 2020-11-16T22:17:49.071 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:17:38.870443+0000', 'modified': '2020-11-16T22:17:48.414393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6057, 'mds_1': 6054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6054': {'gid': 6054, 'name': 'c', 'rank': 1, 'incarnation': 52, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.15.96:6835/774952173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6833', 'nonce': 774952173}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 774952173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6057': {'gid': 6057, 'name': 'b', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6834/2034361528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2034361528}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 2034361528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:17:49.072 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6054:up:creating 2020-11-16T22:17:50.073 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.219+0000 7f791cf97700 1 Processor -- start 2020-11-16T22:17:50.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.219+0000 7f791cf97700 1 -- start start 2020-11-16T22:17:50.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.219+0000 7f791cf97700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f79180b0300 0x7f79180b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.219+0000 7f791cf97700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7918061cf0 0x7f79181475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7918062700 con 0x7f7918061640 2020-11-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180629a0 con 0x7f7918061cf0 2020-11-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7918062c40 con 0x7f79180b0300 2020-11-16T22:17:50.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f79180b0300 0x7f79180b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f7915d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7918061640 0x7f79180b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53522/0 (socket says 172.21.15.185:53522) 2020-11-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f7915d9b700 1 -- 172.21.15.185:0/1882589125 learned_addr learned my addr 172.21.15.185:0/1882589125 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:17:50.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f7916d9d700 1 --2- 172.21.15.185:0/1882589125 >> v2:172.21.15.96:3300/0 conn(0x7f7918061cf0 0x7f79181475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 -- 172.21.15.185:0/1882589125 >> v2:172.21.15.96:3300/0 conn(0x7f7918061cf0 msgr2=0x7f79181475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 --2- 172.21.15.185:0/1882589125 >> v2:172.21.15.96:3300/0 conn(0x7f7918061cf0 0x7f79181475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 -- 172.21.15.185:0/1882589125 >> v1:172.21.15.185:6789/0 conn(0x7f7918061640 legacy=0x7f79180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 -- 172.21.15.185:0/1882589125 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918062ee0 con 0x7f79180b0300 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791559a700 1 -- 172.21.15.185:0/1882589125 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005231967 0 0) 0x7f7918062700 con 0x7f7918061640 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791659c700 1 --2- 172.21.15.185:0/1882589125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79180b0300 0x7f79180b06e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f7918068740 tx=0x7f7908006df0).ready entity=mon.1 client_cookie=58fd6889e5c037c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791559a700 1 -- 172.21.15.185:0/1882589125 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79080097b0 con 0x7f79180b0300 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791559a700 1 -- 172.21.15.185:0/1882589125 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908009910 con 0x7f79180b0300 2020-11-16T22:17:50.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791559a700 1 -- 172.21.15.185:0/1882589125 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908009c90 con 0x7f79180b0300 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- 172.21.15.185:0/1882589125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79180b0300 msgr2=0x7f79180b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 --2- 172.21.15.185:0/1882589125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79180b0300 0x7f79180b06e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f7918068740 tx=0x7f7908006df0).stop 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- 172.21.15.185:0/1882589125 shutdown_connections 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 --2- 172.21.15.185:0/1882589125 >> v2:172.21.15.96:3300/0 conn(0x7f7918061cf0 0x7f79181475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 --2- 172.21.15.185:0/1882589125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f79180b0300 0x7f79180b06e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- 172.21.15.185:0/1882589125 >> 172.21.15.185:0/1882589125 conn(0x7f7918060130 msgr2=0x7f7918060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- 172.21.15.185:0/1882589125 shutdown_connections 2020-11-16T22:17:50.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- 172.21.15.185:0/1882589125 wait complete. 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 Processor -- start 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.223+0000 7f791cf97700 1 -- start start 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7918061cf0 0x7f7918142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 0x7f791813a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791813fae0 con 0x7f79180b0300 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791813fd80 con 0x7f7918061640 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791813f2b0 con 0x7f7918061cf0 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60090/0 (socket says 172.21.15.185:60090) 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f7915d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7918061cf0 0x7f7918142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 -- 172.21.15.185:0/2340058643 learned_addr learned my addr 172.21.15.185:0/2340058643 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f7916d9d700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 0x7f791813a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 -- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7918061cf0 msgr2=0x7f7918142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7918061cf0 0x7f7918142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 -- 172.21.15.185:0/2340058643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 msgr2=0x7f791813a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 0x7f791813a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 -- 172.21.15.185:0/2340058643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791813f750 con 0x7f7918061640 2020-11-16T22:17:50.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f7916d9d700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 0x7f791813a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791659c700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f7918068920 tx=0x7f7908015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908011d50 con 0x7f7918061640 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790801e460 con 0x7f7918061640 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79181404f0 con 0x7f7918061640 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790801e7e0 con 0x7f7918061640 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7918140b00 con 0x7f7918061640 2020-11-16T22:17:50.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.227+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791813ff40 con 0x7f7918061640 2020-11-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.231+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7908025030 con 0x7f7918061640 2020-11-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.231+0000 7f79067fc700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 0x7f78ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:50.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.231+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f790806f470 con 0x7f7918061640 2020-11-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.231+0000 7f7915d9b700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 0x7f78ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:50.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.231+0000 7f7915d9b700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 0x7f78ec057520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f7918143980 tx=0x7f790c007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:50.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.235+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7908045780 con 0x7f7918061640 2020-11-16T22:17:50.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f791805d4e0 con 0x7f7918061640 2020-11-16T22:17:50.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f79067fc700 1 -- 172.21.15.185:0/2340058643 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2517 (secure 0 0 0) 0x7f7908023020 con 0x7f7918061640 2020-11-16T22:17:50.425 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:50.425 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:50.432704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:50.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 msgr2=0x7f78ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 0x7f78ec057520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f7918143980 tx=0x7f790c007a00).stop 2020-11-16T22:17:50.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 msgr2=0x7f7918142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:50.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f7918068920 tx=0x7f7908015000).stop 2020-11-16T22:17:50.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 shutdown_connections 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f78ec055060 0x7f78ec057520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7918061640 0x7f7918142060 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79180b0300 0x7f791813a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 --2- 172.21.15.185:0/2340058643 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7918061cf0 0x7f7918142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 >> 172.21.15.185:0/2340058643 conn(0x7f7918060130 msgr2=0x7f79180b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 shutdown_connections 2020-11-16T22:17:50.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:50.443+0000 7f791cf97700 1 -- 172.21.15.185:0/2340058643 wait complete. 2020-11-16T22:17:50.429 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 55 2020-11-16T22:17:50.740 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:17:38.870443+0000', 'modified': '2020-11-16T22:17:50.432704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6057, 'mds_1': 6054, 'mds_2': 5681}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5681': {'gid': 5681, 'name': 'a', 'rank': 2, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.185:6835/3773729030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 3773729030}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 3773729030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6054': {'gid': 6054, 'name': 'c', 'rank': 1, 'incarnation': 52, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.96:6835/774952173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6833', 'nonce': 774952173}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 774952173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6057': {'gid': 6057, 'name': 'b', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6834/2034361528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2034361528}, {'type': 'v1', 'addr': '172.21.15.96:6834', 'nonce': 2034361528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:17:50.741 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:17:50.741 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:17:50.881 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":5681,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":1.46230175,"mdsmap_epoch":55,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":18.652392285000001} 2020-11-16T22:17:50.882 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 5681, "mdsmap_epoch": 55, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 1.46230175, "state": "up:active", "uptime": 18.652392285, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:17:50.882 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:17:51.021 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":6054,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.6177045460000001,"mdsmap_epoch":55,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":18.788451470999998} 2020-11-16T22:17:51.022 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6054, "mdsmap_epoch": 55, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 2.617704546, "state": "up:active", "uptime": 18.788451471, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:17:51.022 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:17:51.153 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6057,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.292048530000001,"mdsmap_epoch":55,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":18.920179852} 2020-11-16T22:17:51.153 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6057, "mdsmap_epoch": 55, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 12.29204853, "state": "up:active", "uptime": 18.920179852, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:17:51.154 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:17:51.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 Processor -- start 2020-11-16T22:17:51.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 -- start start 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4d98141190 0x7f4d98141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980b3af0 con 0x7f4d980662c0 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d98061e60 con 0x7f4d98141190 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9e5e8700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980b3d20 con 0x7f4d980b2eb0 2020-11-16T22:17:51.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9d5e6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9d5e6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60102/0 (socket says 172.21.15.185:60102) 2020-11-16T22:17:51.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9d5e6700 1 -- 172.21.15.185:0/431845646 learned_addr learned my addr 172.21.15.185:0/431845646 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:51.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/431845646 >> v2:172.21.15.96:3300/0 conn(0x7f4d98141190 0x7f4d98141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 -- 172.21.15.185:0/431845646 >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 msgr2=0x7f4d980b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:17:51.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/431845646 >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 -- 172.21.15.185:0/431845646 >> v1:172.21.15.96:6789/0 conn(0x7f4d980662c0 legacy=0x7f4d9813a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 -- 172.21.15.185:0/431845646 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98140cc0 con 0x7f4d98141190 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9d5e6700 1 --2- 172.21.15.185:0/431845646 >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/431845646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d98141190 0x7f4d98141620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4d980b2e80 tx=0x7f4d90006df0).ready entity=mon.0 client_cookie=caa2a86bcf3711ea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d97fff700 1 -- 172.21.15.185:0/431845646 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900097b0 con 0x7f4d98141190 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d97fff700 1 -- 172.21.15.185:0/431845646 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90009910 con 0x7f4d98141190 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.303+0000 7f4d97fff700 1 -- 172.21.15.185:0/431845646 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90009c90 con 0x7f4d98141190 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/431845646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d98141190 msgr2=0x7f4d98141620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/431845646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d98141190 0x7f4d98141620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f4d980b2e80 tx=0x7f4d90006df0).stop 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/431845646 shutdown_connections 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/431845646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d98141190 0x7f4d98141620 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/431845646 >> v2:172.21.15.185:3300/0 conn(0x7f4d980b2eb0 0x7f4d980b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/431845646 >> 172.21.15.185:0/431845646 conn(0x7f4d98060130 msgr2=0x7f4d98060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/431845646 shutdown_connections 2020-11-16T22:17:51.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/431845646 wait complete. 2020-11-16T22:17:51.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 Processor -- start 2020-11-16T22:17:51.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- start start 2020-11-16T22:17:51.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d980662c0 0x7f4d98153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 0x7f4d98148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98154350 con 0x7f4d980662c0 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9813cb70 con 0x7f4d98141190 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9e5e8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9813ce10 con 0x7f4d980b2eb0 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9cde5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 0x7f4d98148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60110/0 (socket says 172.21.15.185:60110) 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 -- 172.21.15.185:0/4140990667 learned_addr learned my addr 172.21.15.185:0/4140990667 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9d5e6700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d980662c0 0x7f4d98153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 -- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 msgr2=0x7f4d98148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 0x7f4d98148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 -- 172.21.15.185:0/4140990667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d980662c0 msgr2=0x7f4d98153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d980662c0 0x7f4d98153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9dde7700 1 -- 172.21.15.185:0/4140990667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9813d0b0 con 0x7f4d98141190 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.307+0000 7f4d9cde5700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 0x7f4d98148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9dde7700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4d90006fd0 tx=0x7f4d90015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90011e50 con 0x7f4d98141190 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9001e460 con 0x7f4d98141190 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9813d350 con 0x7f4d98141190 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9001e7e0 con 0x7f4d98141190 2020-11-16T22:17:51.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9813b3d0 con 0x7f4d98141190 2020-11-16T22:17:51.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9813b6b0 con 0x7f4d98141190 2020-11-16T22:17:51.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4d90025030 con 0x7f4d98141190 2020-11-16T22:17:51.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 0x7f4d74057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:17:51.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d9006f1b0 con 0x7f4d98141190 2020-11-16T22:17:51.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9d5e6700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 0x7f4d74057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:17:51.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.311+0000 7f4d9d5e6700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 0x7f4d74057520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f4d80006fd0 tx=0x7f4d80006eb0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:17:51.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.319+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4d900454c0 con 0x7f4d98141190 2020-11-16T22:17:51.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.523+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d9813d4e0 con 0x7f4d98141190 2020-11-16T22:17:51.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.523+0000 7f4d957fa700 1 -- 172.21.15.185:0/4140990667 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+2517 (secure 0 0 0) 0x7f4d90023070 con 0x7f4d98141190 2020-11-16T22:17:51.506 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:17:51.507 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:17:50.432704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:17:51.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 msgr2=0x7f4d74057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 0x7f4d74057520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f4d80006fd0 tx=0x7f4d80006eb0).stop 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 msgr2=0x7f4d98152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f4d90006fd0 tx=0x7f4d90015000).stop 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 shutdown_connections 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4d74055060 0x7f4d74057520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4d98141190 0x7f4d98152870 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d980662c0 0x7f4d98153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 --2- 172.21.15.185:0/4140990667 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4d980b2eb0 0x7f4d98148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:17:51.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 >> 172.21.15.185:0/4140990667 conn(0x7f4d98060130 msgr2=0x7f4d980b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:17:51.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 shutdown_connections 2020-11-16T22:17:51.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:17:51.527+0000 7f4d9e5e8700 1 -- 172.21.15.185:0/4140990667 wait complete. 2020-11-16T22:17:51.510 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 55 2020-11-16T22:17:51.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:51.524 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:17:51.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:51.590 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:17:52.166 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:17:52.186+0000 7f4f6a19b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:17:52.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:52.243 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:17:52.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:52.284 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:17:52.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:17:52.321 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:17:52.359 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:17:52.492 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:17:52.493 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:17:52.493 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:17:54.495 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:17:54.633 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:17:54.634 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:17:54.635 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:17:56.636 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:17:56.774 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:17:56.775 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:17:56.776 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:17:58.777 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:17:58.917 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:17:58.917 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:17:58.918 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:18:00.919 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:18:01.057 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:18:01.058 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:18:01.059 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:18:03.060 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:18:03.199 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:18:03.200 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:18:03.201 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:18:03.201 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:18:03.323 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:18:03.324 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:18:03.324 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:18:05.326 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:18:05.465 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:18:05.466 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:18:05.466 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:18:07.467 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:18:07.602 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:18:07.603 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:18:07.604 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:18:09.605 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:18:09.742 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:18:09.743 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:18:09.744 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:18:09.744 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:18:09.882 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:18:09.911 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:18:10.114 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:18:10.115 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:10.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.263+0000 7f576534c700 1 Processor -- start 2020-11-16T22:18:10.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.263+0000 7f576534c700 1 -- start start 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 0x7f57600b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57600662c0 0x7f576013a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600b3af0 con 0x7f576013ae50 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5760061e60 con 0x7f57600b2eb0 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600b3d20 con 0x7f57600662c0 2020-11-16T22:18:10.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57600662c0 0x7f576013a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f57600662c0 0x7f576013a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60116/0 (socket says 172.21.15.185:60116) 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 -- 172.21.15.185:0/1724909638 learned_addr learned my addr 172.21.15.185:0/1724909638 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f575ffff700 1 --2- 172.21.15.185:0/1724909638 >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 0x7f57600b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 -- 172.21.15.185:0/1724909638 >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 msgr2=0x7f57600b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 --2- 172.21.15.185:0/1724909638 >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 0x7f57600b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 -- 172.21.15.185:0/1724909638 >> v1:172.21.15.96:6789/0 conn(0x7f576013ae50 legacy=0x7f5760141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 -- 172.21.15.185:0/1724909638 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760140cc0 con 0x7f57600662c0 2020-11-16T22:18:10.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f575ffff700 1 --2- 172.21.15.185:0/1724909638 >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 0x7f57600b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:10.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f57577fe700 1 --2- 172.21.15.185:0/1724909638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f576013a960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f57600652f0 tx=0x7f5748006eb0).ready entity=mon.1 client_cookie=247bf24878cfcb2e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f575f7fe700 1 -- 172.21.15.185:0/1724909638 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480098d0 con 0x7f57600662c0 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f575f7fe700 1 -- 172.21.15.185:0/1724909638 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748009a30 con 0x7f57600662c0 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f575f7fe700 1 -- 172.21.15.185:0/1724909638 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748009db0 con 0x7f57600662c0 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- 172.21.15.185:0/1724909638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 msgr2=0x7f576013a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 --2- 172.21.15.185:0/1724909638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f576013a960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f57600652f0 tx=0x7f5748006eb0).stop 2020-11-16T22:18:10.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- 172.21.15.185:0/1724909638 shutdown_connections 2020-11-16T22:18:10.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 --2- 172.21.15.185:0/1724909638 >> v2:172.21.15.96:3300/0 conn(0x7f57600b2eb0 0x7f57600b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 --2- 172.21.15.185:0/1724909638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f576013a960 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- 172.21.15.185:0/1724909638 >> 172.21.15.185:0/1724909638 conn(0x7f5760060130 msgr2=0x7f5760060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:10.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- 172.21.15.185:0/1724909638 shutdown_connections 2020-11-16T22:18:10.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.267+0000 7f576534c700 1 -- 172.21.15.185:0/1724909638 wait complete. 2020-11-16T22:18:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 Processor -- start 2020-11-16T22:18:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- start start 2020-11-16T22:18:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f5760149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576013ae50 0x7f5760140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760149710 con 0x7f576013ae50 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57601476f0 con 0x7f57600662c0 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5760147990 con 0x7f57600b2eb0 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f57577fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f5760149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f57577fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53568/0 (socket says 172.21.15.185:53568) 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f57577fe700 1 -- 172.21.15.185:0/953973009 learned_addr learned my addr 172.21.15.185:0/953973009 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:10.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f5764b4b700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576013ae50 0x7f5760140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 -- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 msgr2=0x7f5760148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 -- 172.21.15.185:0/953973009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576013ae50 msgr2=0x7f5760140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576013ae50 0x7f5760140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 -- 172.21.15.185:0/953973009 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760147c30 con 0x7f57600662c0 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f57577fe700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:10.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575ffff700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f5760149b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f575000db60 tx=0x7f5750008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:10.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000de70 con 0x7f57600662c0 2020-11-16T22:18:10.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750009790 con 0x7f57600662c0 2020-11-16T22:18:10.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57601404f0 con 0x7f57600662c0 2020-11-16T22:18:10.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750009b50 con 0x7f57600662c0 2020-11-16T22:18:10.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.271+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5760148880 con 0x7f57600662c0 2020-11-16T22:18:10.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760149880 con 0x7f57600662c0 2020-11-16T22:18:10.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5750013480 con 0x7f57600662c0 2020-11-16T22:18:10.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f575cff9700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 0x7f573c057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f575006e290 con 0x7f57600662c0 2020-11-16T22:18:10.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f57577fe700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 0x7f573c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.275+0000 7f57577fe700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 0x7f573c057550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f576014a160 tx=0x7f5748015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:10.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.283+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5750044490 con 0x7f57600662c0 2020-11-16T22:18:10.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f576005d200 con 0x7f57600662c0 2020-11-16T22:18:10.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f575cff9700 1 -- 172.21.15.185:0/953973009 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2521 (secure 0 0 0) 0x7f5750006a60 con 0x7f57600662c0 2020-11-16T22:18:10.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 msgr2=0x7f573c057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 0x7f573c057550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f576014a160 tx=0x7f5748015000).stop 2020-11-16T22:18:10.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 msgr2=0x7f5760149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f5760149b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f575000db60 tx=0x7f5750008040).stop 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 shutdown_connections 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f573c055090 0x7f573c057550 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f57600662c0 0x7f5760149b70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576013ae50 0x7f5760140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 --2- 172.21.15.185:0/953973009 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f57600b2eb0 0x7f5760148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 >> 172.21.15.185:0/953973009 conn(0x7f5760060130 msgr2=0x7f57601440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:10.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 shutdown_connections 2020-11-16T22:18:10.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.487+0000 7f576534c700 1 -- 172.21.15.185:0/953973009 wait complete. 2020-11-16T22:18:10.670 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 57 2020-11-16T22:18:10.670 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:10.670 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:07.995173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:10.671 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:18:10.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 Processor -- start 2020-11-16T22:18:10.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 -- start start 2020-11-16T22:18:10.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f980b6bf0 0x7f7f980b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f98149180 0x7f7f980ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:10.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9814bc30 con 0x7f7f98063ca0 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9814cb10 con 0x7f7f980b6bf0 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9814be90 con 0x7f7f98149180 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f980b6bf0 0x7f7f980b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7f980b6bf0 0x7f7f980b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60130/0 (socket says 172.21.15.185:60130) 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 -- 172.21.15.185:0/1804737338 learned_addr learned my addr 172.21.15.185:0/1804737338 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:10.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d807700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f98149180 0x7f7f980ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:10.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 -- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f98149180 msgr2=0x7f7f980ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:10.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f98149180 0x7f7f980ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:10.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 -- 172.21.15.185:0/1804737338 >> v1:172.21.15.185:6789/0 conn(0x7f7f98063ca0 legacy=0x7f7f98147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:10.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 -- 172.21.15.185:0/1804737338 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9814c130 con 0x7f7f980b6bf0 2020-11-16T22:18:10.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f980b6bf0 0x7f7f980b6fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f7f94006fd0 tx=0x7f7f94006eb0).ready entity=mon.1 client_cookie=5dada3b2eef7516e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:10.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f8ffff700 1 -- 172.21.15.185:0/1804737338 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94009710 con 0x7f7f980b6bf0 2020-11-16T22:18:10.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f8ffff700 1 -- 172.21.15.185:0/1804737338 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94009870 con 0x7f7f980b6bf0 2020-11-16T22:18:10.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f8ffff700 1 -- 172.21.15.185:0/1804737338 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94009bf0 con 0x7f7f980b6bf0 2020-11-16T22:18:10.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 -- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f980b6bf0 msgr2=0x7f7f980b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.819+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f980b6bf0 0x7f7f980b6fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f7f94006fd0 tx=0x7f7f94006eb0).stop 2020-11-16T22:18:11.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- 172.21.15.185:0/1804737338 shutdown_connections 2020-11-16T22:18:11.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f98149180 0x7f7f980ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/1804737338 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f980b6bf0 0x7f7f980b6fd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- 172.21.15.185:0/1804737338 >> 172.21.15.185:0/1804737338 conn(0x7f7f980657b0 msgr2=0x7f7f98065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- 172.21.15.185:0/1804737338 shutdown_connections 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- 172.21.15.185:0/1804737338 wait complete. 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 Processor -- start 2020-11-16T22:18:11.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- start start 2020-11-16T22:18:11.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 0x7f7f981480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 0x7f7f98146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9815bfc0 con 0x7f7f98149180 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9809be60 con 0x7f7f98063ca0 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9f26a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9809c100 con 0x7f7f980b6bf0 2020-11-16T22:18:11.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9c805700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 0x7f7f981480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60136/0 (socket says 172.21.15.185:60136) 2020-11-16T22:18:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9c805700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 0x7f7f981480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53582/0 (socket says 172.21.15.185:53582) 2020-11-16T22:18:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 -- 172.21.15.185:0/3103625062 learned_addr learned my addr 172.21.15.185:0/3103625062 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d807700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 0x7f7f98146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 -- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 msgr2=0x7f7f981480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 0x7f7f981480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 -- 172.21.15.185:0/3103625062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 msgr2=0x7f7f98146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 0x7f7f98146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d006700 1 -- 172.21.15.185:0/3103625062 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9809c3a0 con 0x7f7f98063ca0 2020-11-16T22:18:11.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.823+0000 7f7f9d807700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 0x7f7f98146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f7f980b7540 tx=0x7f7f94015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 -- 172.21.15.185:0/3103625062 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94011e00 con 0x7f7f98063ca0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 -- 172.21.15.185:0/3103625062 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94006760 con 0x7f7f98063ca0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f98157af0 con 0x7f7f98063ca0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 -- 172.21.15.185:0/3103625062 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9401d800 con 0x7f7f98063ca0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f980af5d0 con 0x7f7f98063ca0 2020-11-16T22:18:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 -- 172.21.15.185:0/3103625062 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7f94025030 con 0x7f7f98063ca0 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 0x7f7f70057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f8d7fa700 1 -- 172.21.15.185:0/3103625062 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f9406f6e0 con 0x7f7f98063ca0 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 0x7f7f70057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.827+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 0x7f7f70057520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f7f980af330 tx=0x7f7f88007b70).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 Processor -- start 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 -- start start 2020-11-16T22:18:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b037f0 0x2afe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29554b0 con 0x283dbb0 2020-11-16T22:18:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a09930 con 0x2b037f0 2020-11-16T22:18:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x28c96f0 con 0x28ab7b0 2020-11-16T22:18:11.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53588/0 (socket says 172.21.15.185:53588) 2020-11-16T22:18:11.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d807700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b037f0 0x2afe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 -- 172.21.15.185:0/1251689049 learned_addr learned my addr 172.21.15.185:0/1251689049 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9c805700 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 -- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b037f0 msgr2=0x2afe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b037f0 0x2afe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 -- 172.21.15.185:0/1251689049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 msgr2=0x2afdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 -- 172.21.15.185:0/1251689049 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28c45a0 con 0x28ab7b0 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f7f94002fc0 tx=0x7f7f94011000).ready entity=mon.2 client_cookie=8c00748ab3c6eaaf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.831+0000 7f7f7a7fc700 1 -- 172.21.15.185:0/1251689049 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f940479a0 con 0x28ab7b0 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7f7a7fc700 1 -- 172.21.15.185:0/1251689049 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9401dc30 con 0x28ab7b0 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7f7a7fc700 1 -- 172.21.15.185:0/1251689049 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94048af0 con 0x28ab7b0 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 msgr2=0x2b03300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f7f94002fc0 tx=0x7f7f94011000).stop 2020-11-16T22:18:11.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1251689049 shutdown_connections 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b037f0 0x2afe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28ab7b0 0x2b03300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1251689049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1251689049 >> 172.21.15.185:0/1251689049 conn(0x2a79e40 msgr2=0x2b02bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1251689049 shutdown_connections 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1251689049 wait complete. 2020-11-16T22:18:11.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- start start 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 Processor -- start 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2ae5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae5990 0x2b0cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.835+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29f66e0 con 0x2ae5990 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x29f6820 con 0x28ab7b0 2020-11-16T22:18:11.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29f7a90 con 0x283dbb0 2020-11-16T22:18:11.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53594/0 (socket says 172.21.15.185:53594) 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 -- 172.21.15.185:0/3391231505 learned_addr learned my addr 172.21.15.185:0/3391231505 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 10 client.0 ll_register_callbacks cb 0x29ecad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d807700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae5990 0x2b0cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2ae5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 -- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 msgr2=0x2ae5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2ae5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 -- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae5990 msgr2=0x2b0cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae5990 0x2b0cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 -- 172.21.15.185:0/3391231505 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x29e7280 con 0x283dbb0 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2ae5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9d006700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f7f94047950 tx=0x7f7f9406f000).ready entity=mon.2 client_cookie=29d74f2545be849 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9403f070 con 0x283dbb0 2020-11-16T22:18:11.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f9406f2e0 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2abee10 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f94047a70 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 10 client.5828 fetch_fsmap learned FSMap version 57 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x281cfd0 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2abee10 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 57) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f7f94045170 con 0x283dbb0 2020-11-16T22:18:11.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 10 client.5828 fetch_fsmap finished waiting for FSMap version 57 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 10 client.5828 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 20 client.5828 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2afc730 0x2b118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 4 client.5828 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x28e7c50 con 0x2afc730 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2afc730 0x2b118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 10 client.5828 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f9c805700 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2afc730 0x2b118b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c8d9e9b111967ead server_cookie=2ee23bb151f1c933 in_seq=0 out_seq=0 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 1 -- 172.21.15.185:0/3391231505 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x28e7c50 con 0x2afc730 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7f677fe700 10 client.5828 handle_command_reply: tid=0 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 client.5828 shutdown 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 msgr2=0x2b0e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f7f94047950 tx=0x7f7f9406f000).stop 2020-11-16T22:18:11.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 shutdown_connections 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.839+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2afc730 0x2b118b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae5990 0x2b0cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7f677fe700 0 client.5828 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b0e550 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/3391231505 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2ae5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 >> 172.21.15.185:0/3391231505 conn(0x2a79e40 msgr2=0x2aff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:11.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 shutdown_connections 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 1 -- 172.21.15.185:0/3391231505 wait complete. 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.843+0000 7f7fadcfb740 20 client.5828 trim_cache size 0 max 16384 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 Processor -- start 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 -- start start 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 0x2afb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29cee40 con 0x283dbb0 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae8320 con 0x28ab7b0 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ac4550 con 0x2a79e40 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 0x2afb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60160/0 (socket says 172.21.15.185:60160) 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 -- 172.21.15.185:0/1373085163 learned_addr learned my addr 172.21.15.185:0/1373085163 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:11.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9c805700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 -- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 msgr2=0x2afb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 0x2afb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 -- 172.21.15.185:0/1373085163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 msgr2=0x2afcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:11.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d807700 1 -- 172.21.15.185:0/1373085163 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0081d0 con 0x28ab7b0 2020-11-16T22:18:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9c805700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.859+0000 7f7f9d006700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 0x2afb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:11.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7f9d807700 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f7f90009bb0 tx=0x7f7f90008040).ready entity=mon.1 client_cookie=d36555e6f643731a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7f7affd700 1 -- 172.21.15.185:0/1373085163 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90009680 con 0x28ab7b0 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7f7affd700 1 -- 172.21.15.185:0/1373085163 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90016c30 con 0x28ab7b0 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7f7affd700 1 -- 172.21.15.185:0/1373085163 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90016da0 con 0x28ab7b0 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 msgr2=0x2afac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f7f90009bb0 tx=0x7f7f90008040).stop 2020-11-16T22:18:11.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1373085163 shutdown_connections 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28ab7b0 0x2afac80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a79e40 0x2afb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/1373085163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x283dbb0 0x2afcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1373085163 >> 172.21.15.185:0/1373085163 conn(0x7f7f6c002320 msgr2=0x2ac8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1373085163 shutdown_connections 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- 172.21.15.185:0/1373085163 wait complete. 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 -- start start 2020-11-16T22:18:11.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.863+0000 7f7fadcfb740 1 Processor -- start 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28ab7b0 0x2b63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 0x2b68c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ac49c0 con 0x28ab7b0 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2af7a80 con 0x2a79e40 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2af7d20 con 0x283dbb0 2020-11-16T22:18:11.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 10 client.0 ll_register_callbacks cb 0x2ac4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b63660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 0x2b68c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d006700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b63660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53608/0 (socket says 172.21.15.185:53608) 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d006700 1 -- 172.21.15.185:0/630882198 learned_addr learned my addr 172.21.15.185:0/630882198 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9c805700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28ab7b0 0x2b63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:11.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 -- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 msgr2=0x2b63660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:12.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b63660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 -- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28ab7b0 msgr2=0x2b63b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:12.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28ab7b0 0x2b63b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 -- 172.21.15.185:0/630882198 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x285b720 con 0x2a79e40 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d807700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 0x2b68c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f7f90009bb0 tx=0x7f7f9001f000).ready entity=mon.1 client_cookie=b427953954d0551b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90029440 con 0x2a79e40 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f9002add0 con 0x2a79e40 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f9401dc30 con 0x2a79e40 2020-11-16T22:18:12.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f90020630 con 0x2a79e40 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 10 client.6400 fetch_fsmap learned FSMap version 57 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x2866e00 con 0x2a79e40 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9401dc30 con 0x2a79e40 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 57) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f7f90030020 con 0x2a79e40 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 10 client.6400 fetch_fsmap finished waiting for FSMap version 57 2020-11-16T22:18:12.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 10 client.6400 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:12.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 20 client.6400 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:12.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2ae5990 0x2b6b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 4 client.6400 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x28eef50 con 0x2ae5990 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d006700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2ae5990 0x2b6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f677fe700 10 client.6400 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.867+0000 7f7f9d006700 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2ae5990 0x2b6b180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=c34528e8d5f0257e server_cookie=37b3cfac9d4009cc in_seq=0 out_seq=0 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f677fe700 1 -- 172.21.15.185:0/630882198 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x28eef50 con 0x2ae5990 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f677fe700 10 client.6400 handle_command_reply: tid=0 2020-11-16T22:18:12.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 client.6400 shutdown 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 msgr2=0x2b68c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 0x2b68c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f7f90009bb0 tx=0x7f7f9001f000).stop 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 shutdown_connections 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a79e40 0x2b68c80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2ae5990 0x2b6b180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x283dbb0 0x2b63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 --2- 172.21.15.185:0/630882198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28ab7b0 0x2b63b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f677fe700 0 client.6400 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 >> 172.21.15.185:0/630882198 conn(0x7f7f6c002320 msgr2=0x2b0df30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 shutdown_connections 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 1 -- 172.21.15.185:0/630882198 wait complete. 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7fadcfb740 20 client.6400 trim_cache size 0 max 16384 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 msgr2=0x7f7f70057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 0x7f7f70057520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f7f980af330 tx=0x7f7f88007b70).stop 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 msgr2=0x7f7f9815bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f7f980b7540 tx=0x7f7f94015000).stop 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 shutdown_connections 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7f70055060 0x7f7f70057520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7f98063ca0 0x7f7f9815bad0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f98149180 0x7f7f98146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 --2- 172.21.15.185:0/3103625062 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7f980b6bf0 0x7f7f981480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 >> 172.21.15.185:0/3103625062 conn(0x7f7f980657b0 msgr2=0x7f7f98147530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:12.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 shutdown_connections 2020-11-16T22:18:12.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:10.891+0000 7f7f9f26a700 1 -- 172.21.15.185:0/3103625062 wait complete. 2020-11-16T22:18:12.474 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:12.475 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:18:12.475 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:18:12.475 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:12.475 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:12.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 Processor -- start 2020-11-16T22:18:12.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 -- start start 2020-11-16T22:18:12.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f895c09d260 0x7f895c0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c1444d0 con 0x7f895c0b5b30 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c144770 con 0x7f895c09d260 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c144a10 con 0x7f895c0b6b30 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53620/0 (socket says 172.21.15.185:53620) 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 -- 172.21.15.185:0/270168704 learned_addr learned my addr 172.21.15.185:0/270168704 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:12.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896184a700 1 --2- 172.21.15.185:0/270168704 >> v2:172.21.15.96:3300/0 conn(0x7f895c09d260 0x7f895c0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f8961049700 1 -- 172.21.15.185:0/270168704 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421489771 0 0) 0x7f895c1444d0 con 0x7f895c0b5b30 2020-11-16T22:18:12.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 -- 172.21.15.185:0/270168704 >> v2:172.21.15.96:3300/0 conn(0x7f895c09d260 msgr2=0x7f895c0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:12.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 --2- 172.21.15.185:0/270168704 >> v2:172.21.15.96:3300/0 conn(0x7f895c09d260 0x7f895c0b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 -- 172.21.15.185:0/270168704 >> v1:172.21.15.96:6789/0 conn(0x7f895c0b5b30 legacy=0x7f895c1437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 -- 172.21.15.185:0/270168704 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c144cb0 con 0x7f895c0b6b30 2020-11-16T22:18:12.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f896284c700 1 --2- 172.21.15.185:0/270168704 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f894c009d50 tx=0x7f894c009290).ready entity=mon.2 client_cookie=22f79245cb6a7fc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:12.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f8961049700 1 -- 172.21.15.185:0/270168704 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00bbc0 con 0x7f895c0b6b30 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f8961049700 1 -- 172.21.15.185:0/270168704 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c00bd20 con 0x7f895c0b6b30 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f8961049700 1 -- 172.21.15.185:0/270168704 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c011610 con 0x7f895c0b6b30 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 -- 172.21.15.185:0/270168704 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 msgr2=0x7f895c143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.623+0000 7f89642af700 1 --2- 172.21.15.185:0/270168704 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f894c009d50 tx=0x7f894c009290).stop 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/270168704 shutdown_connections 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 --2- 172.21.15.185:0/270168704 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b6b30 0x7f895c143f00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 --2- 172.21.15.185:0/270168704 >> v2:172.21.15.96:3300/0 conn(0x7f895c09d260 0x7f895c0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/270168704 >> 172.21.15.185:0/270168704 conn(0x7f895c068330 msgr2=0x7f895c0ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:12.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/270168704 shutdown_connections 2020-11-16T22:18:12.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/270168704 wait complete. 2020-11-16T22:18:12.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 Processor -- start 2020-11-16T22:18:12.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- start start 2020-11-16T22:18:12.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c0b6b30 0x7f895c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c0b10c0 0x7f895c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b5040 con 0x7f895c0b6b30 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b1750 con 0x7f895c0b10c0 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b19b0 con 0x7f895c0b5b30 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53622/0 (socket says 172.21.15.185:53622) 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 -- 172.21.15.185:0/116952080 learned_addr learned my addr 172.21.15.185:0/116952080 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896284c700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c0b10c0 0x7f895c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896184a700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c0b6b30 0x7f895c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 -- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c0b10c0 msgr2=0x7f895c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c0b10c0 0x7f895c0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 -- 172.21.15.185:0/116952080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c0b6b30 msgr2=0x7f895c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c0b6b30 0x7f895c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 -- 172.21.15.185:0/116952080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0b1c50 con 0x7f895c0b5b30 2020-11-16T22:18:12.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f896204b700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f894400c730 tx=0x7f8944007a70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:12.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894400e490 con 0x7f895c0b5b30 2020-11-16T22:18:12.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894400e5f0 con 0x7f895c0b5b30 2020-11-16T22:18:12.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c0b1ef0 con 0x7f895c0b5b30 2020-11-16T22:18:12.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894400e9b0 con 0x7f895c0b5b30 2020-11-16T22:18:12.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.627+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0b0520 con 0x7f895c0b5b30 2020-11-16T22:18:12.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c05d4e0 con 0x7f895c0b5b30 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8944019030 con 0x7f895c0b5b30 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f895a7fc700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f896184a700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f894406d390 con 0x7f895c0b5b30 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.631+0000 7f896184a700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f895c0b3720 tx=0x7f8954006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:12.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.635+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f89440434e0 con 0x7f895c0b5b30 2020-11-16T22:18:12.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f895c060530 con 0x7f895c0b5b30 2020-11-16T22:18:12.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f895a7fc700 1 -- 172.21.15.185:0/116952080 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2521 (secure 0 0 0) 0x7f8944046b00 con 0x7f895c0b5b30 2020-11-16T22:18:12.822 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:12.822 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:07.995173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:12.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 msgr2=0x7f8938057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f895c0b3720 tx=0x7f8954006df0).stop 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 msgr2=0x7f895c153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f894400c730 tx=0x7f8944007a70).stop 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 shutdown_connections 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8938055060 0x7f8938057520 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f895c0b10c0 0x7f895c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f895c0b5b30 0x7f895c153ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 --2- 172.21.15.185:0/116952080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f895c0b6b30 0x7f895c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 >> 172.21.15.185:0/116952080 conn(0x7f895c068330 msgr2=0x7f895c0b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:12.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 shutdown_connections 2020-11-16T22:18:12.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.839+0000 7f89642af700 1 -- 172.21.15.185:0/116952080 wait complete. 2020-11-16T22:18:12.825 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 57 2020-11-16T22:18:12.836 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:18:12.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.979+0000 7f787665e700 1 Processor -- start 2020-11-16T22:18:12.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 -- start start 2020-11-16T22:18:12.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f78700b3830 0x7f7870143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7870060130 0x7f7870143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7870060970 con 0x7f78700ac820 2020-11-16T22:18:12.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700b4610 con 0x7f78700b3830 2020-11-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787665e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7870060ba0 con 0x7f7870060130 2020-11-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7870060130 0x7f7870143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7874e5b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f78700ac820 0x7f787013f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53626/0 (socket says 172.21.15.185:53626) 2020-11-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7874e5b700 1 -- 172.21.15.185:0/357802431 learned_addr learned my addr 172.21.15.185:0/357802431 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f787565c700 1 --2- 172.21.15.185:0/357802431 >> v2:172.21.15.96:3300/0 conn(0x7f78700b3830 0x7f7870143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f785ffff700 1 -- 172.21.15.185:0/357802431 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486994219 0 0) 0x7f7870060970 con 0x7f78700ac820 2020-11-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f785ffff700 1 -- 172.21.15.185:0/357802431 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7858003240 con 0x7f78700ac820 2020-11-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 -- 172.21.15.185:0/357802431 >> v2:172.21.15.96:3300/0 conn(0x7f78700b3830 msgr2=0x7f7870143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 --2- 172.21.15.185:0/357802431 >> v2:172.21.15.96:3300/0 conn(0x7f78700b3830 0x7f7870143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 -- 172.21.15.185:0/357802431 >> v1:172.21.15.185:6789/0 conn(0x7f78700ac820 legacy=0x7f787013f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 -- 172.21.15.185:0/357802431 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870060e40 con 0x7f7870060130 2020-11-16T22:18:12.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f7875e5d700 1 --2- 172.21.15.185:0/357802431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f78700b6c70 tx=0x7f78680091d0).ready entity=mon.1 client_cookie=e77f74e923af8af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:12.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f785ffff700 1 -- 172.21.15.185:0/357802431 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786800cad0 con 0x7f7870060130 2020-11-16T22:18:13.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f785ffff700 1 -- 172.21.15.185:0/357802431 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786800cc30 con 0x7f7870060130 2020-11-16T22:18:13.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.983+0000 7f785ffff700 1 -- 172.21.15.185:0/357802431 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868010410 con 0x7f7870060130 2020-11-16T22:18:13.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- 172.21.15.185:0/357802431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 msgr2=0x7f7870143f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- 172.21.15.185:0/357802431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f78700b6c70 tx=0x7f78680091d0).stop 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- 172.21.15.185:0/357802431 shutdown_connections 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- 172.21.15.185:0/357802431 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143f30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- 172.21.15.185:0/357802431 >> v2:172.21.15.96:3300/0 conn(0x7f78700b3830 0x7f7870143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- 172.21.15.185:0/357802431 >> 172.21.15.185:0/357802431 conn(0x7f7870063df0 msgr2=0x7f78700641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- 172.21.15.185:0/357802431 shutdown_connections 2020-11-16T22:18:13.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- 172.21.15.185:0/357802431 wait complete. 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 Processor -- start 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- start start 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 0x7f78701538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 0x7f7870153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7870141040 con 0x7f78700ac820 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78701542c0 con 0x7f7870060130 2020-11-16T22:18:13.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787665e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787013f610 con 0x7f78700b3830 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7875e5d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 0x7f7870153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60192/0 (socket says 172.21.15.185:60192) 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787565c700 1 -- 172.21.15.185:0/2867002251 learned_addr learned my addr 172.21.15.185:0/2867002251 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 0x7f78701538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 -- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 msgr2=0x7f7870153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:13.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 0x7f7870153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 -- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 msgr2=0x7f7870143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 -- 172.21.15.185:0/2867002251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870141270 con 0x7f78700ac820 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7875e5d700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 0x7f7870153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f787565c700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:13.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.987+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 0x7f78701538e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f786000c6e0 tx=0x7f7860007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:13.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 -- 172.21.15.185:0/2867002251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600072e0 con 0x7f78700ac820 2020-11-16T22:18:13.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 -- 172.21.15.185:0/2867002251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860007440 con 0x7f78700ac820 2020-11-16T22:18:13.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7870141510 con 0x7f78700ac820 2020-11-16T22:18:13.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 -- 172.21.15.185:0/2867002251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860005820 con 0x7f78700ac820 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78700b1220 con 0x7f78700ac820 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 -- 172.21.15.185:0/2867002251 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7860010070 con 0x7f78700ac820 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 0x7f7848057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f787565c700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 0x7f7848057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f785d7fa700 1 -- 172.21.15.185:0/2867002251 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f786006d430 con 0x7f78700ac820 2020-11-16T22:18:13.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.991+0000 7f787565c700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 0x7f7848057520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f7870141b40 tx=0x7f786c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:13.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 Processor -- start 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 -- start start 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 0x2213e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 0x2214380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x20a14b0 con 0x1f89bb0 2020-11-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2155930 con 0x2215da0 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20156f0 con 0x1ff77b0 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53642/0 (socket says 172.21.15.185:53642) 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 -- 172.21.15.185:0/2583629061 learned_addr learned my addr 172.21.15.185:0/2583629061 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7875e5d700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 0x2214380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 0x2213e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 -- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 msgr2=0x2214380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 0x2214380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 -- 172.21.15.185:0/2583629061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 msgr2=0x2213e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 0x2213e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 -- 172.21.15.185:0/2583629061 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20105a0 con 0x1ff77b0 2020-11-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7875e5d700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 0x2214380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f7874e5b700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 0x2213e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.995+0000 7f787565c700 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f786c0135e0 tx=0x7f786c014000).ready entity=mon.2 client_cookie=98a46d96439e191f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:13.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f784e7fc700 1 -- 172.21.15.185:0/2583629061 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00fa00 con 0x1ff77b0 2020-11-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f784e7fc700 1 -- 172.21.15.185:0/2583629061 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c013e00 con 0x1ff77b0 2020-11-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f784e7fc700 1 -- 172.21.15.185:0/2583629061 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c01e610 con 0x1ff77b0 2020-11-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 msgr2=0x22158b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f786c0135e0 tx=0x7f786c014000).stop 2020-11-16T22:18:13.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- 172.21.15.185:0/2583629061 shutdown_connections 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2215da0 0x2214380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x22158b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- 172.21.15.185:0/2583629061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f89bb0 0x2213e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- 172.21.15.185:0/2583629061 >> 172.21.15.185:0/2583629061 conn(0x21c5e40 msgr2=0x2212dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- 172.21.15.185:0/2583629061 shutdown_connections 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- 172.21.15.185:0/2583629061 wait complete. 2020-11-16T22:18:13.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- start start 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 Processor -- start 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x225d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 0x2259850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21426e0 con 0x1ff77b0 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2142820 con 0x1f89bb0 2020-11-16T22:18:13.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2143a90 con 0x2215da0 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:12.999+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60204/0 (socket says 172.21.15.185:60204) 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f787565c700 1 -- 172.21.15.185:0/1150459012 learned_addr learned my addr 172.21.15.185:0/1150459012 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 10 client.0 ll_register_callbacks cb 0x2138ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 0x2259850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7874e5b700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x225d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 -- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 msgr2=0x224b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 -- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 msgr2=0x225d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:13.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x225d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 -- 172.21.15.185:0/1150459012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2133280 con 0x2215da0 2020-11-16T22:18:13.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f787565c700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:13.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 0x2259850 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7868014000 tx=0x7f7868017000).ready entity=mon.2 client_cookie=f84cf6842868e0a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868020070 con 0x2215da0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21f66c0 con 0x2215da0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7868010410 con 0x2215da0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7868006ea0 con 0x2215da0 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 10 client.5849 fetch_fsmap learned FSMap version 57 2020-11-16T22:18:13.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x1f68fd0 con 0x2215da0 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21f66c0 con 0x2215da0 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 57) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f7868018c10 con 0x2215da0 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 10 client.5849 fetch_fsmap finished waiting for FSMap version 57 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 10 client.5849 resolve_mds: resolved ID 'c' to GID 6054 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 20 client.5849 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x22122f0 0x2249950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:13.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 4 client.5849 mds_command: new command op to 6054 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 --> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2033c50 con 0x22122f0 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f787565c700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x22122f0 0x2249950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f787565c700 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x22122f0 0x2249950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=10d2e8e4ca24d41b server_cookie=66d7b9229e19dd35 in_seq=0 out_seq=0 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 10 client.5849 ms_handle_connect on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 1 -- 172.21.15.185:0/1150459012 <== mds.1 v2:172.21.15.96:6833/774952173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2033c50 con 0x22122f0 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.003+0000 7f783f7fe700 10 client.5849 handle_command_reply: tid=0 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 client.5849 shutdown 2020-11-16T22:18:13.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 msgr2=0x2259850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 0x2259850 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7868014000 tx=0x7f7868017000).stop 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 shutdown_connections 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x22122f0 0x2249950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x224b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2215da0 0x2259850 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f783f7fe700 0 client.5849 ms_handle_reset on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:13.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 --2- 172.21.15.185:0/1150459012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x225d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 >> 172.21.15.185:0/1150459012 conn(0x21c5e40 msgr2=0x2213e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 shutdown_connections 2020-11-16T22:18:14.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 1 -- 172.21.15.185:0/1150459012 wait complete. 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.007+0000 7f7886351740 20 client.5849 trim_cache size 0 max 16384 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 Processor -- start 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 -- start start 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21c5e40 0x220a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x2213d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x211ae40 con 0x21c5e40 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2033c50 con 0x1f89bb0 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.023+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2218940 con 0x1ff77b0 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7874e5b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x2213d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53660/0 (socket says 172.21.15.185:53660) 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 -- 172.21.15.185:0/1409636157 learned_addr learned my addr 172.21.15.185:0/1409636157 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7874e5b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x2213d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60212/0 (socket says 172.21.15.185:60212) 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f787565c700 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21c5e40 0x220a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 -- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 msgr2=0x2213d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x2213d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 -- 172.21.15.185:0/1409636157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21c5e40 msgr2=0x220a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21c5e40 0x220a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 -- 172.21.15.185:0/1409636157 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844008160 con 0x1ff77b0 2020-11-16T22:18:14.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f786802a040 tx=0x7f7868023000).ready entity=mon.2 client_cookie=1f0979a451fcb88a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f784effd700 1 -- 172.21.15.185:0/1409636157 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868020040 con 0x1ff77b0 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f784effd700 1 -- 172.21.15.185:0/1409636157 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786800c030 con 0x1ff77b0 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f784effd700 1 -- 172.21.15.185:0/1409636157 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680187e0 con 0x1ff77b0 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 msgr2=0x2214280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f786802a040 tx=0x7f7868023000).stop 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- 172.21.15.185:0/1409636157 shutdown_connections 2020-11-16T22:18:14.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ff77b0 0x2214280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x21c5e40 0x220a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- 172.21.15.185:0/1409636157 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x2213d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- 172.21.15.185:0/1409636157 >> 172.21.15.185:0/1409636157 conn(0x7f7844002320 msgr2=0x225a520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- 172.21.15.185:0/1409636157 shutdown_connections 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- 172.21.15.185:0/1409636157 wait complete. 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- start start 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 Processor -- start 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x22140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x22af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 0x22af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2248f30 con 0x1ff77b0 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22438a0 con 0x1f89bb0 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x220ff60 con 0x21c5e40 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7886351740 10 client.0 ll_register_callbacks cb 0x2248c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x22140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7875e5d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 0x22af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f7874e5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x22af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f787565c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x22140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60220/0 (socket says 172.21.15.185:60220) 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.027+0000 7f787565c700 1 -- 172.21.15.185:0/164374409 learned_addr learned my addr 172.21.15.185:0/164374409 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 -- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 msgr2=0x22140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x22140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 -- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 msgr2=0x22af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x22af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 -- 172.21.15.185:0/164374409 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1fa7720 con 0x21c5e40 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f7875e5d700 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 0x22af630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7868029000 tx=0x7f7868033040).ready entity=mon.2 client_cookie=f45865c95da75585 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.031+0000 7f783f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.035+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868020040 con 0x21c5e40 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.035+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7868018b50 con 0x21c5e40 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.035+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f786c013e00 con 0x21c5e40 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78680187e0 con 0x21c5e40 2020-11-16T22:18:14.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 10 client.5855 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x1fb2e00 con 0x21c5e40 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c013e00 con 0x21c5e40 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f786803a480 con 0x21c5e40 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 10 client.5855 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 10 client.5855 resolve_mds: resolved ID 'c' to GID 6054 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 20 client.5855 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:14.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2215da0 0x22aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 4 client.5855 mds_command: new command op to 6054 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 --> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x203af50 con 0x2215da0 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f787565c700 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2215da0 0x22aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f783f7fe700 10 client.5855 ms_handle_connect on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.039+0000 7f787565c700 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2215da0 0x22aab90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=563b41a43fb9861a server_cookie=64692a815727fb87 in_seq=0 out_seq=0 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f783f7fe700 1 -- 172.21.15.185:0/164374409 <== mds.1 v2:172.21.15.96:6833/774952173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x203af50 con 0x2215da0 2020-11-16T22:18:14.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f783f7fe700 10 client.5855 handle_command_reply: tid=0 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 client.5855 shutdown 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 msgr2=0x22af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 0x22af630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7868029000 tx=0x7f7868033040).stop 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 shutdown_connections 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2215da0 0x22aab90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f89bb0 0x22140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x21c5e40 0x22af630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f7886351740 1 --2- 172.21.15.185:0/164374409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ff77b0 0x22af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.059+0000 7f783f7fe700 0 client.5855 ms_handle_reset on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 >> 172.21.15.185:0/164374409 conn(0x7f7844002320 msgr2=0x220a870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 shutdown_connections 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f7886351740 1 -- 172.21.15.185:0/164374409 wait complete. 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f7886351740 20 client.5855 trim_cache size 0 max 16384 2020-11-16T22:18:14.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 msgr2=0x7f7848057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 0x7f7848057520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f7870141b40 tx=0x7f786c006df0).stop 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 msgr2=0x7f78701538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 0x7f78701538e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f786000c6e0 tx=0x7f7860007a20).stop 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 shutdown_connections 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7848055060 0x7f7848057520 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7870060130 0x7f7870143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f78700b3830 0x7f7870153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 --2- 172.21.15.185:0/2867002251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78700ac820 0x7f78701538e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 >> 172.21.15.185:0/2867002251 conn(0x7f7870063df0 msgr2=0x7f78700b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 shutdown_connections 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:13.063+0000 7f787665e700 1 -- 172.21.15.185:0/2867002251 wait complete. 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:18:14.248 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:18:14.249 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:14.249 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:14.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 Processor -- start 2020-11-16T22:18:14.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 -- start start 2020-11-16T22:18:14.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9a60067070 0x7f9a600b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a60067510 con 0x7f9a600b7470 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a600677b0 con 0x7f9a60067070 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65b93700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600b17a0 con 0x7f9a600682b0 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53674/0 (socket says 172.21.15.185:53674) 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 -- 172.21.15.185:0/60395073 learned_addr learned my addr 172.21.15.185:0/60395073 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a5ffff700 1 --2- 172.21.15.185:0/60395073 >> v2:172.21.15.96:3300/0 conn(0x7f9a60067070 0x7f9a600b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 -- 172.21.15.185:0/60395073 >> v2:172.21.15.96:3300/0 conn(0x7f9a60067070 msgr2=0x7f9a600b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 --2- 172.21.15.185:0/60395073 >> v2:172.21.15.96:3300/0 conn(0x7f9a60067070 0x7f9a600b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 -- 172.21.15.185:0/60395073 >> v1:172.21.15.96:6789/0 conn(0x7f9a600b7470 legacy=0x7f9a600b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 -- 172.21.15.185:0/60395073 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600b1a40 con 0x7f9a600682b0 2020-11-16T22:18:14.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a65392700 1 --2- 172.21.15.185:0/60395073 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f9a58006fd0 tx=0x7f9a58006eb0).ready entity=mon.2 client_cookie=b58897715efda2b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a5f7fe700 1 -- 172.21.15.185:0/60395073 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58009810 con 0x7f9a600682b0 2020-11-16T22:18:14.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a5f7fe700 1 -- 172.21.15.185:0/60395073 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a58009970 con 0x7f9a600682b0 2020-11-16T22:18:14.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.399+0000 7f9a5f7fe700 1 -- 172.21.15.185:0/60395073 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58009cf0 con 0x7f9a600682b0 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/60395073 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 msgr2=0x7f9a60143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- 172.21.15.185:0/60395073 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f9a58006fd0 tx=0x7f9a58006eb0).stop 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/60395073 shutdown_connections 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- 172.21.15.185:0/60395073 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a600682b0 0x7f9a60143a20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- 172.21.15.185:0/60395073 >> v2:172.21.15.96:3300/0 conn(0x7f9a60067070 0x7f9a600b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/60395073 >> 172.21.15.185:0/60395073 conn(0x7f9a600637b0 msgr2=0x7f9a600b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/60395073 shutdown_connections 2020-11-16T22:18:14.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/60395073 wait complete. 2020-11-16T22:18:14.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 Processor -- start 2020-11-16T22:18:14.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- start start 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a600b7470 0x7f9a600b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6013c920 con 0x7f9a600b7470 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6013ba40 con 0x7f9a600682b0 2020-11-16T22:18:14.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6013bce0 con 0x7f9a60067070 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a64b91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60234/0 (socket says 172.21.15.185:60234) 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a64b91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53676/0 (socket says 172.21.15.185:53676) 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 -- 172.21.15.185:0/1798629719 learned_addr learned my addr 172.21.15.185:0/1798629719 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65392700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a600b7470 0x7f9a600b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 -- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 msgr2=0x7f9a6013d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 -- 172.21.15.185:0/1798629719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a600b7470 msgr2=0x7f9a600b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a600b7470 0x7f9a600b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 -- 172.21.15.185:0/1798629719 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60152520 con 0x7f9a600682b0 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a64b91700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:14.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5ffff700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f9a5000ef80 tx=0x7f9a500091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000cc90 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5000cdf0 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a601527c0 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50012600 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.403+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a600b0c90 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6005d200 con 0x7f9a600682b0 2020-11-16T22:18:14.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9a50022070 con 0x7f9a600682b0 2020-11-16T22:18:14.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a5cff9700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 0x7f9a30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9a5006e530 con 0x7f9a600682b0 2020-11-16T22:18:14.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a64b91700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 0x7f9a30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.407+0000 7f9a64b91700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 0x7f9a30057520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f9a48001610 tx=0x7f9a48009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.415+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9a50044790 con 0x7f9a600682b0 2020-11-16T22:18:14.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.619+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a600b13d0 con 0x7f9a600682b0 2020-11-16T22:18:14.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.619+0000 7f9a5cff9700 1 -- 172.21.15.185:0/1798629719 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2523 (secure 0 0 0) 0x7f9a50016070 con 0x7f9a600682b0 2020-11-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:14.603 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:13.031820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 msgr2=0x7f9a30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 0x7f9a30057520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f9a48001610 tx=0x7f9a48009040).stop 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 msgr2=0x7f9a600b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f9a5000ef80 tx=0x7f9a500091d0).stop 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 shutdown_connections 2020-11-16T22:18:14.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9a30055060 0x7f9a30057520 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a600b7470 0x7f9a600b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9a60067070 0x7f9a6013d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 --2- 172.21.15.185:0/1798629719 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9a600682b0 0x7f9a600b02b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 >> 172.21.15.185:0/1798629719 conn(0x7f9a600637b0 msgr2=0x7f9a6013e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 shutdown_connections 2020-11-16T22:18:14.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.623+0000 7f9a65b93700 1 -- 172.21.15.185:0/1798629719 wait complete. 2020-11-16T22:18:14.607 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 58 2020-11-16T22:18:14.617 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:18:14.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 Processor -- start 2020-11-16T22:18:14.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 -- start start 2020-11-16T22:18:14.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 0x7fa9980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa998144860 0x7fa9980ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9980ad490 con 0x7fa9980b7440 2020-11-16T22:18:14.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9980ad730 con 0x7fa998144860 2020-11-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99f10a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980ad9d0 con 0x7fa9980afc60 2020-11-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99d6a7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa998144860 0x7fa9980ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa997fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 0x7fa9980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99cea6700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa9980b7440 0x7fa99813dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53678/0 (socket says 172.21.15.185:53678) 2020-11-16T22:18:14.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa99cea6700 1 -- 172.21.15.185:0/1876122939 learned_addr learned my addr 172.21.15.185:0/1876122939 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:14.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201513867 0 0) 0x7fa9980ad490 con 0x7fa9980b7440 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.763+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa984003240 con 0x7fa9980b7440 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 msgr2=0x7fa9980b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 0x7fa9980b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 898560565 0 0) 0x7fa984003240 con 0x7fa9980b7440 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1876122939 >> v1:172.21.15.185:6789/0 conn(0x7fa9980b7440 legacy=0x7fa99813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1876122939 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99813e5b0 con 0x7fa998144860 2020-11-16T22:18:14.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa997fff700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 0x7fa9980b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa998144860 0x7fa9980ade70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fa990009c90 tx=0x7fa9900091d0).ready entity=mon.1 client_cookie=5953db9ebba13b30 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880014e0 con 0x7fa998144860 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99000b6b0 con 0x7fa998144860 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa9977fe700 1 -- 172.21.15.185:0/1876122939 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000ba30 con 0x7fa998144860 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa998144860 msgr2=0x7fa9980ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa998144860 0x7fa9980ade70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fa990009c90 tx=0x7fa9900091d0).stop 2020-11-16T22:18:14.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- 172.21.15.185:0/1876122939 shutdown_connections 2020-11-16T22:18:14.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa998144860 0x7fa9980ade70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 --2- 172.21.15.185:0/1876122939 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa9980afc60 0x7fa9980b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- 172.21.15.185:0/1876122939 >> 172.21.15.185:0/1876122939 conn(0x7fa998067860 msgr2=0x7fa998067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- 172.21.15.185:0/1876122939 shutdown_connections 2020-11-16T22:18:14.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- 172.21.15.185:0/1876122939 wait complete. 2020-11-16T22:18:14.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 Processor -- start 2020-11-16T22:18:14.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.767+0000 7fa99f10a700 1 -- start start 2020-11-16T22:18:14.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 0x7fa998152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa998144860 0x7fa9980b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981534d0 con 0x7fa9980afc60 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998141370 con 0x7fa9980b7440 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa998141f10 con 0x7fa998144860 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa998144860 0x7fa9980b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60246/0 (socket says 172.21.15.185:60246) 2020-11-16T22:18:14.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 -- 172.21.15.185:0/569343253 learned_addr learned my addr 172.21.15.185:0/569343253 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99cea6700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 0x7fa998152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 -- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa998144860 msgr2=0x7fa9980b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa998144860 0x7fa9980b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 -- 172.21.15.185:0/569343253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 msgr2=0x7fa998152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 0x7fa998152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 -- 172.21.15.185:0/569343253 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9981421b0 con 0x7fa9980b7440 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99cea6700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 0x7fa998152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa997fff700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fa98c00ef50 tx=0x7fa98c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa994ff9700 1 -- 172.21.15.185:0/569343253 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00b5c0 con 0x7fa9980b7440 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa994ff9700 1 -- 172.21.15.185:0/569343253 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00cb30 con 0x7fa9980b7440 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa994ff9700 1 -- 172.21.15.185:0/569343253 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0123f0 con 0x7fa9980b7440 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa998142450 con 0x7fa9980b7440 2020-11-16T22:18:14.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.771+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa998143650 con 0x7fa9980b7440 2020-11-16T22:18:14.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa994ff9700 1 -- 172.21.15.185:0/569343253 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa98c012760 con 0x7fa9980b7440 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa994ff9700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 0x7fa970057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa994ff9700 1 -- 172.21.15.185:0/569343253 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa98c06e3b0 con 0x7fa9980b7440 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa99cea6700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 0x7fa970057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa99cea6700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 0x7fa970057520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa988001670 tx=0x7fa988009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa9adb9b740 1 Processor -- start 2020-11-16T22:18:14.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.775+0000 7fa9adb9b740 1 -- start start 2020-11-16T22:18:14.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x25bef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 0x25bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24164b0 con 0x25c47f0 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24ca930 con 0x236c7b0 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x238a6f0 con 0x22febb0 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa997fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x25bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60256/0 (socket says 172.21.15.185:60256) 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 -- 172.21.15.185:0/3520262623 learned_addr learned my addr 172.21.15.185:0/3520262623 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 0x25bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 -- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 msgr2=0x25bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x25bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 -- 172.21.15.185:0/3520262623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 msgr2=0x25bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 0x25bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 -- 172.21.15.185:0/3520262623 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23855a0 con 0x236c7b0 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 0x25bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:14.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa99cea6700 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fa988004b70 tx=0x7fa988007000).ready entity=mon.1 client_cookie=2294cfdc2562505d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa975ffb700 1 -- 172.21.15.185:0/3520262623 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007390 con 0x236c7b0 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa975ffb700 1 -- 172.21.15.185:0/3520262623 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98801b030 con 0x236c7b0 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa975ffb700 1 -- 172.21.15.185:0/3520262623 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988015c00 con 0x236c7b0 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 msgr2=0x25c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fa988004b70 tx=0x7fa988007000).stop 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3520262623 shutdown_connections 2020-11-16T22:18:14.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x236c7b0 0x25c4300 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25c47f0 0x25bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3520262623 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x25bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3520262623 >> 172.21.15.185:0/3520262623 conn(0x253ae40 msgr2=0x25c3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3520262623 shutdown_connections 2020-11-16T22:18:14.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3520262623 wait complete. 2020-11-16T22:18:14.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 -- start start 2020-11-16T22:18:14.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.779+0000 7fa9adb9b740 1 Processor -- start 2020-11-16T22:18:14.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24b76e0 con 0x22febb0 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24b7820 con 0x25a9a10 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24b8a90 con 0x236c7b0 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 10 client.0 ll_register_callbacks cb 0x24adad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:14.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60264/0 (socket says 172.21.15.185:60264) 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa997fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1351879595 learned_addr learned my addr 172.21.15.185:0/1351879595 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99cea6700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 msgr2=0x25a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1351879595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 msgr2=0x25a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 -- 172.21.15.185:0/1351879595 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x24a8280 con 0x25a9a10 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa997fff700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fa990011000 tx=0x7fa990016000).ready entity=mon.1 client_cookie=92c617dd4b7ba0a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990022070 con 0x25a9a10 2020-11-16T22:18:14.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa99000b820 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x257fe10 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa990006ea0 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 10 client.6442 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x22ddfd0 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x257fe10 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa990011c60 con 0x25a9a10 2020-11-16T22:18:14.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 10 client.6442 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 10 client.6442 resolve_mds: resolved ID 'a' to GID 5681 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 20 client.6442 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25aa2e0 0x25cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 4 client.6442 mds_command: new command op to 5681 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 --> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x23a8c50 con 0x25aa2e0 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99cea6700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25aa2e0 0x25cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 10 client.6442 ms_handle_connect on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa99cea6700 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25aa2e0 0x25cee00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=9ce8e9aa0ae7f5a2 server_cookie=e874d0377854571f in_seq=0 out_seq=0 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 1 -- 172.21.15.185:0/1351879595 <== mds.2 v2:172.21.15.185:6834/3773729030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x23a8c50 con 0x25aa2e0 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa966ffd700 10 client.6442 handle_command_reply: tid=0 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.783+0000 7fa9adb9b740 1 client.6442 shutdown 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 msgr2=0x25b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fa990011000 tx=0x7fa990016000).stop 2020-11-16T22:18:14.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 shutdown_connections 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25aa2e0 0x25cee00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25a9a10 0x25b9880 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa966ffd700 0 client.6442 ms_handle_reset on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/1351879595 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 >> 172.21.15.185:0/1351879595 conn(0x253ae40 msgr2=0x25c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 shutdown_connections 2020-11-16T22:18:14.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 1 -- 172.21.15.185:0/1351879595 wait complete. 2020-11-16T22:18:14.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.787+0000 7fa9adb9b740 20 client.6442 trim_cache size 0 max 16384 2020-11-16T22:18:14.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.803+0000 7fa9adb9b740 1 Processor -- start 2020-11-16T22:18:14.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.803+0000 7fa9adb9b740 1 -- start start 2020-11-16T22:18:14.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x248fe40 con 0x22febb0 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d21f0 con 0x253ae40 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x25d2870 con 0x236c7b0 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99cea6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53714/0 (socket says 172.21.15.185:53714) 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3315135925 learned_addr learned my addr 172.21.15.185:0/3315135925 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa997fff700 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 msgr2=0x25c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3315135925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 msgr2=0x25bba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25bba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3315135925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0081d0 con 0x236c7b0 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa997fff700 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fa990011fd0 tx=0x7fa990022000).ready entity=mon.2 client_cookie=268173285f596c5b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9767fc700 1 -- 172.21.15.185:0/3315135925 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990035070 con 0x236c7b0 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9767fc700 1 -- 172.21.15.185:0/3315135925 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990006ca0 con 0x236c7b0 2020-11-16T22:18:14.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9767fc700 1 -- 172.21.15.185:0/3315135925 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990006ea0 con 0x236c7b0 2020-11-16T22:18:14.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 msgr2=0x25bbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fa990011fd0 tx=0x7fa990022000).stop 2020-11-16T22:18:14.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3315135925 shutdown_connections 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x236c7b0 0x25bbf50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3315135925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x22febb0 0x25bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3315135925 >> 172.21.15.185:0/3315135925 conn(0x7fa96c002320 msgr2=0x25bc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3315135925 shutdown_connections 2020-11-16T22:18:14.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3315135925 wait complete. 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 -- start start 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.807+0000 7fa9adb9b740 1 Processor -- start 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x2584530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 0x25bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x25a55c0 con 0x236c7b0 2020-11-16T22:18:14.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x25bdec0 con 0x253ae40 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x25d1da0 con 0x22febb0 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99cea6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x2584530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 10 client.0 ll_register_callbacks cb 0x2585ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60278/0 (socket says 172.21.15.185:60278) 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3142066095 learned_addr learned my addr 172.21.15.185:0/3142066095 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:14.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99cea6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x2584530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53718/0 (socket says 172.21.15.185:53718) 2020-11-16T22:18:14.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa997fff700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 0x25bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 msgr2=0x2584530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x2584530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3142066095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 msgr2=0x25bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:14.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 0x25bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 -- 172.21.15.185:0/3142066095 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x231c720 con 0x253ae40 2020-11-16T22:18:14.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa997fff700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 0x25bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:14.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99d6a7700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fa990009c90 tx=0x7fa990035000).ready entity=mon.1 client_cookie=8897431663103fa8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:14.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990022040 con 0x253ae40 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa98801b030 con 0x253ae40 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa99000ba30 con 0x253ae40 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa990020b50 con 0x253ae40 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 10 client.6448 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:14.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x2327e00 con 0x253ae40 2020-11-16T22:18:14.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98801b030 con 0x253ae40 2020-11-16T22:18:14.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa990020670 con 0x253ae40 2020-11-16T22:18:14.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 10 client.6448 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:14.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 10 client.6448 resolve_mds: resolved ID 'a' to GID 5681 2020-11-16T22:18:14.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 20 client.6448 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:14.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25a9a10 0x2620910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:14.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 4 client.6448 mds_command: new command op to 5681 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:18:14.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 --> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x23aff50 con 0x25a9a10 2020-11-16T22:18:14.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.811+0000 7fa99cea6700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25a9a10 0x2620910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:14.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.815+0000 7fa966ffd700 10 client.6448 ms_handle_connect on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:14.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.815+0000 7fa99cea6700 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25a9a10 0x2620910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=ed387755776704ae server_cookie=722dacc960a0cc1c in_seq=0 out_seq=0 2020-11-16T22:18:14.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa966ffd700 1 -- 172.21.15.185:0/3142066095 <== mds.2 v2:172.21.15.185:6834/3773729030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x23aff50 con 0x25a9a10 2020-11-16T22:18:14.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa966ffd700 10 client.6448 handle_command_reply: tid=0 2020-11-16T22:18:14.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 client.6448 shutdown 2020-11-16T22:18:14.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 msgr2=0x25bb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fa990009c90 tx=0x7fa990035000).stop 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 shutdown_connections 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x25a9a10 0x2620910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x253ae40 0x25bb920 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa966ffd700 0 client.6448 ms_handle_reset on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22febb0 0x2584530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 --2- 172.21.15.185:0/3142066095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x236c7b0 0x25bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 >> 172.21.15.185:0/3142066095 conn(0x7fa96c002320 msgr2=0x25cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 shutdown_connections 2020-11-16T22:18:14.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 1 -- 172.21.15.185:0/3142066095 wait complete. 2020-11-16T22:18:14.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.831+0000 7fa9adb9b740 20 client.6448 trim_cache size 0 max 16384 2020-11-16T22:18:14.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 msgr2=0x7fa970057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 0x7fa970057520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa988001670 tx=0x7fa988009040).stop 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 msgr2=0x7fa998143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fa98c00ef50 tx=0x7fa98c006df0).stop 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 shutdown_connections 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa970055060 0x7fa970057520 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa998144860 0x7fa9980b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9980afc60 0x7fa998152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 --2- 172.21.15.185:0/569343253 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa9980b7440 0x7fa998143af0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:14.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 >> 172.21.15.185:0/569343253 conn(0x7fa998067860 msgr2=0x7fa99813ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:14.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 shutdown_connections 2020-11-16T22:18:14.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:14.835+0000 7fa99f10a700 1 -- 172.21.15.185:0/569343253 wait complete. 2020-11-16T22:18:14.825 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:14.825 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:18:14.825 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:18:14.826 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:14.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:18:14.834 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:18:14.896 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/ 2020-11-16T22:18:14.896 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4 2020-11-16T22:18:14.896 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5 2020-11-16T22:18:14.896 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:18:14.897 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:18:14.897 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:18:14.900 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:14.901 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:14.901 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:14.902 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:14.902 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:18:14.902 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627778 2020-11-16T22:18:15.204 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:15.205 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:15.205 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:15.206 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:15.206 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:18:15.206 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:15.321 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627779 2020-11-16T22:18:15.508 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:15.509 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:15.509 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:15.509 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:15.509 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:18:15.510 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:15.661 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627780 2020-11-16T22:18:15.851 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:15.852 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:15.852 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:15.852 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:15.853 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:18:15.853 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:15.967 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627781 2020-11-16T22:18:16.155 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:16.156 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:16.156 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:16.156 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:16.156 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:18:16.157 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:16.272 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627782 2020-11-16T22:18:16.461 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:18:16.461 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:18:16.462 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:18:16.462 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:18:16.462 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:18:16.462 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:18:16.617 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627783 2020-11-16T22:18:16.808 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:18:16.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 Processor -- start 2020-11-16T22:18:16.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 -- start start 2020-11-16T22:18:16.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 0x55e84e194950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 0x55e84e21dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e84e1031f0 con 0x55e84e1551e0 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e84e0d1be0 con 0x55e84e18d620 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fced80ccc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e84e223200 con 0x55e84e189a80 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53734/0 (socket says 172.21.15.185:53734) 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec9ed3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 0x55e84e21dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 -- 172.21.15.185:0/241537808 learned_addr learned my addr 172.21.15.185:0/241537808 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:16.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec96d2700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 0x55e84e194950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 -- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 msgr2=0x55e84e21dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:16.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 0x55e84e21dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 -- 172.21.15.185:0/241537808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 msgr2=0x55e84e194950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:16.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 0x55e84e194950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 -- 172.21.15.185:0/241537808 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e84e0fb8a0 con 0x55e84e189a80 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec96d2700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 0x55e84e194950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec9ed3700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 0x55e84e21dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fcebc00b5e0 tx=0x7fcebc001720).ready entity=mon.2 client_cookie=1addf9c9dafcef9a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.859+0000 7fcec37fe700 1 -- 172.21.15.185:0/241537808 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00e430 con 0x55e84e189a80 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fcec37fe700 1 -- 172.21.15.185:0/241537808 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00e5b0 con 0x55e84e189a80 2020-11-16T22:18:16.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fcec37fe700 1 -- 172.21.15.185:0/241537808 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00e970 con 0x55e84e189a80 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 msgr2=0x55e84e21d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fcebc00b5e0 tx=0x7fcebc001720).stop 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- 172.21.15.185:0/241537808 shutdown_connections 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e18d620 0x55e84e21dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e1551e0 0x55e84e194950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 --2- 172.21.15.185:0/241537808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e189a80 0x55e84e21d5c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- 172.21.15.185:0/241537808 >> 172.21.15.185:0/241537808 conn(0x55e84e195be0 msgr2=0x55e84e21ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:16.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- 172.21.15.185:0/241537808 shutdown_connections 2020-11-16T22:18:16.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- 172.21.15.185:0/241537808 wait complete. 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 Processor -- start 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.863+0000 7fced80ccc40 1 -- start start 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 0x55e84e230340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e189a80 0x55e84e21d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e18d620 0x55e84e21aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e84e22f7b0 con 0x55e84e18d620 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e84e233c20 con 0x55e84e189a80 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e84e22f9a0 con 0x55e84e1551e0 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 0x55e84e230340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec8ed1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e189a80 0x55e84e21d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 0x55e84e230340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53738/0 (socket says 172.21.15.185:53738) 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 -- 172.21.15.185:0/1862014846 learned_addr learned my addr 172.21.15.185:0/1862014846 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:16.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec9ed3700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e18d620 0x55e84e21aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 -- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e189a80 msgr2=0x55e84e21d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e189a80 0x55e84e21d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 -- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e18d620 msgr2=0x55e84e21aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e84e18d620 0x55e84e21aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 -- 172.21.15.185:0/1862014846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e84e0f7280 con 0x55e84e1551e0 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55e84e189a80 0x55e84e21d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:16.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec96d2700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 0x55e84e230340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fceb4000db0 tx=0x7fceb4013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:16.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec1ffb700 1 -- 172.21.15.185:0/1862014846 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4012e10 con 0x55e84e1551e0 2020-11-16T22:18:16.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec1ffb700 1 -- 172.21.15.185:0/1862014846 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb401e400 con 0x55e84e1551e0 2020-11-16T22:18:16.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e84e0d5620 con 0x55e84e1551e0 2020-11-16T22:18:16.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fcec1ffb700 1 -- 172.21.15.185:0/1862014846 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb401e780 con 0x55e84e1551e0 2020-11-16T22:18:16.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.867+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e84e222f10 con 0x55e84e1551e0 2020-11-16T22:18:16.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec1ffb700 1 -- 172.21.15.185:0/1862014846 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fceb401eb30 con 0x55e84e1551e0 2020-11-16T22:18:16.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec1ffb700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fceac053ce0 0x7fceac0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec1ffb700 1 -- 172.21.15.185:0/1862014846 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fceb406f040 con 0x55e84e1551e0 2020-11-16T22:18:16.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fceac053ce0 0x7fceac0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fced80ccc40 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x55e84e21b090 0x55e84e23c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 --> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] -- osd_op(unknown.0.0:1 9.2 9:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x55e84e2336d0 con 0x55e84e21b090 2020-11-16T22:18:16.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec8ed1700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fceac053ce0 0x7fceac0561a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x55e84e21a110 tx=0x7fcebc001720).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:16.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec9ed3700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x55e84e21b090 0x55e84e23c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.871+0000 7fcec9ed3700 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x55e84e21b090 0x55e84e23c070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:16.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.875+0000 7fcec9ed3700 1 -- 172.21.15.185:0/1862014846 <== osd.6 v2:172.21.15.96:6808/15253 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v55'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcec400ca60 con 0x55e84e21b090 2020-11-16T22:18:16.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x55e84e21b090 msgr2=0x55e84e23c070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x55e84e21b090 0x55e84e23c070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fceac053ce0 msgr2=0x7fceac0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fceac053ce0 0x7fceac0561a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x55e84e21a110 tx=0x7fcebc001720).stop 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 msgr2=0x55e84e230340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 --2- 172.21.15.185:0/1862014846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55e84e1551e0 0x55e84e230340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fceb4000db0 tx=0x7fceb4013000).stop 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fcec9ed3700 1 -- 172.21.15.185:0/1862014846 reap_dead start 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 shutdown_connections 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 >> 172.21.15.185:0/1862014846 conn(0x55e84e195be0 msgr2=0x55e84e195310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:16.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 shutdown_connections 2020-11-16T22:18:16.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.879+0000 7fced80ccc40 1 -- 172.21.15.185:0/1862014846 wait complete. 2020-11-16T22:18:16.863 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:18:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 Processor -- start 2020-11-16T22:18:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 -- start start 2020-11-16T22:18:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 0x5601fc42b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc426330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 0x5601fc426820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fc39a1f0 con 0x5601fc420a80 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fc368be0 con 0x5601fc3ec1e0 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601fc42c880 con 0x5601fc424620 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb7ecef700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 0x5601fc42b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb7f4f0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 0x5601fc426820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb7ecef700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 0x5601fc42b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60310/0 (socket says 172.21.15.185:60310) 2020-11-16T22:18:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb7ecef700 1 -- 172.21.15.185:0/2035877304 learned_addr learned my addr 172.21.15.185:0/2035877304 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.955+0000 7fbb7e4ee700 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc426330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 -- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 msgr2=0x5601fc42b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 0x5601fc42b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 -- 172.21.15.185:0/2035877304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 msgr2=0x5601fc426330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc426330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:16.942 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 -- 172.21.15.185:0/2035877304 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fc3928a0 con 0x5601fc424620 2020-11-16T22:18:16.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 0x5601fc426820 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fbb7800a960 tx=0x7fbb78014000).ready entity=mon.2 client_cookie=4fec1c0c6c9bd9b1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:16.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7d4ec700 1 -- 172.21.15.185:0/2035877304 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78008990 con 0x5601fc424620 2020-11-16T22:18:16.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7d4ec700 1 -- 172.21.15.185:0/2035877304 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78008af0 con 0x5601fc424620 2020-11-16T22:18:16.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7d4ec700 1 -- 172.21.15.185:0/2035877304 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7801c660 con 0x5601fc424620 2020-11-16T22:18:16.943 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 msgr2=0x5601fc426820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 0x5601fc426820 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fbb7800a960 tx=0x7fbb78014000).stop 2020-11-16T22:18:17.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2035877304 shutdown_connections 2020-11-16T22:18:17.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc3ec1e0 0x5601fc42b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc424620 0x5601fc426820 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2035877304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc426330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2035877304 >> 172.21.15.185:0/2035877304 conn(0x5601fc42cbe0 msgr2=0x5601fc4b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2035877304 shutdown_connections 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2035877304 wait complete. 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 Processor -- start 2020-11-16T22:18:17.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- start start 2020-11-16T22:18:17.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc3ec1e0 0x5601fc4227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc3f0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc424620 0x5601fc4ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fc49f510 con 0x5601fc420a80 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fc3f0b50 con 0x5601fc424620 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb8d6e9c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601fc4b45a0 con 0x5601fc3ec1e0 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7ecef700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc3ec1e0 0x5601fc4227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7f4f0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc424620 0x5601fc4ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7ecef700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc3ec1e0 0x5601fc4227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53762/0 (socket says 172.21.15.185:53762) 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7ecef700 1 -- 172.21.15.185:0/2647513086 learned_addr learned my addr 172.21.15.185:0/2647513086 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.959+0000 7fbb7e4ee700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc3f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 -- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc3ec1e0 msgr2=0x5601fc4227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5601fc3ec1e0 0x5601fc4227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 -- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc424620 msgr2=0x5601fc4ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5601fc424620 0x5601fc4ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 -- 172.21.15.185:0/2647513086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fc38e280 con 0x5601fc420a80 2020-11-16T22:18:17.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7e4ee700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc3f0170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fbb6800dfd0 tx=0x7fbb68003950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 -- 172.21.15.185:0/2647513086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800be50 con 0x5601fc420a80 2020-11-16T22:18:17.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601fc36c620 con 0x5601fc420a80 2020-11-16T22:18:17.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 -- 172.21.15.185:0/2647513086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb680030d0 con 0x5601fc420a80 2020-11-16T22:18:17.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 -- 172.21.15.185:0/2647513086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68003490 con 0x5601fc420a80 2020-11-16T22:18:17.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601fc4284a0 con 0x5601fc420a80 2020-11-16T22:18:17.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 -- 172.21.15.185:0/2647513086 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb68018030 con 0x5601fc420a80 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb60053ce0 0x7fbb600561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb777fe700 1 -- 172.21.15.185:0/2647513086 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb6806bf70 con 0x5601fc420a80 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x5601fc422cd0 0x5601fc4d7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7ecef700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb60053ce0 0x7fbb600561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 --> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] -- osd_op(unknown.0.0:1 9.2 9:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x5601fc428780 con 0x5601fc422cd0 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.963+0000 7fbb7f4f0700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x5601fc422cd0 0x5601fc4d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.967+0000 7fbb7ecef700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb60053ce0 0x7fbb600561a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fbb70002f00 tx=0x7fbb7000c040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.967+0000 7fbb7f4f0700 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x5601fc422cd0 0x5601fc4d7720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb7f4f0700 1 -- 172.21.15.185:0/2647513086 <== osd.6 v2:172.21.15.96:6808/15253 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v55'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbb780084b0 con 0x5601fc422cd0 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x5601fc422cd0 msgr2=0x5601fc4d7720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:6808/15253,v1:172.21.15.96:6810/15253] conn(0x5601fc422cd0 0x5601fc4d7720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb60053ce0 msgr2=0x7fbb600561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb60053ce0 0x7fbb600561a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fbb70002f00 tx=0x7fbb7000c040).stop 2020-11-16T22:18:17.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 msgr2=0x5601fc3f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 --2- 172.21.15.185:0/2647513086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601fc420a80 0x5601fc3f0170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fbb6800dfd0 tx=0x7fbb68003950).stop 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb7f4f0700 1 -- 172.21.15.185:0/2647513086 reap_dead start 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 shutdown_connections 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 >> 172.21.15.185:0/2647513086 conn(0x5601fc42cbe0 msgr2=0x5601fc4aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 shutdown_connections 2020-11-16T22:18:17.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:16.971+0000 7fbb8d6e9c40 1 -- 172.21.15.185:0/2647513086 wait complete. 2020-11-16T22:18:17.358 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:18:17.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 Processor -- start 2020-11-16T22:18:17.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 -- start start 2020-11-16T22:18:17.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 0x5649c3434820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c33a3620 0x5649c3439460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c3318190 con 0x5649c33a3620 2020-11-16T22:18:17.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c33191f0 con 0x5649c339fe60 2020-11-16T22:18:17.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649c32e7be0 con 0x5649c339fa80 2020-11-16T22:18:17.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60332/0 (socket says 172.21.15.185:60332) 2020-11-16T22:18:17.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd8a1dd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 0x5649c3434820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 -- 172.21.15.185:0/2106664742 learned_addr learned my addr 172.21.15.185:0/2106664742 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd8a9de700 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c33a3620 0x5649c3439460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 -- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 msgr2=0x5649c3434820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 0x5649c3434820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 -- 172.21.15.185:0/2106664742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c33a3620 msgr2=0x5649c3439460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c33a3620 0x5649c3439460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 -- 172.21.15.185:0/2106664742 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c330d280 con 0x5649c339fe60 2020-11-16T22:18:17.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd8a1dd700 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 0x5649c3434820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:17.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd899dc700 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbd8000b590 tx=0x7fbd80001740).ready entity=mon.1 client_cookie=17fa7d5b9c8e6932 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd889da700 1 -- 172.21.15.185:0/2106664742 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000e430 con 0x5649c339fe60 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd889da700 1 -- 172.21.15.185:0/2106664742 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8000e590 con 0x5649c339fe60 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.411+0000 7fbd889da700 1 -- 172.21.15.185:0/2106664742 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000e910 con 0x5649c339fe60 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 msgr2=0x5649c33aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbd8000b590 tx=0x7fbd80001740).stop 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/2106664742 shutdown_connections 2020-11-16T22:18:17.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c33a3620 0x5649c3439460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c339fa80 0x5649c3434820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/2106664742 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fe60 0x5649c33aa730 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/2106664742 >> 172.21.15.185:0/2106664742 conn(0x5649c33abbe0 msgr2=0x5649c34320d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/2106664742 shutdown_connections 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/2106664742 wait complete. 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 Processor -- start 2020-11-16T22:18:17.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.415+0000 7fbd98bd7c40 1 -- start start 2020-11-16T22:18:17.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fa80 0x5649c3438db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c339fe60 0x5649c3419270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 0x5649c3419760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c341db90 con 0x5649c339fe60 2020-11-16T22:18:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c3439650 con 0x5649c339fa80 2020-11-16T22:18:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649c341d1c0 con 0x5649c33a3620 2020-11-16T22:18:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 0x5649c3419760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a1dd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fa80 0x5649c3438db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 0x5649c3419760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53784/0 (socket says 172.21.15.185:53784) 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 -- 172.21.15.185:0/1769887311 learned_addr learned my addr 172.21.15.185:0/1769887311 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd899dc700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c339fe60 0x5649c3419270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 -- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fa80 msgr2=0x5649c3438db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5649c339fa80 0x5649c3438db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 -- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c339fe60 msgr2=0x5649c3419270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5649c339fe60 0x5649c3419270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 -- 172.21.15.185:0/1769887311 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c32eb620 con 0x5649c33a3620 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd8a9de700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 0x5649c3419760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fbd7c005a50 tx=0x7fbd7c012000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd7affd700 1 -- 172.21.15.185:0/1769887311 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c014320 con 0x5649c33a3620 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd7affd700 1 -- 172.21.15.185:0/1769887311 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c01b040 con 0x5649c33a3620 2020-11-16T22:18:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5649c341a470 con 0x5649c33a3620 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd7affd700 1 -- 172.21.15.185:0/1769887311 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c012760 con 0x5649c33a3620 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649c34392a0 con 0x5649c33a3620 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.419+0000 7fbd7affd700 1 -- 172.21.15.185:0/1769887311 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbd7c012b90 con 0x5649c33a3620 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd7affd700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd6c053ce0 0x7fbd6c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd7affd700 1 -- 172.21.15.185:0/1769887311 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd7c06e260 con 0x5649c33a3620 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd8a1dd700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd6c053ce0 0x7fbd6c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5649c341b5b0 0x5649c3451cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- osd_op(unknown.0.0:1 9.5 9:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x5649c341bea0 con 0x5649c341b5b0 2020-11-16T22:18:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd8a1dd700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd6c053ce0 0x7fbd6c0561a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fbd84000f40 tx=0x7fbd84008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd899dc700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5649c341b5b0 0x5649c3451cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.423+0000 7fbd899dc700 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5649c341b5b0 0x5649c3451cd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd899dc700 1 -- 172.21.15.185:0/1769887311 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v55'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbd80006030 con 0x5649c341b5b0 2020-11-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5649c341b5b0 msgr2=0x5649c3451cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5649c341b5b0 0x5649c3451cd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd6c053ce0 msgr2=0x7fbd6c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd6c053ce0 0x7fbd6c0561a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fbd84000f40 tx=0x7fbd84008040).stop 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 msgr2=0x5649c3419760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 --2- 172.21.15.185:0/1769887311 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5649c33a3620 0x5649c3419760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fbd7c005a50 tx=0x7fbd7c012000).stop 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd8a9de700 1 -- 172.21.15.185:0/1769887311 reap_dead start 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 shutdown_connections 2020-11-16T22:18:18.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 >> 172.21.15.185:0/1769887311 conn(0x5649c33abbe0 msgr2=0x5649c34300c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 shutdown_connections 2020-11-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:17.427+0000 7fbd98bd7c40 1 -- 172.21.15.185:0/1769887311 wait complete. 2020-11-16T22:18:18.020 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:18:18.022 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:18.028 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:18.030 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 Processor -- start 2020-11-16T22:18:18.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 -- start start 2020-11-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 0x55618e1e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 0x55618e2856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618e1581f0 con 0x55618e1dee60 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618e126be0 con 0x55618e1dea80 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0abeebec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55618e275190 con 0x55618e1aa1e0 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0aafcc3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0aafcc3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60358/0 (socket says 172.21.15.185:60358) 2020-11-16T22:18:18.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0aafcc3700 1 -- 172.21.15.185:0/1334310162 learned_addr learned my addr 172.21.15.185:0/1334310162 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 0x55618e1e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab0cc5700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 0x55618e2856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 -- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 msgr2=0x55618e2851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 -- 172.21.15.185:0/1334310162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 msgr2=0x55618e2856b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:18.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 0x55618e2856b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 -- 172.21.15.185:0/1334310162 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618e1508a0 con 0x55618e1aa1e0 2020-11-16T22:18:18.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0aafcc3700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab0cc5700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 0x55618e2856b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 0x55618e1e73c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f0aa400b5e0 tx=0x7f0aa4001700).ready entity=mon.2 client_cookie=641435f9c577f585 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.071+0000 7f0aaecc1700 1 -- 172.21.15.185:0/1334310162 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400d4e0 con 0x55618e1aa1e0 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0aaecc1700 1 -- 172.21.15.185:0/1334310162 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa400d640 con 0x55618e1aa1e0 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0aaecc1700 1 -- 172.21.15.185:0/1334310162 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400d9c0 con 0x55618e1aa1e0 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 msgr2=0x55618e1e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 0x55618e1e73c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f0aa400b5e0 tx=0x7f0aa4001700).stop 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- 172.21.15.185:0/1334310162 shutdown_connections 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dee60 0x55618e2856b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1aa1e0 0x55618e1e73c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 --2- 172.21.15.185:0/1334310162 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1dea80 0x55618e2851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- 172.21.15.185:0/1334310162 >> 172.21.15.185:0/1334310162 conn(0x55618e1eabe0 msgr2=0x55618e1e9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- 172.21.15.185:0/1334310162 shutdown_connections 2020-11-16T22:18:18.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- 172.21.15.185:0/1334310162 wait complete. 2020-11-16T22:18:18.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 Processor -- start 2020-11-16T22:18:18.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.075+0000 7f0abeebec40 1 -- start start 2020-11-16T22:18:18.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1aa1e0 0x55618e1b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dea80 0x55618e1ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 0x55618e1e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618e271160 con 0x55618e1dea80 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618e271400 con 0x55618e1aa1e0 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55618e2716a0 con 0x55618e1dee60 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 0x55618e1e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab04c4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1aa1e0 0x55618e1b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 0x55618e1e9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53808/0 (socket says 172.21.15.185:53808) 2020-11-16T22:18:18.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 -- 172.21.15.185:0/2288506932 learned_addr learned my addr 172.21.15.185:0/2288506932 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:18.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0aafcc3700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dea80 0x55618e1ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 -- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1aa1e0 msgr2=0x55618e1b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55618e1aa1e0 0x55618e1b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 -- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dea80 msgr2=0x55618e1ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618e1dea80 0x55618e1ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 -- 172.21.15.185:0/2288506932 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618e14c280 con 0x55618e1dee60 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0ab0cc5700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 0x55618e1e9e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f0aa800f000 tx=0x7f0aa8014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0aad4be700 1 -- 172.21.15.185:0/2288506932 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801d430 con 0x55618e1dee60 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0aad4be700 1 -- 172.21.15.185:0/2288506932 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8021070 con 0x55618e1dee60 2020-11-16T22:18:18.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55618e12a620 con 0x55618e1dee60 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0aad4be700 1 -- 172.21.15.185:0/2288506932 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801d590 con 0x55618e1dee60 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.079+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618e1e6ec0 con 0x55618e1dee60 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0aad4be700 1 -- 172.21.15.185:0/2288506932 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0aa802d070 con 0x55618e1dee60 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0aad4be700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a9c053ce0 0x7f0a9c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0aad4be700 1 -- 172.21.15.185:0/2288506932 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0aa806f0b0 con 0x55618e1dee60 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a9c053ce0 0x7f0a9c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0abeebec40 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x55618e270c90 0x55618e25cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 --> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] -- osd_op(unknown.0.0:1 9.1 9:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x55618e1afc90 con 0x55618e270c90 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0ab04c4700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a9c053ce0 0x7f0a9c0561a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0aa400b4d0 tx=0x55618e1e6240).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0aafcc3700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x55618e270c90 0x55618e25cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.083+0000 7f0aafcc3700 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x55618e270c90 0x55618e25cd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0aafcc3700 1 -- 172.21.15.185:0/2288506932 <== osd.7 v2:172.21.15.96:6811/15252 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v55'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0aa0012040 con 0x55618e270c90 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x55618e270c90 msgr2=0x55618e25cd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.96:6811/15252,v1:172.21.15.96:6814/15252] conn(0x55618e270c90 0x55618e25cd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a9c053ce0 msgr2=0x7f0a9c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0a9c053ce0 0x7f0a9c0561a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0aa400b4d0 tx=0x55618e1e6240).stop 2020-11-16T22:18:18.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 msgr2=0x55618e1e9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 --2- 172.21.15.185:0/2288506932 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55618e1dee60 0x55618e1e9e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f0aa800f000 tx=0x7f0aa8014000).stop 2020-11-16T22:18:18.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0ab0cc5700 1 -- 172.21.15.185:0/2288506932 reap_dead start 2020-11-16T22:18:18.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 shutdown_connections 2020-11-16T22:18:18.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 >> 172.21.15.185:0/2288506932 conn(0x55618e1eabe0 msgr2=0x55618e1e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:18.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 shutdown_connections 2020-11-16T22:18:18.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.087+0000 7f0abeebec40 1 -- 172.21.15.185:0/2288506932 wait complete. 2020-11-16T22:18:18.683 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:18:18.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 Processor -- start 2020-11-16T22:18:18.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- start start 2020-11-16T22:18:18.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab872ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab900ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ab7e21f0 con 0x5637ab8f8590 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ab7b0be0 con 0x5637ab902150 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637ab8f7280 con 0x5637ab870440 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adec87700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab872ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60378/0 (socket says 172.21.15.185:60378) 2020-11-16T22:18:18.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 -- 172.21.15.185:0/3946669555 learned_addr learned my addr 172.21.15.185:0/3946669555 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:18.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3ade486700 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab900ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 -- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 msgr2=0x5637ab872ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:18.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab872ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 -- 172.21.15.185:0/3946669555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 msgr2=0x5637ab900ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab900ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 -- 172.21.15.185:0/3946669555 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ab7da8a0 con 0x5637ab902150 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3adf488700 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3ad0009b50 tx=0x7f3ad000c040).ready entity=mon.1 client_cookie=2e9fa108298c576f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3add484700 1 -- 172.21.15.185:0/3946669555 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000f040 con 0x5637ab902150 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3add484700 1 -- 172.21.15.185:0/3946669555 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0008be0 con 0x5637ab902150 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3add484700 1 -- 172.21.15.185:0/3946669555 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0014430 con 0x5637ab902150 2020-11-16T22:18:18.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 msgr2=0x5637ab901390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3ad0009b50 tx=0x7f3ad000c040).stop 2020-11-16T22:18:18.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- 172.21.15.185:0/3946669555 shutdown_connections 2020-11-16T22:18:18.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab901390 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab872ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 --2- 172.21.15.185:0/3946669555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab900ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- 172.21.15.185:0/3946669555 >> 172.21.15.185:0/3946669555 conn(0x5637ab871670 msgr2=0x5637ab822c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:18.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- 172.21.15.185:0/3946669555 shutdown_connections 2020-11-16T22:18:18.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.735+0000 7f3aed681c40 1 -- 172.21.15.185:0/3946669555 wait complete. 2020-11-16T22:18:18.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 Processor -- start 2020-11-16T22:18:18.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 -- start start 2020-11-16T22:18:18.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab8fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab8735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab8f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ab86cdd0 con 0x5637ab8f8590 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ab86a2b0 con 0x5637ab902150 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3aed681c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637ab8fb1a0 con 0x5637ab870440 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3adf488700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab8f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3adf488700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab8f9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60386/0 (socket says 172.21.15.185:60386) 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3adf488700 1 -- 172.21.15.185:0/2988105103 learned_addr learned my addr 172.21.15.185:0/2988105103 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3adec87700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab8fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.739+0000 7f3ade486700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab8735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 -- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 msgr2=0x5637ab8fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5637ab870440 0x5637ab8fb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 -- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 msgr2=0x5637ab8735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:18.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637ab8f8590 0x5637ab8735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 -- 172.21.15.185:0/2988105103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ab7d6280 con 0x5637ab902150 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3adf488700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab8f9220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x5637ab872b40 tx=0x7f3ad0018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3ad77fe700 1 -- 172.21.15.185:0/2988105103 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000f070 con 0x5637ab902150 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3ad77fe700 1 -- 172.21.15.185:0/2988105103 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad00136c0 con 0x5637ab902150 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637ab7b4620 con 0x5637ab902150 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3ad77fe700 1 -- 172.21.15.185:0/2988105103 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0013a40 con 0x5637ab902150 2020-11-16T22:18:18.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637ab86a4e0 con 0x5637ab902150 2020-11-16T22:18:18.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.743+0000 7f3ad77fe700 1 -- 172.21.15.185:0/2988105103 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3ad001e460 con 0x5637ab902150 2020-11-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3ad77fe700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ac0053ce0 0x7f3ac00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3adec87700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ac0053ce0 0x7f3ac00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3ad77fe700 1 -- 172.21.15.185:0/2988105103 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3ad006eb10 con 0x5637ab902150 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3aed681c40 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5637ab8699d0 0x5637ab91b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- osd_op(unknown.0.0:1 9.5 9:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x5637ab8f9710 con 0x5637ab8699d0 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3adec87700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ac0053ce0 0x7f3ac00561a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f3ad8000f40 tx=0x7f3ad80069c0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3ade486700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5637ab8699d0 0x5637ab91b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.747+0000 7f3ade486700 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5637ab8699d0 0x5637ab91b550 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3ade486700 1 -- 172.21.15.185:0/2988105103 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v55'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3ac80054b0 con 0x5637ab8699d0 2020-11-16T22:18:19.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5637ab8699d0 msgr2=0x5637ab91b550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5637ab8699d0 0x5637ab91b550 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ac0053ce0 msgr2=0x7f3ac00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ac0053ce0 0x7f3ac00561a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f3ad8000f40 tx=0x7f3ad80069c0).stop 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 msgr2=0x5637ab8f9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 --2- 172.21.15.185:0/2988105103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5637ab902150 0x5637ab8f9220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x5637ab872b40 tx=0x7f3ad0018000).stop 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3adf488700 1 -- 172.21.15.185:0/2988105103 reap_dead start 2020-11-16T22:18:19.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 shutdown_connections 2020-11-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 >> 172.21.15.185:0/2988105103 conn(0x5637ab871670 msgr2=0x5637ab8f75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 shutdown_connections 2020-11-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:18.751+0000 7f3aed681c40 1 -- 172.21.15.185:0/2988105103 wait complete. 2020-11-16T22:18:19.141 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:18:19.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.191+0000 7fc364073c40 1 Processor -- start 2020-11-16T22:18:19.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.191+0000 7fc364073c40 1 -- start start 2020-11-16T22:18:19.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 0x558d28e3f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28ee7490 0x558d28eebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d28dcb1f0 con 0x558d28e53e40 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d28d99be0 con 0x558d28e1ea40 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d28e1d110 con 0x558d28ee7490 2020-11-16T22:18:19.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60398/0 (socket says 172.21.15.185:60398) 2020-11-16T22:18:19.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 -- 172.21.15.185:0/3165122840 learned_addr learned my addr 172.21.15.185:0/3165122840 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:19.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355e7a700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28ee7490 0x558d28eebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc354e78700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 0x558d28e3f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 -- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28ee7490 msgr2=0x558d28eebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28ee7490 0x558d28eebbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 -- 172.21.15.185:0/3165122840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 msgr2=0x558d28e3f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 0x558d28e3f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 -- 172.21.15.185:0/3165122840 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d28dc38a0 con 0x558d28e1ea40 2020-11-16T22:18:19.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc354e78700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 0x558d28e3f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:19.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc355679700 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fc34000b670 tx=0x7fc340001700).ready entity=mon.1 client_cookie=1f4cef1dea99da6e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc34f7fe700 1 -- 172.21.15.185:0/3165122840 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34000d5a0 con 0x558d28e1ea40 2020-11-16T22:18:19.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc34f7fe700 1 -- 172.21.15.185:0/3165122840 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34000d700 con 0x558d28e1ea40 2020-11-16T22:18:19.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc34f7fe700 1 -- 172.21.15.185:0/3165122840 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34000da80 con 0x558d28e1ea40 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 msgr2=0x558d28e1cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fc34000b670 tx=0x7fc340001700).stop 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- 172.21.15.185:0/3165122840 shutdown_connections 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e1ea40 0x558d28e1cc20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28ee7490 0x558d28eebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 --2- 172.21.15.185:0/3165122840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28e53e40 0x558d28e3f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- 172.21.15.185:0/3165122840 >> 172.21.15.185:0/3165122840 conn(0x558d28e57370 msgr2=0x558d28e1c390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- 172.21.15.185:0/3165122840 shutdown_connections 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.195+0000 7fc364073c40 1 -- 172.21.15.185:0/3165122840 wait complete. 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 Processor -- start 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 -- start start 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28e1ea40 0x558d28e32be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e53e40 0x558d28e330d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28ee7490 0x558d28e306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d28e335c0 con 0x558d28ee7490 2020-11-16T22:18:19.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d28eea8f0 con 0x558d28e53e40 2020-11-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc364073c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d28e3e960 con 0x558d28e1ea40 2020-11-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc354e78700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e53e40 0x558d28e330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28e1ea40 0x558d28e32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc354e78700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e53e40 0x558d28e330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60406/0 (socket says 172.21.15.185:60406) 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc354e78700 1 -- 172.21.15.185:0/2015751821 learned_addr learned my addr 172.21.15.185:0/2015751821 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355e7a700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28ee7490 0x558d28e306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 -- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e53e40 msgr2=0x558d28e330d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558d28e53e40 0x558d28e330d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 -- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28ee7490 msgr2=0x558d28e306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28ee7490 0x558d28e306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 -- 172.21.15.185:0/2015751821 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d28dbf280 con 0x558d28e1ea40 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355e7a700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d28ee7490 0x558d28e306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.199+0000 7fc355679700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28e1ea40 0x558d28e32be0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fc340011000 tx=0x7fc340001700).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 -- 172.21.15.185:0/2015751821 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34000eaa0 con 0x558d28e1ea40 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 -- 172.21.15.185:0/2015751821 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34000ec00 con 0x558d28e1ea40 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 -- 172.21.15.185:0/2015751821 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34001b440 con 0x558d28e1ea40 2020-11-16T22:18:19.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558d28d9d620 con 0x558d28e1ea40 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d28efb8d0 con 0x558d28e1ea40 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 -- 172.21.15.185:0/2015751821 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc34001b830 con 0x558d28e1ea40 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc33c053ce0 0x7fc33c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc34dffb700 1 -- 172.21.15.185:0/2015751821 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc34006f4a0 con 0x558d28e1ea40 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc354e78700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc33c053ce0 0x7fc33c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc364073c40 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x558d28ee68b0 0x558d28e3ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- osd_op(unknown.0.0:1 9.5 9:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e55) v8 -- 0x558d28f03ef0 con 0x558d28ee68b0 2020-11-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc354e78700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc33c053ce0 0x7fc33c0561a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x558d28e40060 tx=0x7fc348016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.203+0000 7fc355e7a700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x558d28ee68b0 0x558d28e3ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.207+0000 7fc355e7a700 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x558d28ee68b0 0x558d28e3ccc0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc355e7a700 1 -- 172.21.15.185:0/2015751821 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v55'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc350002800 con 0x558d28ee68b0 2020-11-16T22:18:19.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x558d28ee68b0 msgr2=0x558d28e3ccc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x558d28ee68b0 0x558d28e3ccc0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc33c053ce0 msgr2=0x7fc33c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc33c053ce0 0x7fc33c0561a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x558d28e40060 tx=0x7fc348016000).stop 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28e1ea40 msgr2=0x558d28e32be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 --2- 172.21.15.185:0/2015751821 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558d28e1ea40 0x558d28e32be0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fc340011000 tx=0x7fc340001700).stop 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc355e7a700 1 -- 172.21.15.185:0/2015751821 reap_dead start 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 shutdown_connections 2020-11-16T22:18:19.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 >> 172.21.15.185:0/2015751821 conn(0x558d28e57370 msgr2=0x558d28e3f710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:19.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 shutdown_connections 2020-11-16T22:18:19.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.211+0000 7fc364073c40 1 -- 172.21.15.185:0/2015751821 wait complete. 2020-11-16T22:18:19.812 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:19.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 Processor -- start 2020-11-16T22:18:19.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- start start 2020-11-16T22:18:19.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f31d80b2160 0x7f31d8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d8098cc0 con 0x7f31d80b17e0 2020-11-16T22:18:19.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80b3950 con 0x7f31d8066240 2020-11-16T22:18:19.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d814c4f0 con 0x7f31d80b2160 2020-11-16T22:18:19.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dddb2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f31d8066240 0x7f31d8065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53856/0 (socket says 172.21.15.185:53856) 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dddb2700 1 -- 172.21.15.185:0/3665834342 learned_addr learned my addr 172.21.15.185:0/3665834342 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dcdb0700 1 --2- 172.21.15.185:0/3665834342 >> v2:172.21.15.96:3300/0 conn(0x7f31d80b2160 0x7f31d8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117932761 0 0) 0x7f31d80b3950 con 0x7f31d8066240 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b8003240 con 0x7f31d8066240 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115555428 0 0) 0x7f31d8098cc0 con 0x7f31d80b17e0 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d80b3950 con 0x7f31d80b17e0 2020-11-16T22:18:19.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dcdb0700 1 -- 172.21.15.185:0/3665834342 >> v1:172.21.15.185:6789/0 conn(0x7f31d8066240 legacy=0x7f31d8065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dcdb0700 1 -- 172.21.15.185:0/3665834342 >> v1:172.21.15.96:6789/0 conn(0x7f31d80b17e0 legacy=0x7f31d814bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dcdb0700 1 -- 172.21.15.185:0/3665834342 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80b3bf0 con 0x7f31d80b2160 2020-11-16T22:18:19.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31dcdb0700 1 --2- 172.21.15.185:0/3665834342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8067fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f31c000b9a0 tx=0x7f31c0006df0).ready entity=mon.0 client_cookie=bfdffc4b14dc6255 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0009680 con 0x7f31d80b2160 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c00097e0 con 0x7f31d80b2160 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31d7fff700 1 -- 172.21.15.185:0/3665834342 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0009b60 con 0x7f31d80b2160 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- 172.21.15.185:0/3665834342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 msgr2=0x7f31d8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 --2- 172.21.15.185:0/3665834342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8067fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f31c000b9a0 tx=0x7f31c0006df0).stop 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- 172.21.15.185:0/3665834342 shutdown_connections 2020-11-16T22:18:19.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 --2- 172.21.15.185:0/3665834342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8067fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- 172.21.15.185:0/3665834342 >> 172.21.15.185:0/3665834342 conn(0x7f31d80604a0 msgr2=0x7f31d80608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- 172.21.15.185:0/3665834342 shutdown_connections 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.967+0000 7f31df815700 1 -- 172.21.15.185:0/3665834342 wait complete. 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 Processor -- start 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- start start 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 0x7f31d808ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8095af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d805cd50 con 0x7f31d80b2160 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80ad5c0 con 0x7f31d80b17e0 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80ad860 con 0x7f31d8066240 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dd5b1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 0x7f31d808ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60422/0 (socket says 172.21.15.185:60422) 2020-11-16T22:18:19.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dd5b1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 0x7f31d808ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53864/0 (socket says 172.21.15.185:53864) 2020-11-16T22:18:19.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 -- 172.21.15.185:0/119582741 learned_addr learned my addr 172.21.15.185:0/119582741 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:19.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dddb2700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8095af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:19.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 -- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 msgr2=0x7f31d808ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 0x7f31d808ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 -- 172.21.15.185:0/119582741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 msgr2=0x7f31d8095af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8095af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 -- 172.21.15.185:0/119582741 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80ad040 con 0x7f31d80b17e0 2020-11-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dddb2700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8095af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31dcdb0700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f31c0009380 tx=0x7f31c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:19.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0011d70 con 0x7f31d80b17e0 2020-11-16T22:18:20.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c001e460 con 0x7f31d80b17e0 2020-11-16T22:18:20.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c001e800 con 0x7f31d80b17e0 2020-11-16T22:18:20.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d808f0b0 con 0x7f31d80b17e0 2020-11-16T22:18:20.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.971+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8158730 con 0x7f31d80b17e0 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d80ad1d0 con 0x7f31d80b17e0 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f31c001a040 con 0x7f31d80b17e0 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31d57fa700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 0x7f31b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f31c006f080 con 0x7f31d80b17e0 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31dd5b1700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 0x7f31b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.975+0000 7f31dd5b1700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 0x7f31b4057520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f31c8007fd0 tx=0x7f31c8007d10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:19.983+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f31c0045390 con 0x7f31d80b17e0 2020-11-16T22:18:20.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.187+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31d8158a30 con 0x7f31d80b17e0 2020-11-16T22:18:20.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.187+0000 7f31d57fa700 1 -- 172.21.15.185:0/119582741 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2523 (secure 0 0 0) 0x7f31c0023070 con 0x7f31d80b17e0 2020-11-16T22:18:20.169 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:20.170 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:13.031820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 msgr2=0x7f31b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 0x7f31b4057520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f31c8007fd0 tx=0x7f31c8007d10).stop 2020-11-16T22:18:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 msgr2=0x7f31d8095600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f31c0009380 tx=0x7f31c0015000).stop 2020-11-16T22:18:20.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 shutdown_connections 2020-11-16T22:18:20.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f31b4055060 0x7f31b4057520 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31d80b2160 0x7f31d8095af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31d8066240 0x7f31d808ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 --2- 172.21.15.185:0/119582741 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31d80b17e0 0x7f31d8095600 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 >> 172.21.15.185:0/119582741 conn(0x7f31d80604a0 msgr2=0x7f31d814e620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 shutdown_connections 2020-11-16T22:18:20.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.191+0000 7f31df815700 1 -- 172.21.15.185:0/119582741 wait complete. 2020-11-16T22:18:20.376 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 58 2020-11-16T22:18:20.377 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub start /d1/d2/d3 recursive force 2020-11-16T22:18:20.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 Processor -- start 2020-11-16T22:18:20.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 -- start start 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540148160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540144170 con 0x7fd5400b7170 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5400b6ed0 con 0x7fd5400acee0 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd544cf1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540144f80 con 0x7fd5400b7550 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd5400acee0 0x7fd54013dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53870/0 (socket says 172.21.15.185:53870) 2020-11-16T22:18:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53ffff700 1 -- 172.21.15.185:0/1514643560 learned_addr learned my addr 172.21.15.185:0/1514643560 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53effd700 1 --2- 172.21.15.185:0/1514643560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435137697 0 0) 0x7fd5400b6ed0 con 0x7fd5400acee0 2020-11-16T22:18:20.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003240 con 0x7fd5400acee0 2020-11-16T22:18:20.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53effd700 1 -- 172.21.15.185:0/1514643560 >> v1:172.21.15.185:6789/0 conn(0x7fd5400acee0 legacy=0x7fd54013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53effd700 1 -- 172.21.15.185:0/1514643560 >> v1:172.21.15.96:6789/0 conn(0x7fd5400b7170 legacy=0x7fd540144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740563971 0 0) 0x7fd540144170 con 0x7fd5400b7170 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53effd700 1 -- 172.21.15.185:0/1514643560 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400b6c30 con 0x7fd5400b7550 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53effd700 1 --2- 172.21.15.185:0/1514643560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540148160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fd53400b9d0 tx=0x7fd534006df0).ready entity=mon.2 client_cookie=1268706f43b9e344 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340097b0 con 0x7fd5400b7550 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534009910 con 0x7fd5400b7550 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.531+0000 7fd53e7fc700 1 -- 172.21.15.185:0/1514643560 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534009c90 con 0x7fd5400b7550 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- 172.21.15.185:0/1514643560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 msgr2=0x7fd540148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1514643560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540148160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fd53400b9d0 tx=0x7fd534006df0).stop 2020-11-16T22:18:20.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- 172.21.15.185:0/1514643560 shutdown_connections 2020-11-16T22:18:20.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1514643560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540148160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- 172.21.15.185:0/1514643560 >> 172.21.15.185:0/1514643560 conn(0x7fd540067860 msgr2=0x7fd540067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- 172.21.15.185:0/1514643560 shutdown_connections 2020-11-16T22:18:20.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- 172.21.15.185:0/1514643560 wait complete. 2020-11-16T22:18:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 Processor -- start 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- start start 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5400b7170 0x7fd540153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54013ff60 con 0x7fd5400b7170 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54013eab0 con 0x7fd5400acee0 2020-11-16T22:18:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd544cf1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54013ed50 con 0x7fd5400b7550 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60434/0 (socket says 172.21.15.185:60434) 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1098431086 learned_addr learned my addr 172.21.15.185:0/1098431086 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.535+0000 7fd53effd700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5400b7170 0x7fd540153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 msgr2=0x7fd540153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1098431086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5400b7170 msgr2=0x7fd540153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5400b7170 0x7fd540153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1098431086 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400b0b70 con 0x7fd5400acee0 2020-11-16T22:18:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fd53000c710 tx=0x7fd530007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 -- 172.21.15.185:0/1098431086 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000e490 con 0x7fd5400acee0 2020-11-16T22:18:20.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 -- 172.21.15.185:0/1098431086 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000e5f0 con 0x7fd5400acee0 2020-11-16T22:18:20.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 -- 172.21.15.185:0/1098431086 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000e9b0 con 0x7fd5400acee0 2020-11-16T22:18:20.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd540153610 con 0x7fd5400acee0 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5401544c0 con 0x7fd5400acee0 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 -- 172.21.15.185:0/1098431086 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd530019030 con 0x7fd5400acee0 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 0x7fd518057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53effd700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 0x7fd518057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd51f7fe700 1 -- 172.21.15.185:0/1098431086 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd53006d210 con 0x7fd5400acee0 2020-11-16T22:18:20.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.539+0000 7fd53effd700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 0x7fd518057520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fd53400b9d0 tx=0x7fd534015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 Processor -- start 2020-11-16T22:18:20.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 -- start start 2020-11-16T22:18:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1beef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7c560 con 0x19d2860 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b309e0 con 0x1964c60 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19f07a0 con 0x1bf0e50 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd53effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1beef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53890/0 (socket says 172.21.15.185:53890) 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd53ffff700 1 -- 172.21.15.185:0/3544146537 learned_addr learned my addr 172.21.15.185:0/3544146537 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.543+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 -- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 msgr2=0x1beef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1beef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 -- 172.21.15.185:0/3544146537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 msgr2=0x1bf0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 -- 172.21.15.185:0/3544146537 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19eb650 con 0x1bf0e50 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53ffff700 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fd5380073d0 tx=0x7fd538007ca0).ready entity=mon.2 client_cookie=99904b05ba1c2acc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd51cff9700 1 -- 172.21.15.185:0/3544146537 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538006a50 con 0x1bf0e50 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd51cff9700 1 -- 172.21.15.185:0/3544146537 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538001d00 con 0x1bf0e50 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd51cff9700 1 -- 172.21.15.185:0/3544146537 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538002080 con 0x1bf0e50 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 msgr2=0x1bef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fd5380073d0 tx=0x7fd538007ca0).stop 2020-11-16T22:18:20.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- 172.21.15.185:0/3544146537 shutdown_connections 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1bef430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3544146537 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1beef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- 172.21.15.185:0/3544146537 >> 172.21.15.185:0/3544146537 conn(0x1ba0ef0 msgr2=0x1bede80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- 172.21.15.185:0/3544146537 shutdown_connections 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- 172.21.15.185:0/3544146537 wait complete. 2020-11-16T22:18:20.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- start start 2020-11-16T22:18:20.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 Processor -- start 2020-11-16T22:18:20.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1c271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1c1ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b1d790 con 0x19d2860 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b1d8d0 con 0x1964c60 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b1eb40 con 0x1bf0e50 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd5549e4740 10 client.0 ll_register_callbacks cb 0x1b13b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:20.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.547+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60448/0 (socket says 172.21.15.185:60448) 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1946932509 learned_addr learned my addr 172.21.15.185:0/1946932509 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53ffff700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1c1ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53effd700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1c271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 msgr2=0x1c1ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1c1ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 msgr2=0x1c271c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1c271c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1946932509 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b0e330 con 0x1964c60 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53effd700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1c271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd53000cfd0 tx=0x7fd530010000).ready entity=mon.1 client_cookie=8d6cd98694c910fd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53006b630 con 0x1964c60 2020-11-16T22:18:20.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd5300462a0 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1bd1770 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd530046c70 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 10 client.6529 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x1944080 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1bd1770 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd530031020 con 0x1964c60 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 10 client.6529 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:20.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 10 client.6529 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 20 client.6529 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf1f90 0x1c38080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 4 client.6529 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1a0ed00 con 0x1bf1f90 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53effd700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf1f90 0x1c38080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd53effd700 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf1f90 0x1c38080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=51747d6da7dca44d server_cookie=f610e61746a457d5 in_seq=0 out_seq=0 2020-11-16T22:18:20.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 10 client.6529 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 1 -- 172.21.15.185:0/1946932509 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1a0ed00 con 0x1bf1f90 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 10 client.6529 handle_command_reply: tid=0 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 client.6529 shutdown 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 msgr2=0x1c267e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd53000cfd0 tx=0x7fd530010000).stop 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 shutdown_connections 2020-11-16T22:18:20.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1c267e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf1f90 0x1c38080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1bf0e50 0x1c1ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1946932509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1c271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd515ffb700 0 client.6529 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.551+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 >> 172.21.15.185:0/1946932509 conn(0x1ba0ef0 msgr2=0x1beeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.555+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 shutdown_connections 2020-11-16T22:18:20.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.555+0000 7fd5549e4740 1 -- 172.21.15.185:0/1946932509 wait complete. 2020-11-16T22:18:20.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.555+0000 7fd5549e4740 20 client.6529 trim_cache size 0 max 16384 2020-11-16T22:18:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.571+0000 7fd5549e4740 1 Processor -- start 2020-11-16T22:18:20.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.571+0000 7fd5549e4740 1 -- start start 2020-11-16T22:18:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 0x1bedc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1bf14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af5ef0 con 0x19d2860 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c23a20 con 0x1964c60 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1bd1770 con 0x1ba0ef0 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 0x1bedc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1bf14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 0x1bedc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53900/0 (socket says 172.21.15.185:53900) 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53f7fe700 1 -- 172.21.15.185:0/3147549976 learned_addr learned my addr 172.21.15.185:0/3147549976 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53ffff700 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 -- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 msgr2=0x1bedc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 0x1bedc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 -- 172.21.15.185:0/3147549976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 msgr2=0x1bf1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 -- 172.21.15.185:0/3147549976 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c008160 con 0x1964c60 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53ffff700 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd53effd700 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1bf14a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fd5400bb920 tx=0x7fd534013000).ready entity=mon.1 client_cookie=ca45ca35aea933ef server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd51d7fa700 1 -- 172.21.15.185:0/3147549976 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534011c80 con 0x1964c60 2020-11-16T22:18:20.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd51d7fa700 1 -- 172.21.15.185:0/3147549976 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534011de0 con 0x1964c60 2020-11-16T22:18:20.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd51d7fa700 1 -- 172.21.15.185:0/3147549976 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53401e640 con 0x1964c60 2020-11-16T22:18:20.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 msgr2=0x1bf14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1bf14a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7fd5400bb920 tx=0x7fd534013000).stop 2020-11-16T22:18:20.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- 172.21.15.185:0/3147549976 shutdown_connections 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19d2860 0x1bf1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ba0ef0 0x1bedc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- 172.21.15.185:0/3147549976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1964c60 0x1bf14a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- 172.21.15.185:0/3147549976 >> 172.21.15.185:0/3147549976 conn(0x7fd50c002320 msgr2=0x1c1eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- 172.21.15.185:0/3147549976 shutdown_connections 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- 172.21.15.185:0/3147549976 wait complete. 2020-11-16T22:18:20.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- start start 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 Processor -- start 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 0x1c2eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 0x1c26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 0x1c26b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c347d0 con 0x1ba0ef0 2020-11-16T22:18:20.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c2d320 con 0x19d2860 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1be5850 con 0x1964c60 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.575+0000 7fd5549e4740 10 client.0 ll_register_callbacks cb 0x1beaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 0x1c2eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 0x1c26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 0x1c2eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53906/0 (socket says 172.21.15.185:53906) 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53f7fe700 1 -- 172.21.15.185:0/1017119414 learned_addr learned my addr 172.21.15.185:0/1017119414 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53ffff700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 0x1c26b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 -- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 msgr2=0x1c2eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 0x1c2eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 -- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 msgr2=0x1c26b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 0x1c26b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 -- 172.21.15.185:0/1017119414 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19827d0 con 0x19d2860 2020-11-16T22:18:20.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53ffff700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 0x1c26b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53effd700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 0x1c26630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fd534006e30 tx=0x7fd53402c000).ready entity=mon.1 client_cookie=464e7f167dba4d2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534011c80 con 0x19d2860 2020-11-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd534023040 con 0x19d2860 2020-11-16T22:18:20.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd538001d00 con 0x19d2860 2020-11-16T22:18:20.779 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:18:20.586+0000 7f4f66994700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1) n(v4 rc2020-11-16T22:17:52.371298+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@5},l=6346 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5594ad0f2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:17:52.371298+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.780 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:18:20.590+0000 7f4f6c9a0700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1) n(v3 rc2020-11-16T22:17:52.371298+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6346=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5594ad0f2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:17:52.371298+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.780 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:18:20.586+0000 7f88859b1700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1) n(v2 rc2020-11-16T22:17:52.371298+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@4},l=6346 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55dd1ad82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:17:52.371298+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.780 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:18:20.590+0000 7f888b9bd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1) n(v1 rc2020-11-16T22:17:52.371298+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6346=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55dd1ad82c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:17:52.371298+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd53401ee50 con 0x19d2860 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 10 client.6535 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x198deb0 con 0x19d2860 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538001d00 con 0x19d2860 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd534028040 con 0x19d2860 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 10 client.6535 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:20.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 10 client.6535 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 20 client.6535 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf0e50 0x1c81c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 4 client.6535 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x1a16000 con 0x1bf0e50 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf0e50 0x1c81c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 10 client.6535 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:20.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd53f7fe700 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf0e50 0x1c81c80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fdb7f58079d18f08 server_cookie=7974f9c9f7a622d in_seq=0 out_seq=0 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 1 -- 172.21.15.185:0/1017119414 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1a16000 con 0x1bf0e50 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd515ffb700 10 client.6535 handle_command_reply: tid=0 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 client.6535 shutdown 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 msgr2=0x1c26630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.579+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 0x1c26630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7fd534006e30 tx=0x7fd53402c000).stop 2020-11-16T22:18:20.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 shutdown_connections 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1bf0e50 0x1c81c80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ba0ef0 0x1c26b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1964c60 0x1c2eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd515ffb700 0 client.6535 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 --2- 172.21.15.185:0/1017119414 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d2860 0x1c26630 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 >> 172.21.15.185:0/1017119414 conn(0x7fd50c002320 msgr2=0x1c1eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.784 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 shutdown_connections 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 1 -- 172.21.15.185:0/1017119414 wait complete. 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd5549e4740 20 client.6535 trim_cache size 0 max 16384 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 msgr2=0x7fd518057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 0x7fd518057520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fd53400b9d0 tx=0x7fd534015000).stop 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 msgr2=0x7fd5400b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fd53000c710 tx=0x7fd530007a60).stop 2020-11-16T22:18:20.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 shutdown_connections 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd518055060 0x7fd518057520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5400acee0 0x7fd5400b32f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5400b7550 0x7fd540153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 --2- 172.21.15.185:0/1098431086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5400b7170 0x7fd540153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 >> 172.21.15.185:0/1098431086 conn(0x7fd540067860 msgr2=0x7fd5400b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 shutdown_connections 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.583+0000 7fd544cf1700 1 -- 172.21.15.185:0/1098431086 wait complete. 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0, 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stdout: "scrub_tag": "60107533-a7d2-43e9-b4cd-7eaa98fbbc4b", 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stdout: "mode": "asynchronous" 2020-11-16T22:18:20.787 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:20.788 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:18:20.583+0000 7f233a740700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1) n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@4},l=6346 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a2715fa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.788 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:18:20.587+0000 7f233a740700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/AsLsXsFsx@2},l=6346 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a2715fa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:18:20.788 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:20.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.935+0000 7fc40e822700 1 Processor -- start 2020-11-16T22:18:20.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.935+0000 7fc40e822700 1 -- start start 2020-11-16T22:18:20.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc408066680 0x7fc408066a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc408149350 con 0x7fc4080abef0 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080685c0 con 0x7fc408066680 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4081490b0 con 0x7fc408060c50 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53914/0 (socket says 172.21.15.185:53914) 2020-11-16T22:18:20.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 -- 172.21.15.185:0/1785248797 learned_addr learned my addr 172.21.15.185:0/1785248797 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d820700 1 --2- 172.21.15.185:0/1785248797 >> v2:172.21.15.96:3300/0 conn(0x7fc408066680 0x7fc408066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 -- 172.21.15.185:0/1785248797 >> v2:172.21.15.96:3300/0 conn(0x7fc408066680 msgr2=0x7fc408066a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:20.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 --2- 172.21.15.185:0/1785248797 >> v2:172.21.15.96:3300/0 conn(0x7fc408066680 0x7fc408066a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 -- 172.21.15.185:0/1785248797 >> v1:172.21.15.96:6789/0 conn(0x7fc4080abef0 legacy=0x7fc4080b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 -- 172.21.15.185:0/1785248797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080682b0 con 0x7fc408060c50 2020-11-16T22:18:20.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40d01f700 1 --2- 172.21.15.185:0/1785248797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fc404009d50 tx=0x7fc404009290).ready entity=mon.2 client_cookie=e229e62050e29f5f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40c81e700 1 -- 172.21.15.185:0/1785248797 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400bc50 con 0x7fc408060c50 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40c81e700 1 -- 172.21.15.185:0/1785248797 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400bdb0 con 0x7fc408060c50 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40c81e700 1 -- 172.21.15.185:0/1785248797 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011610 con 0x7fc408060c50 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- 172.21.15.185:0/1785248797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 msgr2=0x7fc4081484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 --2- 172.21.15.185:0/1785248797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fc404009d50 tx=0x7fc404009290).stop 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- 172.21.15.185:0/1785248797 shutdown_connections 2020-11-16T22:18:20.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 --2- 172.21.15.185:0/1785248797 >> v2:172.21.15.96:3300/0 conn(0x7fc408066680 0x7fc408066a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 --2- 172.21.15.185:0/1785248797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408060c50 0x7fc4081484e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- 172.21.15.185:0/1785248797 >> 172.21.15.185:0/1785248797 conn(0x7fc408060130 msgr2=0x7fc408060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:20.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- 172.21.15.185:0/1785248797 shutdown_connections 2020-11-16T22:18:20.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.939+0000 7fc40e822700 1 -- 172.21.15.185:0/1785248797 wait complete. 2020-11-16T22:18:20.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 Processor -- start 2020-11-16T22:18:20.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- start start 2020-11-16T22:18:20.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 0x7fc408142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 0x7fc4081432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 0x7fc4080afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080acea0 con 0x7fc408060c50 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408144330 con 0x7fc4080abef0 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080ad0d0 con 0x7fc408066680 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40d01f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 0x7fc4081432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 0x7fc4080afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40d01f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 0x7fc4081432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53922/0 (socket says 172.21.15.185:53922) 2020-11-16T22:18:20.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40d01f700 1 -- 172.21.15.185:0/2408189528 learned_addr learned my addr 172.21.15.185:0/2408189528 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40d820700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 0x7fc408142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 -- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 msgr2=0x7fc4081432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 0x7fc4081432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 -- 172.21.15.185:0/2408189528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 msgr2=0x7fc408142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 0x7fc408142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 -- 172.21.15.185:0/2408189528 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080ad370 con 0x7fc4080abef0 2020-11-16T22:18:20.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40d820700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 0x7fc408142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e021700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 0x7fc4080afdf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fc40000c6e0 tx=0x7fc400007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40000f470 con 0x7fc4080abef0 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40000f5d0 con 0x7fc4080abef0 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40000f950 con 0x7fc4080abef0 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4080ad810 con 0x7fc4080abef0 2020-11-16T22:18:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.943+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40814c2d0 con 0x7fc4080abef0 2020-11-16T22:18:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40805d200 con 0x7fc4080abef0 2020-11-16T22:18:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc400019030 con 0x7fc4080abef0 2020-11-16T22:18:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc3fdffb700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 0x7fc3e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc40006cf30 con 0x7fc4080abef0 2020-11-16T22:18:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc40d820700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 0x7fc3e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.947+0000 7fc40d820700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 0x7fc3e0057520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fc4080ac930 tx=0x7fc3f8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:20.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:20.951+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc400043240 con 0x7fc4080abef0 2020-11-16T22:18:21.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.155+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4080ad500 con 0x7fc4080abef0 2020-11-16T22:18:21.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc3fdffb700 1 -- 172.21.15.185:0/2408189528 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2523 (secure 0 0 0) 0x7fc400007bc0 con 0x7fc4080abef0 2020-11-16T22:18:21.140 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:21.140 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:13.031820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:21.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 msgr2=0x7fc3e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 0x7fc3e0057520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fc4080ac930 tx=0x7fc3f8006df0).stop 2020-11-16T22:18:21.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 msgr2=0x7fc4080afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 0x7fc4080afdf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7fc40000c6e0 tx=0x7fc400007aa0).stop 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 shutdown_connections 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3e0055060 0x7fc3e0057520 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4080abef0 0x7fc4080afdf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc408060c50 0x7fc408142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 --2- 172.21.15.185:0/2408189528 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc408066680 0x7fc4081432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 >> 172.21.15.185:0/2408189528 conn(0x7fc408060130 msgr2=0x7fc4080b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 shutdown_connections 2020-11-16T22:18:21.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.159+0000 7fc40e822700 1 -- 172.21.15.185:0/2408189528 wait complete. 2020-11-16T22:18:21.144 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 58 2020-11-16T22:18:21.155 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.291+0000 7f337f59e700 1 Processor -- start 2020-11-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.291+0000 7f337f59e700 1 -- start start 2020-11-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f33800b3830 0x7f3380143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f33800ac820 0x7f3380143f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3380060970 con 0x7f3380060130 2020-11-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338005c190 con 0x7f33800ac820 2020-11-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33800b4610 con 0x7f33800b3830 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f33800b3830 0x7f3380143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337ed9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3380060130 0x7f338005ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53926/0 (socket says 172.21.15.185:53926) 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337ed9d700 1 -- 172.21.15.185:0/1253100099 learned_addr learned my addr 172.21.15.185:0/1253100099 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f3375d9b700 1 --2- 172.21.15.185:0/1253100099 >> v2:172.21.15.96:3300/0 conn(0x7f33800ac820 0x7f3380143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 -- 172.21.15.185:0/1253100099 >> v2:172.21.15.96:3300/0 conn(0x7f33800ac820 msgr2=0x7f3380143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 --2- 172.21.15.185:0/1253100099 >> v2:172.21.15.96:3300/0 conn(0x7f33800ac820 0x7f3380143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 -- 172.21.15.185:0/1253100099 >> v1:172.21.15.185:6789/0 conn(0x7f3380060130 legacy=0x7f338005ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337dd9b700 1 -- 172.21.15.185:0/1253100099 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134504600 0 0) 0x7f3380060970 con 0x7f3380060130 2020-11-16T22:18:21.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 -- 172.21.15.185:0/1253100099 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380060e40 con 0x7f33800b3830 2020-11-16T22:18:21.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337e59c700 1 --2- 172.21.15.185:0/1253100099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f33800b3830 0x7f3380143a40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f33800b7d40 tx=0x7f3370006df0).ready entity=mon.1 client_cookie=207f660a960b832b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337dd9b700 1 -- 172.21.15.185:0/1253100099 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700097b0 con 0x7f33800b3830 2020-11-16T22:18:21.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337dd9b700 1 -- 172.21.15.185:0/1253100099 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370009910 con 0x7f33800b3830 2020-11-16T22:18:21.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337dd9b700 1 -- 172.21.15.185:0/1253100099 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370009c90 con 0x7f33800b3830 2020-11-16T22:18:21.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- 172.21.15.185:0/1253100099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f33800b3830 msgr2=0x7f3380143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 --2- 172.21.15.185:0/1253100099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f33800b3830 0x7f3380143a40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f33800b7d40 tx=0x7f3370006df0).stop 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- 172.21.15.185:0/1253100099 shutdown_connections 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 --2- 172.21.15.185:0/1253100099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f33800b3830 0x7f3380143a40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 --2- 172.21.15.185:0/1253100099 >> v2:172.21.15.96:3300/0 conn(0x7f33800ac820 0x7f3380143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- 172.21.15.185:0/1253100099 >> 172.21.15.185:0/1253100099 conn(0x7f3380063df0 msgr2=0x7f33800641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- 172.21.15.185:0/1253100099 shutdown_connections 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 -- 172.21.15.185:0/1253100099 wait complete. 2020-11-16T22:18:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.295+0000 7f337f59e700 1 Processor -- start 2020-11-16T22:18:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- start start 2020-11-16T22:18:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3380060130 0x7f338013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 0x7f338013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33800b0f30 con 0x7f33800ac820 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3380139130 con 0x7f3380060130 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33800b0590 con 0x7f33800aec20 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3380060130 0x7f338013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53938/0 (socket says 172.21.15.185:53938) 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 -- 172.21.15.185:0/3154757695 learned_addr learned my addr 172.21.15.185:0/3154757695 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3375d9b700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 0x7f338013c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 -- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3380060130 msgr2=0x7f338013bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3380060130 0x7f338013bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 -- 172.21.15.185:0/3154757695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 msgr2=0x7f338013c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 0x7f338013c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 -- 172.21.15.185:0/3154757695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800b1160 con 0x7f33800aec20 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3375d9b700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 0x7f338013c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f337800cab0 tx=0x7f3378007d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 -- 172.21.15.185:0/3154757695 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378005b80 con 0x7f33800aec20 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 -- 172.21.15.185:0/3154757695 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378005ce0 con 0x7f33800aec20 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3380139390 con 0x7f33800aec20 2020-11-16T22:18:21.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 -- 172.21.15.185:0/3154757695 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378007320 con 0x7f33800aec20 2020-11-16T22:18:21.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33800b0b50 con 0x7f33800aec20 2020-11-16T22:18:21.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 -- 172.21.15.185:0/3154757695 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3378010030 con 0x7f33800aec20 2020-11-16T22:18:21.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 0x7f335c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.299+0000 7f3376ffd700 1 -- 172.21.15.185:0/3154757695 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f337806dd80 con 0x7f33800aec20 2020-11-16T22:18:21.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.303+0000 7f337e59c700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 0x7f335c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.303+0000 7f337e59c700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 0x7f335c057520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3380061cd0 tx=0x7f3370015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.303+0000 7f339376c740 1 Processor -- start 2020-11-16T22:18:21.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.303+0000 7f339376c740 1 -- start start 2020-11-16T22:18:21.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2740bb0 0x29cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29ccda0 0x29cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x28584b0 con 0x29ccda0 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x290c930 con 0x27ae7b0 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x27cc6f0 con 0x2740bb0 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f3375d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2740bb0 0x29cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60498/0 (socket says 172.21.15.185:60498) 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 -- 172.21.15.185:0/3810362151 learned_addr learned my addr 172.21.15.185:0/3810362151 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29ccda0 0x29cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 -- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2740bb0 msgr2=0x29cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2740bb0 0x29cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 -- 172.21.15.185:0/3810362151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29ccda0 msgr2=0x29cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29ccda0 0x29cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 -- 172.21.15.185:0/3810362151 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27c75a0 con 0x27ae7b0 2020-11-16T22:18:21.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f337e59c700 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f338005c7e0 tx=0x7f337001f000).ready entity=mon.1 client_cookie=b52eec826299ec43 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f334f7fe700 1 -- 172.21.15.185:0/3810362151 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370006b10 con 0x27ae7b0 2020-11-16T22:18:21.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f334f7fe700 1 -- 172.21.15.185:0/3810362151 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370011c80 con 0x27ae7b0 2020-11-16T22:18:21.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f334f7fe700 1 -- 172.21.15.185:0/3810362151 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002d480 con 0x27ae7b0 2020-11-16T22:18:21.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 msgr2=0x29cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f338005c7e0 tx=0x7f337001f000).stop 2020-11-16T22:18:21.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- 172.21.15.185:0/3810362151 shutdown_connections 2020-11-16T22:18:21.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x27ae7b0 0x29cc8b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29ccda0 0x29cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 --2- 172.21.15.185:0/3810362151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2740bb0 0x29cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- 172.21.15.185:0/3810362151 >> 172.21.15.185:0/3810362151 conn(0x297ce40 msgr2=0x29c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- 172.21.15.185:0/3810362151 shutdown_connections 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- 172.21.15.185:0/3810362151 wait complete. 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 -- start start 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.307+0000 7f339376c740 1 Processor -- start 2020-11-16T22:18:21.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x2a14210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 0x2a14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x28f96e0 con 0x2740bb0 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x28f9820 con 0x29ccda0 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x28faa90 con 0x27ae7b0 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f3375d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x2a14210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 0x2a14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f3375d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x2a14210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53950/0 (socket says 172.21.15.185:53950) 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f3375d9b700 1 -- 172.21.15.185:0/3787503091 learned_addr learned my addr 172.21.15.185:0/3787503091 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 10 client.0 ll_register_callbacks cb 0x28efad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337e59c700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 -- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 msgr2=0x2a14210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x2a14210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 -- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 msgr2=0x29cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 -- 172.21.15.185:0/3787503091 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x28ea280 con 0x29ccda0 2020-11-16T22:18:21.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f337ed9d700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 0x2a14700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f3378001520 tx=0x7f3378011000).ready entity=mon.1 client_cookie=b81827f79371fe11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:21.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3378047190 con 0x29ccda0 2020-11-16T22:18:21.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f33780472f0 con 0x29ccda0 2020-11-16T22:18:21.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x29ad6c0 con 0x29ccda0 2020-11-16T22:18:21.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3378011570 con 0x29ccda0 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 10 client.6553 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x271ffd0 con 0x29ccda0 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x29ad6c0 con 0x29ccda0 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3378043a90 con 0x29ccda0 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 10 client.6553 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 10 client.6553 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:21.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 20 client.6553 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2a10620 0x29fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 4 client.6553 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.311+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x27eac50 con 0x2a10620 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f337e59c700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2a10620 0x29fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f33557fa700 10 client.6553 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f337e59c700 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2a10620 0x29fb130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e8bd8b21c5001d0d server_cookie=27b7585dcd380f90 in_seq=0 out_seq=0 2020-11-16T22:18:21.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f33557fa700 1 -- 172.21.15.185:0/3787503091 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x27eac50 con 0x2a10620 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f33557fa700 10 client.6553 handle_command_reply: tid=0 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 client.6553 shutdown 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 msgr2=0x2a14700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 0x2a14700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f3378001520 tx=0x7f3378011000).stop 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 shutdown_connections 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29ccda0 0x2a14700 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x2a10620 0x29fb130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f33557fa700 0 client.6553 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 --2- 172.21.15.185:0/3787503091 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x2a14210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 >> 172.21.15.185:0/3787503091 conn(0x297ce40 msgr2=0x29fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 shutdown_connections 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 1 -- 172.21.15.185:0/3787503091 wait complete. 2020-11-16T22:18:21.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.315+0000 7f339376c740 20 client.6553 trim_cache size 0 max 16384 2020-11-16T22:18:21.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 Processor -- start 2020-11-16T22:18:21.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- start start 2020-11-16T22:18:21.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x29cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x28d1e40 con 0x2740bb0 2020-11-16T22:18:21.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x29ca8f0 con 0x297ce40 2020-11-16T22:18:21.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a02610 con 0x27ae7b0 2020-11-16T22:18:21.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x29cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60512/0 (socket says 172.21.15.185:60512) 2020-11-16T22:18:21.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 -- 172.21.15.185:0/2839994280 learned_addr learned my addr 172.21.15.185:0/2839994280 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f3375d9b700 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 -- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 msgr2=0x29cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x29cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 -- 172.21.15.185:0/2839994280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 msgr2=0x29c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 -- 172.21.15.185:0/2839994280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480081d0 con 0x297ce40 2020-11-16T22:18:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f3375d9b700 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f337e59c700 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f337000c010 tx=0x7f3370013000).ready entity=mon.1 client_cookie=f673d99285d6aac8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f3357fff700 1 -- 172.21.15.185:0/2839994280 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000b620 con 0x297ce40 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f3357fff700 1 -- 172.21.15.185:0/2839994280 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370032040 con 0x297ce40 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f3357fff700 1 -- 172.21.15.185:0/2839994280 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002d870 con 0x297ce40 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 msgr2=0x29c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f337000c010 tx=0x7f3370013000).stop 2020-11-16T22:18:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- 172.21.15.185:0/2839994280 shutdown_connections 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x297ce40 0x29c8540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x27ae7b0 0x29cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 --2- 172.21.15.185:0/2839994280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2740bb0 0x29c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- 172.21.15.185:0/2839994280 >> 172.21.15.185:0/2839994280 conn(0x7f3348002320 msgr2=0x29fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- 172.21.15.185:0/2839994280 shutdown_connections 2020-11-16T22:18:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.335+0000 7f339376c740 1 -- 172.21.15.185:0/2839994280 wait complete. 2020-11-16T22:18:21.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- start start 2020-11-16T22:18:21.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 Processor -- start 2020-11-16T22:18:21.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x27ae7b0 0x29fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 0x2a62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29cc840 con 0x27ae7b0 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x29ce190 con 0x2740bb0 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29fb2b0 con 0x297ce40 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 10 client.0 ll_register_callbacks cb 0x29cf400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60518/0 (socket says 172.21.15.185:60518) 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 -- 172.21.15.185:0/2991395126 learned_addr learned my addr 172.21.15.185:0/2991395126 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 0x2a62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f3375d9b700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x27ae7b0 0x29fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 -- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 msgr2=0x2a62b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 0x2a62b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 -- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x27ae7b0 msgr2=0x29fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:21.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x27ae7b0 0x29fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 -- 172.21.15.185:0/2991395126 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x275e720 con 0x2740bb0 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337ed9d700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 0x2a62b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f337e59c700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f33700098e0 tx=0x7f337003c000).ready entity=mon.1 client_cookie=2153f9051ab42fee server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002d480 con 0x2740bb0 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3370026030 con 0x2740bb0 2020-11-16T22:18:21.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2a01b70 con 0x2740bb0 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f337002d5e0 con 0x2740bb0 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 10 client.6559 fetch_fsmap learned FSMap version 58 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x2769e00 con 0x2740bb0 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2a01b70 con 0x2740bb0 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 58) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3370040070 con 0x2740bb0 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 10 client.6559 fetch_fsmap finished waiting for FSMap version 58 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 10 client.6559 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:21.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 20 client.6559 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x29ccda0 0x29c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 4 client.6559 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x7f3370011c80 con 0x29ccda0 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.339+0000 7f3375d9b700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x29ccda0 0x29c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f33557fa700 10 client.6559 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f3375d9b700 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x29ccda0 0x29c9960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9e23df9a11ce269 server_cookie=5485c18ce0b6770e in_seq=0 out_seq=0 2020-11-16T22:18:21.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f33557fa700 1 -- 172.21.15.185:0/2991395126 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7f3370011c80 con 0x29ccda0 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f33557fa700 10 client.6559 handle_command_reply: tid=0 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 client.6559 shutdown 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 msgr2=0x2a031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f33700098e0 tx=0x7f337003c000).stop 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 shutdown_connections 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2740bb0 0x2a031e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x29ccda0 0x29c9960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x297ce40 0x2a62b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f33557fa700 0 client.6559 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 --2- 172.21.15.185:0/2991395126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x27ae7b0 0x29fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 >> 172.21.15.185:0/2991395126 conn(0x7f3348002320 msgr2=0x29c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 shutdown_connections 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 1 -- 172.21.15.185:0/2991395126 wait complete. 2020-11-16T22:18:21.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f339376c740 20 client.6559 trim_cache size 0 max 16384 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 msgr2=0x7f335c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 0x7f335c057520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f3380061cd0 tx=0x7f3370015000).stop 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 msgr2=0x7f33800af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f337800cab0 tx=0x7f3378007d00).stop 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 shutdown_connections 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f335c055060 0x7f335c057520 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3380060130 0x7f338013bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f33800aec20 0x7f33800af0a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 --2- 172.21.15.185:0/3154757695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33800ac820 0x7f338013c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 >> 172.21.15.185:0/3154757695 conn(0x7f3380063df0 msgr2=0x7f338013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 shutdown_connections 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:21.343+0000 7f337f59e700 1 -- 172.21.15.185:0/3154757695 wait complete. 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:21.335 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: "60107533-a7d2-43e9-b4cd-7eaa98fbbc4b": { 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "60107533-a7d2-43e9-b4cd-7eaa98fbbc4b", 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:21.336 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:21.337 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:21.343 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:18:23.172 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:18:23.194+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:18:26.344 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:26.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 Processor -- start 2020-11-16T22:18:26.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- start start 2020-11-16T22:18:26.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 0x7ff248067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff248098cc0 con 0x7ff248066240 2020-11-16T22:18:26.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2480b3950 con 0x7ff2480b2160 2020-11-16T22:18:26.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24814c4f0 con 0x7ff2480b17e0 2020-11-16T22:18:26.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d810700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff2480b2160 0x7ff24814bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53966/0 (socket says 172.21.15.185:53966) 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d810700 1 -- 172.21.15.185:0/1517660631 learned_addr learned my addr 172.21.15.185:0/1517660631 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24e011700 1 --2- 172.21.15.185:0/1517660631 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 0x7ff248067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d00f700 1 -- 172.21.15.185:0/1517660631 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378220933 0 0) 0x7ff2480b3950 con 0x7ff2480b2160 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d00f700 1 -- 172.21.15.185:0/1517660631 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff230003240 con 0x7ff2480b2160 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24e011700 1 -- 172.21.15.185:0/1517660631 >> v1:172.21.15.185:6789/0 conn(0x7ff2480b2160 legacy=0x7ff24814bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24e011700 1 -- 172.21.15.185:0/1517660631 >> v1:172.21.15.96:6789/0 conn(0x7ff248066240 legacy=0x7ff248065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24e011700 1 -- 172.21.15.185:0/1517660631 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480b3bf0 con 0x7ff2480b17e0 2020-11-16T22:18:26.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24e011700 1 --2- 172.21.15.185:0/1517660631 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 0x7ff248067fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff23c00b9d0 tx=0x7ff23c006df0).ready entity=mon.2 client_cookie=5eb1503d1b521202 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d00f700 1 -- 172.21.15.185:0/1517660631 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0097b0 con 0x7ff2480b17e0 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d00f700 1 -- 172.21.15.185:0/1517660631 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c009910 con 0x7ff2480b17e0 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff24d00f700 1 -- 172.21.15.185:0/1517660631 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c009c90 con 0x7ff2480b17e0 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- 172.21.15.185:0/1517660631 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 msgr2=0x7ff248067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 --2- 172.21.15.185:0/1517660631 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 0x7ff248067fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7ff23c00b9d0 tx=0x7ff23c006df0).stop 2020-11-16T22:18:26.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- 172.21.15.185:0/1517660631 shutdown_connections 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 --2- 172.21.15.185:0/1517660631 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b17e0 0x7ff248067fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.491+0000 7ff250275700 1 -- 172.21.15.185:0/1517660631 >> 172.21.15.185:0/1517660631 conn(0x7ff2480604a0 msgr2=0x7ff2480608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- 172.21.15.185:0/1517660631 shutdown_connections 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- 172.21.15.185:0/1517660631 wait complete. 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 Processor -- start 2020-11-16T22:18:26.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- start start 2020-11-16T22:18:26.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 0x7ff248095410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2480adb00 0x7ff2480adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480ae470 con 0x7ff248066240 2020-11-16T22:18:26.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480af870 con 0x7ff2480adb00 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480afb10 con 0x7ff2480b2160 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53978/0 (socket says 172.21.15.185:53978) 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 -- 172.21.15.185:0/2677060095 learned_addr learned my addr 172.21.15.185:0/2677060095 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24e812700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2480adb00 0x7ff2480adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24e011700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 0x7ff248095410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 -- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2480adb00 msgr2=0x7ff2480adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2480adb00 0x7ff2480adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 -- 172.21.15.185:0/2677060095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 msgr2=0x7ff248095410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 0x7ff248095410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 -- 172.21.15.185:0/2677060095 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480afdb0 con 0x7ff2480b2160 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24e011700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 0x7ff248095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff24d810700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff24400c710 tx=0x7ff244007a40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440072e0 con 0x7ff2480b2160 2020-11-16T22:18:26.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244007440 con 0x7ff2480b2160 2020-11-16T22:18:26.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24808fd30 con 0x7ff2480b2160 2020-11-16T22:18:26.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244005820 con 0x7ff2480b2160 2020-11-16T22:18:26.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.495+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24814c230 con 0x7ff2480b2160 2020-11-16T22:18:26.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2480aff40 con 0x7ff2480b2160 2020-11-16T22:18:26.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff244010070 con 0x7ff2480b2160 2020-11-16T22:18:26.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff23a7fc700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 0x7ff220057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff24e011700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 0x7ff220057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff24406db10 con 0x7ff2480b2160 2020-11-16T22:18:26.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.499+0000 7ff24e011700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 0x7ff220057520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff248064fa0 tx=0x7ff23c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.503+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff244043c80 con 0x7ff2480b2160 2020-11-16T22:18:26.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.707+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff24805d4e0 con 0x7ff2480b2160 2020-11-16T22:18:26.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.707+0000 7ff23a7fc700 1 -- 172.21.15.185:0/2677060095 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7ff2440472a0 con 0x7ff2480b2160 2020-11-16T22:18:26.690 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:26.690 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:26.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 msgr2=0x7ff220057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 0x7ff220057520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7ff248064fa0 tx=0x7ff23c015000).stop 2020-11-16T22:18:26.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 msgr2=0x7ff248095900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff24400c710 tx=0x7ff244007a40).stop 2020-11-16T22:18:26.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 shutdown_connections 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff220055060 0x7ff220057520 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff2480adb00 0x7ff2480adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff248066240 0x7ff248095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 --2- 172.21.15.185:0/2677060095 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff2480b2160 0x7ff248095900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 >> 172.21.15.185:0/2677060095 conn(0x7ff2480604a0 msgr2=0x7ff2480b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 shutdown_connections 2020-11-16T22:18:26.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.711+0000 7ff250275700 1 -- 172.21.15.185:0/2677060095 wait complete. 2020-11-16T22:18:26.694 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:26.705 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:18:26.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 Processor -- start 2020-11-16T22:18:26.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- start start 2020-11-16T22:18:26.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a601446e0 con 0x7f2a600b5c40 2020-11-16T22:18:26.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a60144980 con 0x7f2a600ba800 2020-11-16T22:18:26.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601444f0 con 0x7f2a600b6020 2020-11-16T22:18:26.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53986/0 (socket says 172.21.15.185:53986) 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 -- 172.21.15.185:0/185513906 learned_addr learned my addr 172.21.15.185:0/185513906 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a650b2700 1 --1- 172.21.15.185:0/185513906 >> v1:172.21.15.185:6789/0 conn(0x7f2a600ba800 0x7f2a600b1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:53982/0 (socket says 172.21.15.185:53982) 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 -- 172.21.15.185:0/185513906 >> v1:172.21.15.185:6789/0 conn(0x7f2a600ba800 legacy=0x7f2a600b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a648b1700 1 -- 172.21.15.185:0/185513906 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863838635 0 0) 0x7f2a60144980 con 0x7f2a600ba800 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 -- 172.21.15.185:0/185513906 >> v1:172.21.15.96:6789/0 conn(0x7f2a600b5c40 legacy=0x7f2a6013dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 -- 172.21.15.185:0/185513906 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60144c20 con 0x7f2a600b6020 2020-11-16T22:18:26.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a660b4700 1 --2- 172.21.15.185:0/185513906 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f2a58009d80 tx=0x7f2a58009290).ready entity=mon.2 client_cookie=c5f73804bb23da5a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a648b1700 1 -- 172.21.15.185:0/185513906 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800bc80 con 0x7f2a600b6020 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a648b1700 1 -- 172.21.15.185:0/185513906 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5800bde0 con 0x7f2a600b6020 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a648b1700 1 -- 172.21.15.185:0/185513906 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58011610 con 0x7f2a600b6020 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- 172.21.15.185:0/185513906 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 msgr2=0x7f2a60144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 --2- 172.21.15.185:0/185513906 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f2a58009d80 tx=0x7f2a58009290).stop 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- 172.21.15.185:0/185513906 shutdown_connections 2020-11-16T22:18:26.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 --2- 172.21.15.185:0/185513906 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600b6020 0x7f2a60144000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- 172.21.15.185:0/185513906 >> 172.21.15.185:0/185513906 conn(0x7f2a6005ba80 msgr2=0x7f2a6005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:26.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- 172.21.15.185:0/185513906 shutdown_connections 2020-11-16T22:18:26.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.847+0000 7f2a668b5700 1 -- 172.21.15.185:0/185513906 wait complete. 2020-11-16T22:18:26.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 Processor -- start 2020-11-16T22:18:26.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- start start 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a600b5c40 0x7f2a60143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 0x7f2a60156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013e670 con 0x7f2a600b5c40 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013e8a0 con 0x7f2a600b6020 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6013eb40 con 0x7f2a600ba800 2020-11-16T22:18:26.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 0x7f2a60156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:53994/0 (socket says 172.21.15.185:53994) 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a660b4700 1 -- 172.21.15.185:0/3741817436 learned_addr learned my addr 172.21.15.185:0/3741817436 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a658b3700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a600b5c40 0x7f2a60143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 -- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 msgr2=0x7f2a601571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 -- 172.21.15.185:0/3741817436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a600b5c40 msgr2=0x7f2a60143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a600b5c40 0x7f2a60143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 -- 172.21.15.185:0/3741817436 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6013ede0 con 0x7f2a600b6020 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a660b4700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a650b2700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 0x7f2a60156cb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f2a54000bc0 tx=0x7f2a540079a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a5dffb700 1 -- 172.21.15.185:0/3741817436 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540072e0 con 0x7f2a600b6020 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a5dffb700 1 -- 172.21.15.185:0/3741817436 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54005530 con 0x7f2a600b6020 2020-11-16T22:18:26.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a5dffb700 1 -- 172.21.15.185:0/3741817436 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540058f0 con 0x7f2a600b6020 2020-11-16T22:18:26.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a60156810 con 0x7f2a600b6020 2020-11-16T22:18:26.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.851+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a60144070 con 0x7f2a600b6020 2020-11-16T22:18:26.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.855+0000 7f2a5dffb700 1 -- 172.21.15.185:0/3741817436 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2a54010070 con 0x7f2a600b6020 2020-11-16T22:18:26.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.855+0000 7f2a5dffb700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 0x7f2a38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.855+0000 7f2a658b3700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 0x7f2a38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.855+0000 7f2a5dffb700 1 -- 172.21.15.185:0/3741817436 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a5406d310 con 0x7f2a600b6020 2020-11-16T22:18:26.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.855+0000 7f2a658b3700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 0x7f2a38057520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f2a50003300 tx=0x7f2a50007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 Processor -- start 2020-11-16T22:18:26.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 -- start start 2020-11-16T22:18:26.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ecfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1ed0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d5d4b0 con 0x1c45bb0 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e11930 con 0x1cb37b0 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cd16f0 con 0x1ed1da0 2020-11-16T22:18:26.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60554/0 (socket says 172.21.15.185:60554) 2020-11-16T22:18:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1ed0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 -- 172.21.15.185:0/479573888 learned_addr learned my addr 172.21.15.185:0/479573888 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a650b2700 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ecfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 -- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 msgr2=0x1ed0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1ed0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 -- 172.21.15.185:0/479573888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 msgr2=0x1ecfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ecfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a658b3700 1 -- 172.21.15.185:0/479573888 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ccc5a0 con 0x1cb37b0 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.859+0000 7f2a650b2700 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ecfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f2a50015b60 tx=0x7f2a5000a000).ready entity=mon.1 client_cookie=276c13ffb37a3217 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a3effd700 1 -- 172.21.15.185:0/479573888 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50015350 con 0x1cb37b0 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a3effd700 1 -- 172.21.15.185:0/479573888 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50017860 con 0x1cb37b0 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a3effd700 1 -- 172.21.15.185:0/479573888 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50017be0 con 0x1cb37b0 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 msgr2=0x1ed18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f2a50015b60 tx=0x7f2a5000a000).stop 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/479573888 shutdown_connections 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1ed18b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1ed0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- 172.21.15.185:0/479573888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ecfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/479573888 >> 172.21.15.185:0/479573888 conn(0x1e81e40 msgr2=0x1ecedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/479573888 shutdown_connections 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/479573888 wait complete. 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- start start 2020-11-16T22:18:26.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 Processor -- start 2020-11-16T22:18:26.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cb37b0 0x1f19700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1effba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dfe6e0 con 0x1cb37b0 2020-11-16T22:18:26.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dfe820 con 0x1c45bb0 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dffa90 con 0x1ed1da0 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 10 client.0 ll_register_callbacks cb 0x1df4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1effba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60560/0 (socket says 172.21.15.185:60560) 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1effba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54008/0 (socket says 172.21.15.185:54008) 2020-11-16T22:18:26.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 -- 172.21.15.185:0/1342033070 learned_addr learned my addr 172.21.15.185:0/1342033070 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a650b2700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cb37b0 0x1f19700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 -- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 msgr2=0x1effba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1effba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 -- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cb37b0 msgr2=0x1f19700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cb37b0 0x1f19700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 -- 172.21.15.185:0/1342033070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1def280 con 0x1c45bb0 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a658b3700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f2a50015810 tx=0x7f2a50012000).ready entity=mon.1 client_cookie=641cce33add9a452 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:26.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a33fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50026070 con 0x1c45bb0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1eb26c0 con 0x1c45bb0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.863+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a50020610 con 0x1c45bb0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a50017c20 con 0x1c45bb0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 10 client.6577 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1c24fd0 con 0x1c45bb0 2020-11-16T22:18:26.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1eb26c0 con 0x1c45bb0 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f2a5001e020 con 0x1c45bb0 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 10 client.6577 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 10 client.6577 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 20 client.6577 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ecd640 0x1f05950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 4 client.6577 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:26.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1cefc50 con 0x1ecd640 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a650b2700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ecd640 0x1f05950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a650b2700 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ecd640 0x1f05950 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7701eba362248a14 server_cookie=c083c0ba11b839ba in_seq=0 out_seq=0 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 10 client.6577 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 1 -- 172.21.15.185:0/1342033070 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1cefc50 con 0x1ecd640 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 10 client.6577 handle_command_reply: tid=0 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 client.6577 shutdown 2020-11-16T22:18:26.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 msgr2=0x1f19210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:26.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f2a50015810 tx=0x7f2a50012000).stop 2020-11-16T22:18:26.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 shutdown_connections 2020-11-16T22:18:26.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1c45bb0 0x1f19210 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ecd640 0x1f05950 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ed1da0 0x1effba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 --2- 172.21.15.185:0/1342033070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cb37b0 0x1f19700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a33fff700 0 client.6577 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:27.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 >> 172.21.15.185:0/1342033070 conn(0x1e81e40 msgr2=0x1ecfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:27.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 shutdown_connections 2020-11-16T22:18:27.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 1 -- 172.21.15.185:0/1342033070 wait complete. 2020-11-16T22:18:27.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.867+0000 7f2a765a8740 20 client.6577 trim_cache size 0 max 16384 2020-11-16T22:18:27.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.883+0000 7f2a765a8740 1 Processor -- start 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.883+0000 7f2a765a8740 1 -- start start 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ed4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd6e40 con 0x1c45bb0 2020-11-16T22:18:27.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ed43f0 con 0x1cb37b0 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e2e680 con 0x1e81e40 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54014/0 (socket says 172.21.15.185:54014) 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 -- 172.21.15.185:0/2119402337 learned_addr learned my addr 172.21.15.185:0/2119402337 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a650b2700 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ed4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 -- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 msgr2=0x1f05140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f05140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 -- 172.21.15.185:0/2119402337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 msgr2=0x1ed4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ed4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 -- 172.21.15.185:0/2119402337 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c008160 con 0x1e81e40 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a658b3700 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2a50000ee0 tx=0x7f2a50026000).ready entity=mon.2 client_cookie=3618fd78eceeeac4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a3f7fe700 1 -- 172.21.15.185:0/2119402337 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5003a030 con 0x1e81e40 2020-11-16T22:18:27.067 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a3f7fe700 1 -- 172.21.15.185:0/2119402337 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50015120 con 0x1e81e40 2020-11-16T22:18:27.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a3f7fe700 1 -- 172.21.15.185:0/2119402337 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50017c00 con 0x1e81e40 2020-11-16T22:18:27.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 msgr2=0x1ed48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:27.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2a50000ee0 tx=0x7f2a50026000).stop 2020-11-16T22:18:27.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- 172.21.15.185:0/2119402337 shutdown_connections 2020-11-16T22:18:27.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f05140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1ed48c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- 172.21.15.185:0/2119402337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1ed4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- 172.21.15.185:0/2119402337 >> 172.21.15.185:0/2119402337 conn(0x7f2a2c002320 msgr2=0x1ec66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- 172.21.15.185:0/2119402337 shutdown_connections 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- 172.21.15.185:0/2119402337 wait complete. 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- start start 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 Processor -- start 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1f67430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f0dbe0 con 0x1c45bb0 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f073d0 con 0x1cb37b0 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f6c240 con 0x1e81e40 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.887+0000 7f2a765a8740 10 client.0 ll_register_callbacks cb 0x1f04b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a650b2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54018/0 (socket says 172.21.15.185:54018) 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 -- 172.21.15.185:0/806834441 learned_addr learned my addr 172.21.15.185:0/806834441 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a658b3700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1f67430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 -- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 msgr2=0x1f67920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f67920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 -- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 msgr2=0x1f67430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1f67430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 -- 172.21.15.185:0/806834441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c63720 con 0x1e81e40 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a658b3700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1f67430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a660b4700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2a5800b7b0 tx=0x7f2a58016000).ready entity=mon.2 client_cookie=a2999bc49b1f4847 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58022070 con 0x1e81e40 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a58011e30 con 0x1e81e40 2020-11-16T22:18:27.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a50017860 con 0x1e81e40 2020-11-16T22:18:27.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a58011610 con 0x1e81e40 2020-11-16T22:18:27.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 10 client.5993 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1c6ee00 con 0x1e81e40 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50017860 con 0x1e81e40 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f2a58027070 con 0x1e81e40 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 10 client.5993 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 10 client.5993 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 20 client.5993 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:27.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ed1da0 0x1f68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 4 client.5993 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1cf6f50 con 0x1ed1da0 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a658b3700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ed1da0 0x1f68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a33fff700 10 client.5993 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.891+0000 7f2a658b3700 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ed1da0 0x1f68840 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=80933348fdcd7d88 server_cookie=f8d02ecc7423fdf8 in_seq=0 out_seq=0 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a33fff700 1 -- 172.21.15.185:0/806834441 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1cf6f50 con 0x1ed1da0 2020-11-16T22:18:27.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a33fff700 10 client.5993 handle_command_reply: tid=0 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 client.5993 shutdown 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 msgr2=0x1f68350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2a5800b7b0 tx=0x7f2a58016000).stop 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 shutdown_connections 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x1ed1da0 0x1f68840 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1e81e40 0x1f68350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1c45bb0 0x1f67430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a33fff700 0 client.5993 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:27.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 --2- 172.21.15.185:0/806834441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cb37b0 0x1f67920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 >> 172.21.15.185:0/806834441 conn(0x7f2a2c002320 msgr2=0x1f042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:27.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 shutdown_connections 2020-11-16T22:18:27.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 1 -- 172.21.15.185:0/806834441 wait complete. 2020-11-16T22:18:27.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a765a8740 20 client.5993 trim_cache size 0 max 16384 2020-11-16T22:18:27.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 msgr2=0x7f2a38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:27.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 0x7f2a38057520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f2a50003300 tx=0x7f2a50007b90).stop 2020-11-16T22:18:27.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 msgr2=0x7f2a60156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:27.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 0x7f2a60156cb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f2a54000bc0 tx=0x7f2a540079a0).stop 2020-11-16T22:18:27.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 shutdown_connections 2020-11-16T22:18:27.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a38055060 0x7f2a38057520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2a600ba800 0x7f2a601571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a600b5c40 0x7f2a60143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 --2- 172.21.15.185:0/3741817436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2a600b6020 0x7f2a60156cb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:27.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 >> 172.21.15.185:0/3741817436 conn(0x7f2a6005ba80 msgr2=0x7f2a600b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:27.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 shutdown_connections 2020-11-16T22:18:27.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:26.895+0000 7f2a668b5700 1 -- 172.21.15.185:0/3741817436 wait complete. 2020-11-16T22:18:27.486 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:27.487 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "IDLE (waiting for more scrubs)", 2020-11-16T22:18:27.487 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:18:27.487 INFO:teuthology.orchestra.run.smithi185.stdout: "60107533-a7d2-43e9-b4cd-7eaa98fbbc4b": { 2020-11-16T22:18:27.488 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:18:27.488 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "60107533-a7d2-43e9-b4cd-7eaa98fbbc4b", 2020-11-16T22:18:27.488 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:18:27.488 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:27.489 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:27.489 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:27.489 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2020-11-16T22:18:31.974 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:18:31.998+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:18:32.491 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:32.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 Processor -- start 2020-11-16T22:18:32.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 -- start start 2020-11-16T22:18:32.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91d80b0300 0x7f91d80b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d8062700 con 0x7f91d8061640 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80629a0 con 0x7f91d80b0300 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91dcec7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8062c40 con 0x7f91d8061cf0 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54030/0 (socket says 172.21.15.185:54030) 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 -- 172.21.15.185:0/617020573 learned_addr learned my addr 172.21.15.185:0/617020573 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:32.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d659c700 1 --2- 172.21.15.185:0/617020573 >> v2:172.21.15.96:3300/0 conn(0x7f91d80b0300 0x7f91d80b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 -- 172.21.15.185:0/617020573 >> v2:172.21.15.96:3300/0 conn(0x7f91d80b0300 msgr2=0x7f91d80b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 --2- 172.21.15.185:0/617020573 >> v2:172.21.15.96:3300/0 conn(0x7f91d80b0300 0x7f91d80b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 -- 172.21.15.185:0/617020573 >> v1:172.21.15.96:6789/0 conn(0x7f91d8061640 legacy=0x7f91d80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d559a700 1 -- 172.21.15.185:0/617020573 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142650726 0 0) 0x7f91d8062700 con 0x7f91d8061640 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 -- 172.21.15.185:0/617020573 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8062ee0 con 0x7f91d8061cf0 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.639+0000 7f91d6d9d700 1 --2- 172.21.15.185:0/617020573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f91d0009d50 tx=0x7f91d0009290).ready entity=mon.2 client_cookie=88dcb1a77eb668d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91d559a700 1 -- 172.21.15.185:0/617020573 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000bbc0 con 0x7f91d8061cf0 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91d559a700 1 -- 172.21.15.185:0/617020573 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000bd20 con 0x7f91d8061cf0 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91d559a700 1 -- 172.21.15.185:0/617020573 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0011610 con 0x7f91d8061cf0 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 -- 172.21.15.185:0/617020573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 msgr2=0x7f91d81475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 --2- 172.21.15.185:0/617020573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f91d0009d50 tx=0x7f91d0009290).stop 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 -- 172.21.15.185:0/617020573 shutdown_connections 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 --2- 172.21.15.185:0/617020573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d81475f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 --2- 172.21.15.185:0/617020573 >> v2:172.21.15.96:3300/0 conn(0x7f91d80b0300 0x7f91d80b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 -- 172.21.15.185:0/617020573 >> 172.21.15.185:0/617020573 conn(0x7f91d8060130 msgr2=0x7f91d8060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:32.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 -- 172.21.15.185:0/617020573 shutdown_connections 2020-11-16T22:18:32.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.643+0000 7f91dcec7700 1 -- 172.21.15.185:0/617020573 wait complete. 2020-11-16T22:18:32.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 Processor -- start 2020-11-16T22:18:32.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- start start 2020-11-16T22:18:32.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8061640 0x7f91d81445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 0x7f91d8152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8143eb0 con 0x7f91d8061640 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d813fd10 con 0x7f91d80b0300 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81440e0 con 0x7f91d8061cf0 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 0x7f91d8152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54034/0 (socket says 172.21.15.185:54034) 2020-11-16T22:18:32.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d659c700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8061640 0x7f91d81445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d5d9b700 1 -- 172.21.15.185:0/2902148863 learned_addr learned my addr 172.21.15.185:0/2902148863 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:32.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 -- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 msgr2=0x7f91d8144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 -- 172.21.15.185:0/2902148863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8061640 msgr2=0x7f91d81445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8061640 0x7f91d81445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 -- 172.21.15.185:0/2902148863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d813ff70 con 0x7f91d80b0300 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d5d9b700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91d6d9d700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 0x7f91d8152fd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f91d0009d50 tx=0x7f91d0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000be90 con 0x7f91d80b0300 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d0011e30 con 0x7f91d80b0300 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0010d30 con 0x7f91d80b0300 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8140210 con 0x7f91d80b0300 2020-11-16T22:18:32.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.647+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d81406b0 con 0x7f91d80b0300 2020-11-16T22:18:32.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d805d4e0 con 0x7f91d80b0300 2020-11-16T22:18:32.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f91d0017070 con 0x7f91d80b0300 2020-11-16T22:18:32.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91c67fc700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 0x7f91ac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91d659c700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 0x7f91ac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f91d006fed0 con 0x7f91d80b0300 2020-11-16T22:18:32.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.651+0000 7f91d659c700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 0x7f91ac057520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f91cc006fd0 tx=0x7f91cc006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.655+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f91d00460a0 con 0x7f91d80b0300 2020-11-16T22:18:32.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.855+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91d8063850 con 0x7f91d80b0300 2020-11-16T22:18:32.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91c67fc700 1 -- 172.21.15.185:0/2902148863 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f91d0015070 con 0x7f91d80b0300 2020-11-16T22:18:32.840 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:32.840 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 msgr2=0x7f91ac057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 0x7f91ac057520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f91cc006fd0 tx=0x7f91cc006df0).stop 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 msgr2=0x7f91d8152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 0x7f91d8152fd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f91d0009d50 tx=0x7f91d0016000).stop 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 shutdown_connections 2020-11-16T22:18:32.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91ac055060 0x7f91ac057520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91d80b0300 0x7f91d8152fd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8061640 0x7f91d81445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 --2- 172.21.15.185:0/2902148863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91d8061cf0 0x7f91d8144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 >> 172.21.15.185:0/2902148863 conn(0x7f91d8060130 msgr2=0x7f91d80b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 shutdown_connections 2020-11-16T22:18:32.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.859+0000 7f91dcec7700 1 -- 172.21.15.185:0/2902148863 wait complete. 2020-11-16T22:18:32.844 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:32.854 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:18:32.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.999+0000 7f5e46630700 1 Processor -- start 2020-11-16T22:18:32.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:32.999+0000 7f5e46630700 1 -- start start 2020-11-16T22:18:32.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5e400b5d70 0x7f5e400ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 0x7f5e40143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e400b7290 con 0x7f5e400b4470 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400b4bf0 con 0x7f5e40143470 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400b74c0 con 0x7f5e400b5d70 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5e400b5d70 0x7f5e400ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5e400b5d70 0x7f5e400ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60598/0 (socket says 172.21.15.185:60598) 2020-11-16T22:18:32.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e44e2d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5e400b4470 0x7f5e400b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54034/0 (socket says 172.21.15.185:54034) 2020-11-16T22:18:32.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 -- 172.21.15.185:0/1439777029 learned_addr learned my addr 172.21.15.185:0/1439777029 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:32.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/1439777029 >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 0x7f5e40143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 -- 172.21.15.185:0/1439777029 >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 msgr2=0x7f5e40143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:32.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1439777029 >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 0x7f5e40143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 -- 172.21.15.185:0/1439777029 >> v1:172.21.15.185:6789/0 conn(0x7f5e400b4470 legacy=0x7f5e400b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e37fff700 1 -- 172.21.15.185:0/1439777029 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126184191 0 0) 0x7f5e400b7290 con 0x7f5e400b4470 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 -- 172.21.15.185:0/1439777029 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400b7760 con 0x7f5e400b5d70 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/1439777029 >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 0x7f5e40143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1439777029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e400ba800 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5e4005c5f0 tx=0x7f5e30006eb0).ready entity=mon.1 client_cookie=6fc67638dddf1260 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e37fff700 1 -- 172.21.15.185:0/1439777029 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e300098d0 con 0x7f5e400b5d70 2020-11-16T22:18:32.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e37fff700 1 -- 172.21.15.185:0/1439777029 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e30009a30 con 0x7f5e400b5d70 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e37fff700 1 -- 172.21.15.185:0/1439777029 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30009db0 con 0x7f5e400b5d70 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- 172.21.15.185:0/1439777029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 msgr2=0x7f5e400ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 --2- 172.21.15.185:0/1439777029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e400ba800 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5e4005c5f0 tx=0x7f5e30006eb0).stop 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- 172.21.15.185:0/1439777029 shutdown_connections 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 --2- 172.21.15.185:0/1439777029 >> v2:172.21.15.96:3300/0 conn(0x7f5e40143470 0x7f5e40143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 --2- 172.21.15.185:0/1439777029 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e400ba800 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- 172.21.15.185:0/1439777029 >> 172.21.15.185:0/1439777029 conn(0x7f5e4005ba80 msgr2=0x7f5e4005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:32.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- 172.21.15.185:0/1439777029 shutdown_connections 2020-11-16T22:18:32.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- 172.21.15.185:0/1439777029 wait complete. 2020-11-16T22:18:32.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 Processor -- start 2020-11-16T22:18:32.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.003+0000 7f5e46630700 1 -- start start 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e400b4470 0x7f5e400b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 0x7f5e40142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400b1740 con 0x7f5e40143470 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400af250 con 0x7f5e400b5d70 2020-11-16T22:18:32.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400af4f0 con 0x7f5e400b4470 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e4562e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e400b4470 0x7f5e400b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60604/0 (socket says 172.21.15.185:60604) 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1330874500 learned_addr learned my addr 172.21.15.185:0/1330874500 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 0x7f5e40142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e400b4470 msgr2=0x7f5e400b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e400b4470 0x7f5e400b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1330874500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 msgr2=0x7f5e40142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 0x7f5e40142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1330874500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400af790 con 0x7f5e400b5d70 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 0x7f5e40142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f5e3c00c730 tx=0x7f5e3c007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 -- 172.21.15.185:0/1330874500 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0072e0 con 0x7f5e400b5d70 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 -- 172.21.15.185:0/1330874500 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c005530 con 0x7f5e400b5d70 2020-11-16T22:18:32.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 -- 172.21.15.185:0/1330874500 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0058b0 con 0x7f5e400b5d70 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e400afa30 con 0x7f5e400b5d70 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e400ae4a0 con 0x7f5e400b5d70 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 -- 172.21.15.185:0/1330874500 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5e3c010070 con 0x7f5e400b5d70 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 0x7f5e18057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e357fa700 1 -- 172.21.15.185:0/1330874500 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e3c06d4d0 con 0x7f5e400b5d70 2020-11-16T22:18:32.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.007+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 0x7f5e18057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.011+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 0x7f5e18057520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5e30006ef0 tx=0x7f5e30015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.011+0000 7f5e56323740 1 Processor -- start 2020-11-16T22:18:32.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.011+0000 7f5e56323740 1 -- start start 2020-11-16T22:18:32.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x12e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x12df380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x116c4b0 con 0x10c27b0 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1220930 con 0x1054bb0 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e06f0 con 0x12e0da0 2020-11-16T22:18:32.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e45e2f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x12df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60612/0 (socket says 172.21.15.185:60612) 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 -- 172.21.15.185:0/2304156752 learned_addr learned my addr 172.21.15.185:0/2304156752 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x12df380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54058/0 (socket says 172.21.15.185:54058) 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e4562e700 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x12e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 -- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 msgr2=0x12df380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:32.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x12df380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 -- 172.21.15.185:0/2304156752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 msgr2=0x12e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x12e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 -- 172.21.15.185:0/2304156752 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10db5a0 con 0x1054bb0 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5e3c047060 tx=0x7f5e3c010000).ready entity=mon.1 client_cookie=4bce1dc862d389d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e227fc700 1 -- 172.21.15.185:0/2304156752 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c045600 con 0x1054bb0 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e227fc700 1 -- 172.21.15.185:0/2304156752 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c005c60 con 0x1054bb0 2020-11-16T22:18:32.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e227fc700 1 -- 172.21.15.185:0/2304156752 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c06d0d0 con 0x1054bb0 2020-11-16T22:18:33.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 msgr2=0x12dee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f5e3c047060 tx=0x7f5e3c010000).stop 2020-11-16T22:18:33.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- 172.21.15.185:0/2304156752 shutdown_connections 2020-11-16T22:18:33.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x12df380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x12e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 --2- 172.21.15.185:0/2304156752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x12dee90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- 172.21.15.185:0/2304156752 >> 172.21.15.185:0/2304156752 conn(0x1290e40 msgr2=0x12dddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- 172.21.15.185:0/2304156752 shutdown_connections 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- 172.21.15.185:0/2304156752 wait complete. 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 -- start start 2020-11-16T22:18:33.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.015+0000 7f5e56323740 1 Processor -- start 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x12e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x130ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x120d6e0 con 0x1054bb0 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x120d820 con 0x10c27b0 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x120ea90 con 0x12e0da0 2020-11-16T22:18:33.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e45e2f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x130ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 10 client.0 ll_register_callbacks cb 0x1203ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e45e2f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x130ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54064/0 (socket says 172.21.15.185:54064) 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e45e2f700 1 -- 172.21.15.185:0/1537609693 learned_addr learned my addr 172.21.15.185:0/1537609693 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x12e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 msgr2=0x130ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x130ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 msgr2=0x1324600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 -- 172.21.15.185:0/1537609693 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11fe280 con 0x10c27b0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x12e3da0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f5e3c007520 tx=0x7f5e3c04c000).ready entity=mon.1 client_cookie=96284e17079ebd37 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c05a070 con 0x10c27b0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e3c010300 con 0x10c27b0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12c16c0 con 0x10c27b0 2020-11-16T22:18:33.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3c0431e0 con 0x10c27b0 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 10 client.6601 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1033fd0 con 0x10c27b0 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12c16c0 con 0x10c27b0 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f5e3c01b020 con 0x10c27b0 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 10 client.6601 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 10 client.6601 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:33.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 20 client.6601 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x12e04e0 0x13283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 4 client.6601 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x10fec50 con 0x12e04e0 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.019+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x12e04e0 0x13283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e0f7fe700 10 client.6601 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e4562e700 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x12e04e0 0x13283a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f67c05a2b6bfd916 server_cookie=3c0bf4bff9b3c064 in_seq=0 out_seq=0 2020-11-16T22:18:33.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/1537609693 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x10fec50 con 0x12e04e0 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e0f7fe700 10 client.6601 handle_command_reply: tid=0 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 client.6601 shutdown 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 msgr2=0x12e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x12e3da0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f5e3c007520 tx=0x7f5e3c04c000).stop 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 shutdown_connections 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x12e04e0 0x13283a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x130ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e0f7fe700 0 client.6601 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 --2- 172.21.15.185:0/1537609693 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x12e3da0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 >> 172.21.15.185:0/1537609693 conn(0x1290e40 msgr2=0x1310390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 shutdown_connections 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 1 -- 172.21.15.185:0/1537609693 wait complete. 2020-11-16T22:18:33.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.023+0000 7f5e56323740 20 client.6601 trim_cache size 0 max 16384 2020-11-16T22:18:33.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 Processor -- start 2020-11-16T22:18:33.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 -- start start 2020-11-16T22:18:33.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 0x1312e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x1323150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x11e5e40 con 0x1054bb0 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1325950 con 0x10c27b0 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x12ba800 con 0x1290e40 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e45e2f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x1323150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e45e2f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x1323150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60624/0 (socket says 172.21.15.185:60624) 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e45e2f700 1 -- 172.21.15.185:0/911419284 learned_addr learned my addr 172.21.15.185:0/911419284 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.043+0000 7f5e4562e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 0x1312e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 -- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 msgr2=0x1323150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x1323150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 -- 172.21.15.185:0/911419284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 msgr2=0x1324c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 -- 172.21.15.185:0/911419284 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12e4100 con 0x1290e40 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e4562e700 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 0x1312e90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f5e30012000 tx=0x7f5e30022040).ready entity=mon.2 client_cookie=f3a13cf66ed18ecb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e22ffd700 1 -- 172.21.15.185:0/911419284 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000b780 con 0x1290e40 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e22ffd700 1 -- 172.21.15.185:0/911419284 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e300300c0 con 0x1290e40 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e22ffd700 1 -- 172.21.15.185:0/911419284 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30011610 con 0x1290e40 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 msgr2=0x1312e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 0x1312e90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f5e30012000 tx=0x7f5e30022040).stop 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- 172.21.15.185:0/911419284 shutdown_connections 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x10c27b0 0x1323150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1290e40 0x1312e90 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0x7f5e30012000 tx=0x7f5e30022040).stop 2020-11-16T22:18:33.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 --2- 172.21.15.185:0/911419284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1054bb0 0x1324c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- 172.21.15.185:0/911419284 >> 172.21.15.185:0/911419284 conn(0x7f5e14002320 msgr2=0x12df380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- 172.21.15.185:0/911419284 shutdown_connections 2020-11-16T22:18:33.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- 172.21.15.185:0/911419284 wait complete. 2020-11-16T22:18:33.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 -- start start 2020-11-16T22:18:33.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.047+0000 7f5e56323740 1 Processor -- start 2020-11-16T22:18:33.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x137aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x137b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x12e20a0 con 0x10c27b0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x12dd8e0 con 0x1054bb0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x12ddb10 con 0x12e0da0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60630/0 (socket says 172.21.15.185:60630) 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 -- 172.21.15.185:0/430393680 learned_addr learned my addr 172.21.15.185:0/430393680 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 10 client.0 ll_register_callbacks cb 0x1313840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e45e2f700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x137b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 -- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 msgr2=0x137b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x137b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 -- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 msgr2=0x137aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x137aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 -- 172.21.15.185:0/430393680 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e140081d0 con 0x1054bb0 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e4562e700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f5e30010000 tx=0x7f5e30032040).ready entity=mon.1 client_cookie=bfefa4789c72480c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:33.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000b780 con 0x1054bb0 2020-11-16T22:18:33.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12dba60 con 0x1054bb0 2020-11-16T22:18:33.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e30026040 con 0x1054bb0 2020-11-16T22:18:33.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e30011de0 con 0x1054bb0 2020-11-16T22:18:33.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 10 client.6607 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:33.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1072720 con 0x1054bb0 2020-11-16T22:18:33.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12dba60 con 0x1054bb0 2020-11-16T22:18:33.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f5e30037070 con 0x1054bb0 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 10 client.6607 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 10 client.6607 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 20 client.6607 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x137b680 0x1318fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 4 client.6607 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x7f5e3c005c60 con 0x137b680 2020-11-16T22:18:33.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.051+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x137b680 0x1318fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e44e2d700 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x137b680 0x1318fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bd7f714f7dd53f0c server_cookie=f1ea45504f34050d in_seq=0 out_seq=0 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e0f7fe700 10 client.6607 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e0f7fe700 1 -- 172.21.15.185:0/430393680 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f5e3c005c60 con 0x137b680 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e0f7fe700 10 client.6607 handle_command_reply: tid=0 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 client.6607 shutdown 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 msgr2=0x1325070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f5e30010000 tx=0x7f5e30032040).stop 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 shutdown_connections 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1054bb0 0x1325070 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x137b680 0x1318fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12e0da0 0x137b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e0f7fe700 0 client.6607 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 --2- 172.21.15.185:0/430393680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x10c27b0 0x137aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:18:33.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 >> 172.21.15.185:0/430393680 conn(0x7f5e14002320 msgr2=0x130f890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 shutdown_connections 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 1 -- 172.21.15.185:0/430393680 wait complete. 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e56323740 20 client.6607 trim_cache size 0 max 16384 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 msgr2=0x7f5e18057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 0x7f5e18057520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5e30006ef0 tx=0x7f5e30015000).stop 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 msgr2=0x7f5e40147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f5e3c00c730 tx=0x7f5e3c007a30).stop 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 shutdown_connections 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e18055060 0x7f5e18057520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e40143470 0x7f5e40142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e400b4470 0x7f5e400b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 --2- 172.21.15.185:0/1330874500 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e400b5d70 0x7f5e40147b90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 >> 172.21.15.185:0/1330874500 conn(0x7f5e4005ba80 msgr2=0x7f5e400b91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 shutdown_connections 2020-11-16T22:18:33.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.055+0000 7f5e46630700 1 -- 172.21.15.185:0/1330874500 wait complete. 2020-11-16T22:18:33.045 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:33.045 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:18:33.045 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:18:33.045 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:33.053 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s 2020-11-16T22:18:33.054 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:33.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.199+0000 7f8d83fff700 1 Processor -- start 2020-11-16T22:18:33.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.199+0000 7f8d83fff700 1 -- start start 2020-11-16T22:18:33.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 0x7f8d840674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d83fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8d8413f9b0 0x7f8d8413fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d83fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840b3b70 con 0x7f8d840670c0 2020-11-16T22:18:33.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d82ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f8d840670c0 0x7f8d84068170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54076/0 (socket says 172.21.15.185:54076) 2020-11-16T22:18:33.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 --2- 172.21.15.185:0/1271585050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 0x7f8d840674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d82ffd700 1 -- 172.21.15.185:0/1271585050 learned_addr learned my addr 172.21.15.185:0/1271585050 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:33.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d84067990 con 0x7f8d8413f9b0 2020-11-16T22:18:33.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84144500 con 0x7f8d840b2450 2020-11-16T22:18:33.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d837fe700 1 --2- 172.21.15.185:0/1271585050 >> v2:172.21.15.185:3300/0 conn(0x7f8d8413f9b0 0x7f8d8413fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975386292 0 0) 0x7f8d840b3b70 con 0x7f8d840670c0 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d70003240 con 0x7f8d840670c0 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 -- 172.21.15.185:0/1271585050 >> v2:172.21.15.185:3300/0 conn(0x7f8d8413f9b0 msgr2=0x7f8d8413fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 --2- 172.21.15.185:0/1271585050 >> v2:172.21.15.185:3300/0 conn(0x7f8d8413f9b0 0x7f8d8413fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 657614108 0 0) 0x7f8d70003240 con 0x7f8d840670c0 2020-11-16T22:18:33.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 -- 172.21.15.185:0/1271585050 >> v1:172.21.15.185:6789/0 conn(0x7f8d840670c0 legacy=0x7f8d84068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 -- 172.21.15.185:0/1271585050 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d841447a0 con 0x7f8d840b2450 2020-11-16T22:18:33.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d827fc700 1 --2- 172.21.15.185:0/1271585050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 0x7f8d840674a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f8d7400b9a0 tx=0x7f8d74006df0).ready entity=mon.2 client_cookie=efa2c19b8e225164 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c001860 con 0x7f8d840b2450 2020-11-16T22:18:33.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74011400 con 0x7f8d840b2450 2020-11-16T22:18:33.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.203+0000 7f8d81ffb700 1 -- 172.21.15.185:0/1271585050 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011780 con 0x7f8d840b2450 2020-11-16T22:18:33.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 msgr2=0x7f8d840674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1271585050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 0x7f8d840674a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f8d7400b9a0 tx=0x7f8d74006df0).stop 2020-11-16T22:18:33.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 shutdown_connections 2020-11-16T22:18:33.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1271585050 >> v2:172.21.15.185:3300/0 conn(0x7f8d8413f9b0 0x7f8d8413fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1271585050 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d840b2450 0x7f8d840674a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 >> 172.21.15.185:0/1271585050 conn(0x7f8d84060130 msgr2=0x7f8d84060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 shutdown_connections 2020-11-16T22:18:33.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- 172.21.15.185:0/1271585050 wait complete. 2020-11-16T22:18:33.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 Processor -- start 2020-11-16T22:18:33.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.207+0000 7f8d83fff700 1 -- start start 2020-11-16T22:18:33.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d8413f9b0 0x7f8d8413e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 0x7f8d84150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8413a4e0 con 0x7f8d8413f9b0 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84144010 con 0x7f8d840670c0 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840aeb30 con 0x7f8d8413e670 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d82ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d82ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60644/0 (socket says 172.21.15.185:60644) 2020-11-16T22:18:33.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 0x7f8d84150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d82ffd700 1 -- 172.21.15.185:0/1772023898 learned_addr learned my addr 172.21.15.185:0/1772023898 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d827fc700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d8413f9b0 0x7f8d8413e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 -- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 msgr2=0x7f8d840af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 -- 172.21.15.185:0/1772023898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d8413f9b0 msgr2=0x7f8d8413e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d8413f9b0 0x7f8d8413e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 -- 172.21.15.185:0/1772023898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840af0c0 con 0x7f8d8413e670 2020-11-16T22:18:33.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d82ffd700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:33.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d837fe700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 0x7f8d84150540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8d7800e950 tx=0x7f8d78006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800c6b0 con 0x7f8d8413e670 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7800c810 con 0x7f8d8413e670 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840af360 con 0x7f8d8413e670 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800cbd0 con 0x7f8d8413e670 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8413bc10 con 0x7f8d8413e670 2020-11-16T22:18:33.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.211+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d84060bf0 con 0x7f8d8413e670 2020-11-16T22:18:33.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.215+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8d7801a030 con 0x7f8d8413e670 2020-11-16T22:18:33.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.215+0000 7f8d6affd700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 0x7f8d60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.215+0000 7f8d82ffd700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 0x7f8d60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.215+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8d7806d8e0 con 0x7f8d8413e670 2020-11-16T22:18:33.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.215+0000 7f8d82ffd700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 0x7f8d60057520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8d7c001a10 tx=0x7f8d7c0021d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.219+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8d78043b40 con 0x7f8d8413e670 2020-11-16T22:18:33.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8405d200 con 0x7f8d8413e670 2020-11-16T22:18:33.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d6affd700 1 -- 172.21.15.185:0/1772023898 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f8d78047160 con 0x7f8d8413e670 2020-11-16T22:18:33.401 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:33.402 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:33.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 msgr2=0x7f8d60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 0x7f8d60057520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f8d7c001a10 tx=0x7f8d7c0021d0).stop 2020-11-16T22:18:33.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 msgr2=0x7f8d84150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.419+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 0x7f8d84150540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8d7800e950 tx=0x7f8d78006df0).stop 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 shutdown_connections 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8d60055060 0x7f8d60057520 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8d840670c0 0x7f8d840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8d8413e670 0x7f8d84150540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 --2- 172.21.15.185:0/1772023898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d8413f9b0 0x7f8d8413e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 >> 172.21.15.185:0/1772023898 conn(0x7f8d84060130 msgr2=0x7f8d840b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 shutdown_connections 2020-11-16T22:18:33.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.423+0000 7f8d83fff700 1 -- 172.21.15.185:0/1772023898 wait complete. 2020-11-16T22:18:33.405 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:33.416 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls 2020-11-16T22:18:33.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 Processor -- start 2020-11-16T22:18:33.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 -- start start 2020-11-16T22:18:33.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 0x7f706c0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0b7c20 con 0x7f706c13e1f0 2020-11-16T22:18:33.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0b3a80 con 0x7f706c0b3570 2020-11-16T22:18:33.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.559+0000 7f70718b8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0b3d20 con 0x7f706c13de10 2020-11-16T22:18:33.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70710b7700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f706c0b3570 0x7f706c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54090/0 (socket says 172.21.15.185:54090) 2020-11-16T22:18:33.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 0x7f706c0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70710b7700 1 -- 172.21.15.185:0/1033431452 learned_addr learned my addr 172.21.15.185:0/1033431452 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:33.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f706b7fe700 1 -- 172.21.15.185:0/1033431452 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810890460 0 0) 0x7f706c0b3a80 con 0x7f706c0b3570 2020-11-16T22:18:33.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70708b6700 1 -- 172.21.15.185:0/1033431452 >> v1:172.21.15.185:6789/0 conn(0x7f706c0b3570 legacy=0x7f706c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70708b6700 1 -- 172.21.15.185:0/1033431452 >> v1:172.21.15.96:6789/0 conn(0x7f706c13e1f0 legacy=0x7f706c0b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70708b6700 1 -- 172.21.15.185:0/1033431452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0b3fc0 con 0x7f706c13de10 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70708b6700 1 --2- 172.21.15.185:0/1033431452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 0x7f706c0666b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f706400b9d0 tx=0x7f7064006df0).ready entity=mon.2 client_cookie=c284e704bca56cd1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f706b7fe700 1 -- 172.21.15.185:0/1033431452 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064009870 con 0x7f706c13de10 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f706b7fe700 1 -- 172.21.15.185:0/1033431452 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70640099d0 con 0x7f706c13de10 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f706b7fe700 1 -- 172.21.15.185:0/1033431452 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064009d50 con 0x7f706c13de10 2020-11-16T22:18:33.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- 172.21.15.185:0/1033431452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 msgr2=0x7f706c0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 --2- 172.21.15.185:0/1033431452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 0x7f706c0666b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f706400b9d0 tx=0x7f7064006df0).stop 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- 172.21.15.185:0/1033431452 shutdown_connections 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 --2- 172.21.15.185:0/1033431452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c13de10 0x7f706c0666b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- 172.21.15.185:0/1033431452 >> 172.21.15.185:0/1033431452 conn(0x7f706c064b90 msgr2=0x7f706c063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- 172.21.15.185:0/1033431452 shutdown_connections 2020-11-16T22:18:33.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- 172.21.15.185:0/1033431452 wait complete. 2020-11-16T22:18:33.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 Processor -- start 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.563+0000 7f70718b8700 1 -- start start 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 0x7f706c067940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 0x7f706c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c156f00 con 0x7f706c13de10 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c148720 con 0x7f706c157820 2020-11-16T22:18:33.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c147ee0 con 0x7f706c0b3570 2020-11-16T22:18:33.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 0x7f706c067940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60662/0 (socket says 172.21.15.185:60662) 2020-11-16T22:18:33.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 -- 172.21.15.185:0/3696483119 learned_addr learned my addr 172.21.15.185:0/3696483119 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f706bfff700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 0x7f706c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 -- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 msgr2=0x7f706c067940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 0x7f706c067940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 -- 172.21.15.185:0/3696483119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 msgr2=0x7f706c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 0x7f706c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 -- 172.21.15.185:0/3696483119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c154490 con 0x7f706c157820 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70708b6700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 0x7f706c067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f706bfff700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 0x7f706c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70710b7700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f705c00c760 tx=0x7f705c007940).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f7068ff9700 1 -- 172.21.15.185:0/3696483119 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0072e0 con 0x7f706c157820 2020-11-16T22:18:33.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f7068ff9700 1 -- 172.21.15.185:0/3696483119 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c007440 con 0x7f706c157820 2020-11-16T22:18:33.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f7068ff9700 1 -- 172.21.15.185:0/3696483119 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c005840 con 0x7f706c157820 2020-11-16T22:18:33.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c154730 con 0x7f706c157820 2020-11-16T22:18:33.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.567+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c147bd0 con 0x7f706c157820 2020-11-16T22:18:33.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.571+0000 7f7068ff9700 1 -- 172.21.15.185:0/3696483119 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f705c010070 con 0x7f706c157820 2020-11-16T22:18:33.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.571+0000 7f7068ff9700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 0x7f7048057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.571+0000 7f7068ff9700 1 -- 172.21.15.185:0/3696483119 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f705c06d3b0 con 0x7f706c157820 2020-11-16T22:18:33.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.571+0000 7f70708b6700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 0x7f7048057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.571+0000 7f70708b6700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 0x7f7048057520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f706400b150 tx=0x7f7064015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 Processor -- start 2020-11-16T22:18:33.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- start start 2020-11-16T22:18:33.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15634b0 con 0x144bbb0 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1617930 con 0x14b97b0 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x14d76f0 con 0x16d7da0 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54114/0 (socket says 172.21.15.185:54114) 2020-11-16T22:18:33.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60668/0 (socket says 172.21.15.185:60668) 2020-11-16T22:18:33.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 -- 172.21.15.185:0/3213726594 learned_addr learned my addr 172.21.15.185:0/3213726594 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.559 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f706bfff700 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 -- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 msgr2=0x16d78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.560 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 -- 172.21.15.185:0/3213726594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x16d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 -- 172.21.15.185:0/3213726594 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14d25a0 con 0x16d7da0 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70708b6700 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70710b7700 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f705c046b70 tx=0x7f705c009000).ready entity=mon.2 client_cookie=97515bf85f972617 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f7061ffb700 1 -- 172.21.15.185:0/3213726594 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c04e070 con 0x16d7da0 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f7061ffb700 1 -- 172.21.15.185:0/3213726594 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c043870 con 0x16d7da0 2020-11-16T22:18:33.561 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f7061ffb700 1 -- 172.21.15.185:0/3213726594 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c046860 con 0x16d7da0 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 msgr2=0x16d6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f705c046b70 tx=0x7f705c009000).stop 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- 172.21.15.185:0/3213726594 shutdown_connections 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16d7da0 0x16d6380 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 --2- 172.21.15.185:0/3213726594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.562 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- 172.21.15.185:0/3213726594 >> 172.21.15.185:0/3213726594 conn(0x1687e40 msgr2=0x16d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- 172.21.15.185:0/3213726594 shutdown_connections 2020-11-16T22:18:33.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.575+0000 7f70815ab740 1 -- 172.21.15.185:0/3213726594 wait complete. 2020-11-16T22:18:33.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- start start 2020-11-16T22:18:33.563 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 Processor -- start 2020-11-16T22:18:33.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x16d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16d7da0 0x171f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16046e0 con 0x16d7da0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1604820 con 0x144bbb0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1605a90 con 0x14b97b0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 10 client.0 ll_register_callbacks cb 0x15faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54118/0 (socket says 172.21.15.185:54118) 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x16d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 -- 172.21.15.185:0/2136544570 learned_addr learned my addr 172.21.15.185:0/2136544570 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70710b7700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16d7da0 0x171f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 -- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 msgr2=0x16d9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x16d9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 -- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16d7da0 msgr2=0x171f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16d7da0 0x171f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 -- 172.21.15.185:0/2136544570 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15f5280 con 0x14b97b0 2020-11-16T22:18:33.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f706bfff700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f706c061210 tx=0x7f7054009040).ready entity=mon.2 client_cookie=487f40ed13035c78 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f703affd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540109d0 con 0x14b97b0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7054010b30 con 0x14b97b0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16b86c0 con 0x14b97b0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70540030d0 con 0x14b97b0 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 10 client.6032 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:33.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.579+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x142afd0 con 0x14b97b0 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16b86c0 con 0x14b97b0 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f705401c020 con 0x14b97b0 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 10 client.6032 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 10 client.6032 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 20 client.6032 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x170df90 0x171b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 4 client.6032 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x14f5c50 con 0x170df90 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70708b6700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x170df90 0x171b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 10 client.6032 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70708b6700 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x170df90 0x171b650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bcea43b2fc5ef9b6 server_cookie=3d6f65824325e18a in_seq=0 out_seq=0 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 1 -- 172.21.15.185:0/2136544570 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x14f5c50 con 0x170df90 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 10 client.6032 handle_command_reply: tid=0 2020-11-16T22:18:33.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 client.6032 shutdown 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 msgr2=0x170b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f706c061210 tx=0x7f7054009040).stop 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 shutdown_connections 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x16d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x170df90 0x171b650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16d7da0 0x171f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f703affd700 0 client.6032 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 --2- 172.21.15.185:0/2136544570 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x170b140 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 >> 172.21.15.185:0/2136544570 conn(0x1687e40 msgr2=0x16d5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 shutdown_connections 2020-11-16T22:18:33.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 1 -- 172.21.15.185:0/2136544570 wait complete. 2020-11-16T22:18:33.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.583+0000 7f70815ab740 20 client.6032 trim_cache size 0 max 16384 2020-11-16T22:18:33.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 Processor -- start 2020-11-16T22:18:33.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 -- start start 2020-11-16T22:18:33.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15dce40 con 0x144bbb0 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b9200 con 0x14b97b0 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x170a550 con 0x1687e40 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70708b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60682/0 (socket says 172.21.15.185:60682) 2020-11-16T22:18:33.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 -- 172.21.15.185:0/1428649560 learned_addr learned my addr 172.21.15.185:0/1428649560 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 -- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 msgr2=0x16d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f706bfff700 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 -- 172.21.15.185:0/1428649560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x16cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 -- 172.21.15.185:0/1428649560 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040008160 con 0x14b97b0 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f706bfff700 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:33.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.599+0000 7f70710b7700 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f705c046b70 tx=0x7f705c018000).ready entity=mon.1 client_cookie=692c7d12aade52f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70627fc700 1 -- 172.21.15.185:0/1428649560 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c05a070 con 0x14b97b0 2020-11-16T22:18:33.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70627fc700 1 -- 172.21.15.185:0/1428649560 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c0439e0 con 0x14b97b0 2020-11-16T22:18:33.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70627fc700 1 -- 172.21.15.185:0/1428649560 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c046860 con 0x14b97b0 2020-11-16T22:18:33.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 msgr2=0x16d5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f705c046b70 tx=0x7f705c018000).stop 2020-11-16T22:18:33.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/1428649560 shutdown_connections 2020-11-16T22:18:33.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d5d90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- 172.21.15.185:0/1428649560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/1428649560 >> 172.21.15.185:0/1428649560 conn(0x7f7040002320 msgr2=0x1706890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/1428649560 shutdown_connections 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/1428649560 wait complete. 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- start start 2020-11-16T22:18:33.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 Processor -- start 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1713e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1706030 con 0x144bbb0 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x17066c0 con 0x1687e40 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1777720 con 0x14b97b0 2020-11-16T22:18:33.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 10 client.0 ll_register_callbacks cb 0x170a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f706bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1713e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60690/0 (socket says 172.21.15.185:60690) 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f706bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1713e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54132/0 (socket says 172.21.15.185:54132) 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 -- 172.21.15.185:0/2764121892 learned_addr learned my addr 172.21.15.185:0/2764121892 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70708b6700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 -- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 msgr2=0x1713e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1713e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 -- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x16d72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 -- 172.21.15.185:0/2764121892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1469720 con 0x1687e40 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70708b6700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70710b7700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f705c00c730 tx=0x7f705c05a000).ready entity=mon.1 client_cookie=decaf797c361cef5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f703affd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c06bde0 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f705c042f50 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705c043870 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705c0460c0 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 10 client.6634 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.603+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x1474e00 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c043870 con 0x1687e40 2020-11-16T22:18:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f705c01b030 con 0x1687e40 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 10 client.6634 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 10 client.6634 resolve_mds: resolved ID 'b' to GID 6057 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 20 client.6634 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x16d7da0 0x177a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 4 client.6634 mds_command: new command op to 6057 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 --> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x14fcf50 con 0x16d7da0 2020-11-16T22:18:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70708b6700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x16d7da0 0x177a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 10 client.6634 ms_handle_connect on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70708b6700 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x16d7da0 0x177a1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=985619a51805407f server_cookie=fb0d9acc0828f69f in_seq=0 out_seq=0 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 1 -- 172.21.15.185:0/2764121892 <== mds.0 v2:172.21.15.96:6832/2034361528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+272 (crc 0 0 0) 0x14fcf50 con 0x16d7da0 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 10 client.6634 handle_command_reply: tid=0 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 client.6634 shutdown 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 msgr2=0x1714380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f705c00c730 tx=0x7f705c05a000).stop 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 shutdown_connections 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1687e40 0x1714380 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:6832/2034361528,v1:172.21.15.96:6834/2034361528] conn(0x16d7da0 0x177a1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 --2- 172.21.15.185:0/2764121892 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1713e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 >> 172.21.15.185:0/2764121892 conn(0x7f7040002320 msgr2=0x1702c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f703affd700 0 client.6634 ms_handle_reset on v2:172.21.15.96:6832/2034361528 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 shutdown_connections 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 1 -- 172.21.15.185:0/2764121892 wait complete. 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70815ab740 20 client.6634 trim_cache size 0 max 16384 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:33.598 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 2013393217, 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627778, 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3" 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 2416256392, 2020-11-16T22:18:33.599 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627779, 2020-11-16T22:18:33.600 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3/d4" 2020-11-16T22:18:33.600 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:33.600 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 msgr2=0x7f7048057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 0x7f7048057520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f706400b150 tx=0x7f7064015000).stop 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 msgr2=0x7f706c146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f705c00c760 tx=0x7f705c007940).stop 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 shutdown_connections 2020-11-16T22:18:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7048055060 0x7f7048057520 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f706c157820 0x7f706c146510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f706c0b3570 0x7f706c067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 --2- 172.21.15.185:0/3696483119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f706c13de10 0x7f706c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 >> 172.21.15.185:0/3696483119 conn(0x7f706c064b90 msgr2=0x7f706c13d670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 shutdown_connections 2020-11-16T22:18:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.607+0000 7f70718b8700 1 -- 172.21.15.185:0/3696483119 wait complete. 2020-11-16T22:18:33.606 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:33.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 Processor -- start 2020-11-16T22:18:33.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 -- start start 2020-11-16T22:18:33.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 0x7f3ef00674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00b3b70 con 0x7f3ef013f9b0 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef0067990 con 0x7f3ef00b2450 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0144500 con 0x7f3ef00670c0 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eee7fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3ef00b2450 0x7f3ef0068170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54132/0 (socket says 172.21.15.185:54132) 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eee7fc700 1 -- 172.21.15.185:0/1344115982 learned_addr learned my addr 172.21.15.185:0/1344115982 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeeffd700 1 --2- 172.21.15.185:0/1344115982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 0x7f3ef00674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019590768 0 0) 0x7f3ef0067990 con 0x7f3ef00b2450 2020-11-16T22:18:33.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3edc003240 con 0x7f3ef00b2450 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015375988 0 0) 0x7f3ef00b3b70 con 0x7f3ef013f9b0 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef0067990 con 0x7f3ef013f9b0 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2997846498 0 0) 0x7f3edc003240 con 0x7f3ef00b2450 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeeffd700 1 -- 172.21.15.185:0/1344115982 >> v1:172.21.15.185:6789/0 conn(0x7f3ef00b2450 legacy=0x7f3ef0068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeeffd700 1 -- 172.21.15.185:0/1344115982 >> v1:172.21.15.96:6789/0 conn(0x7f3ef013f9b0 legacy=0x7f3ef013fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.737 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeeffd700 1 -- 172.21.15.185:0/1344115982 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef01447a0 con 0x7f3ef00670c0 2020-11-16T22:18:33.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.751+0000 7f3eeeffd700 1 --2- 172.21.15.185:0/1344115982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 0x7f3ef00674a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f3ee0009c40 tx=0x7f3ee00091d0).ready entity=mon.2 client_cookie=df864b13554023e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40014e0 con 0x7f3ef00670c0 2020-11-16T22:18:33.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee000b730 con 0x7f3ef00670c0 2020-11-16T22:18:33.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eedffb700 1 -- 172.21.15.185:0/1344115982 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000bab0 con 0x7f3ef00670c0 2020-11-16T22:18:33.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- 172.21.15.185:0/1344115982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 msgr2=0x7f3ef00674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 --2- 172.21.15.185:0/1344115982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 0x7f3ef00674a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f3ee0009c40 tx=0x7f3ee00091d0).stop 2020-11-16T22:18:33.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- 172.21.15.185:0/1344115982 shutdown_connections 2020-11-16T22:18:33.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 --2- 172.21.15.185:0/1344115982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef00670c0 0x7f3ef00674a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- 172.21.15.185:0/1344115982 >> 172.21.15.185:0/1344115982 conn(0x7f3ef0060130 msgr2=0x7f3ef0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- 172.21.15.185:0/1344115982 shutdown_connections 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- 172.21.15.185:0/1344115982 wait complete. 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 Processor -- start 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- start start 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3ef00670c0 0x7f3ef00ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef013f9b0 0x7f3ef01532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0157330 con 0x7f3ef013f9b0 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef013f330 con 0x7f3ef00670c0 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eeffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0157560 con 0x7f3ef013e760 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.755+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54146/0 (socket says 172.21.15.185:54146) 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 -- 172.21.15.185:0/3964588925 learned_addr learned my addr 172.21.15.185:0/3964588925 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:33.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eee7fc700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef013f9b0 0x7f3ef01532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eeeffd700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3ef00670c0 0x7f3ef00ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 -- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3ef00670c0 msgr2=0x7f3ef00ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3ef00670c0 0x7f3ef00ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 -- 172.21.15.185:0/3964588925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef013f9b0 msgr2=0x7f3ef01532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef013f9b0 0x7f3ef01532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 -- 172.21.15.185:0/3964588925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00adbd0 con 0x7f3ef013e760 2020-11-16T22:18:33.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eef7fe700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f3ee800c740 tx=0x7f3ee80079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800e490 con 0x7f3ef013e760 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee800e5f0 con 0x7f3ef013e760 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800e970 con 0x7f3ef013e760 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef013f5d0 con 0x7f3ef013e760 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef013dad0 con 0x7f3ef013e760 2020-11-16T22:18:33.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef005d200 con 0x7f3ef013e760 2020-11-16T22:18:33.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.759+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3ee8019030 con 0x7f3ef013e760 2020-11-16T22:18:33.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.763+0000 7f3ed6ffd700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 0x7f3ecc057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:33.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.763+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3ee806d090 con 0x7f3ef013e760 2020-11-16T22:18:33.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.763+0000 7f3eeeffd700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 0x7f3ecc057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:33.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.763+0000 7f3eeeffd700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 0x7f3ecc057550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f3ee000b640 tx=0x7f3ee0016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:33.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.771+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3ee8043340 con 0x7f3ef013e760 2020-11-16T22:18:33.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.975+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ef0061ed0 con 0x7f3ef013e760 2020-11-16T22:18:33.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3ed6ffd700 1 -- 172.21.15.185:0/3964588925 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f3ee8046960 con 0x7f3ef013e760 2020-11-16T22:18:33.959 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:33.960 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:33.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 msgr2=0x7f3ecc057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 0x7f3ecc057550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f3ee000b640 tx=0x7f3ee0016000).stop 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 msgr2=0x7f3ef01537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f3ee800c740 tx=0x7f3ee80079e0).stop 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 shutdown_connections 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3ecc055090 0x7f3ecc057550 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3ef00670c0 0x7f3ef00ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3ef013e760 0x7f3ef01537a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 --2- 172.21.15.185:0/3964588925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef013f9b0 0x7f3ef01532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:33.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 >> 172.21.15.185:0/3964588925 conn(0x7f3ef0060130 msgr2=0x7f3ef00b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:33.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 shutdown_connections 2020-11-16T22:18:33.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:33.979+0000 7f3eeffff700 1 -- 172.21.15.185:0/3964588925 wait complete. 2020-11-16T22:18:33.963 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:33.974 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls 2020-11-16T22:18:34.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 Processor -- start 2020-11-16T22:18:34.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 -- start start 2020-11-16T22:18:34.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e5c063df0 0x7f5e5c0641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0b4360 con 0x7f5e5c062ff0 2020-11-16T22:18:34.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c0b4600 con 0x7f5e5c063df0 2020-11-16T22:18:34.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5b59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0b48a0 con 0x7f5e5c061240 2020-11-16T22:18:34.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54152/0 (socket says 172.21.15.185:54152) 2020-11-16T22:18:34.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e59d9b700 1 -- 172.21.15.185:0/2104745613 learned_addr learned my addr 172.21.15.185:0/2104745613 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.119+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/2104745613 >> v2:172.21.15.96:3300/0 conn(0x7f5e5c063df0 0x7f5e5c0641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e59d9b700 1 -- 172.21.15.185:0/2104745613 >> v2:172.21.15.96:3300/0 conn(0x7f5e5c063df0 msgr2=0x7f5e5c0641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.105 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/2104745613 >> v2:172.21.15.96:3300/0 conn(0x7f5e5c063df0 0x7f5e5c0641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e59d9b700 1 -- 172.21.15.185:0/2104745613 >> v1:172.21.15.96:6789/0 conn(0x7f5e5c062ff0 legacy=0x7f5e5c0b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e59d9b700 1 -- 172.21.15.185:0/2104745613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0b5f70 con 0x7f5e5c061240 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/2104745613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f5e50006fd0 tx=0x7f5e50006eb0).ready entity=mon.2 client_cookie=503ce58d5b131c4e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5959a700 1 -- 172.21.15.185:0/2104745613 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50009810 con 0x7f5e5c061240 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5959a700 1 -- 172.21.15.185:0/2104745613 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e50009970 con 0x7f5e5c061240 2020-11-16T22:18:34.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5959a700 1 -- 172.21.15.185:0/2104745613 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50009cf0 con 0x7f5e5c061240 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- 172.21.15.185:0/2104745613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 msgr2=0x7f5e5c062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/2104745613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f5e50006fd0 tx=0x7f5e50006eb0).stop 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- 172.21.15.185:0/2104745613 shutdown_connections 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/2104745613 >> v2:172.21.15.96:3300/0 conn(0x7f5e5c063df0 0x7f5e5c0641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/2104745613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c061240 0x7f5e5c062b00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- 172.21.15.185:0/2104745613 >> 172.21.15.185:0/2104745613 conn(0x7f5e5c067940 msgr2=0x7f5e5c067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- 172.21.15.185:0/2104745613 shutdown_connections 2020-11-16T22:18:34.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- 172.21.15.185:0/2104745613 wait complete. 2020-11-16T22:18:34.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 Processor -- start 2020-11-16T22:18:34.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.123+0000 7f5e5b59e700 1 -- start start 2020-11-16T22:18:34.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e5c061240 0x7f5e5c142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c062ff0 0x7f5e5c0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c149f50 con 0x7f5e5c061240 2020-11-16T22:18:34.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ae480 con 0x7f5e5c063df0 2020-11-16T22:18:34.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ad830 con 0x7f5e5c062ff0 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c062ff0 0x7f5e5c0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60716/0 (socket says 172.21.15.185:60716) 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 -- 172.21.15.185:0/3303338958 learned_addr learned my addr 172.21.15.185:0/3303338958 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e5c061240 0x7f5e5c142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 -- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c062ff0 msgr2=0x7f5e5c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c062ff0 0x7f5e5c0ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 -- 172.21.15.185:0/3303338958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e5c061240 msgr2=0x7f5e5c142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e5c061240 0x7f5e5c142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 -- 172.21.15.185:0/3303338958 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0acc60 con 0x7f5e5c063df0 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f5e5400cac0 tx=0x7f5e54007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e4a7fc700 1 -- 172.21.15.185:0/3303338958 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400ce90 con 0x7f5e5c063df0 2020-11-16T22:18:34.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e4a7fc700 1 -- 172.21.15.185:0/3303338958 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5400a730 con 0x7f5e5c063df0 2020-11-16T22:18:34.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c0b1ce0 con 0x7f5e5c063df0 2020-11-16T22:18:34.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e4a7fc700 1 -- 172.21.15.185:0/3303338958 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400aaf0 con 0x7f5e5c063df0 2020-11-16T22:18:34.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c0ae6b0 con 0x7f5e5c063df0 2020-11-16T22:18:34.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.127+0000 7f5e4a7fc700 1 -- 172.21.15.185:0/3303338958 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5e54019030 con 0x7f5e5c063df0 2020-11-16T22:18:34.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.131+0000 7f5e4a7fc700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 0x7f5e30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.131+0000 7f5e4a7fc700 1 -- 172.21.15.185:0/3303338958 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e5406d3e0 con 0x7f5e5c063df0 2020-11-16T22:18:34.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.131+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 0x7f5e30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.131+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 0x7f5e30057520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5e4c006fd0 tx=0x7f5e4c006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 Processor -- start 2020-11-16T22:18:34.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- start start 2020-11-16T22:18:34.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x2496e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x2497380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23244b0 con 0x2498da0 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d8930 con 0x227a7b0 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x22986f0 con 0x220cbb0 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x2496e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60722/0 (socket says 172.21.15.185:60722) 2020-11-16T22:18:34.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 -- 172.21.15.185:0/2799522035 learned_addr learned my addr 172.21.15.185:0/2799522035 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x2497380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 -- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 msgr2=0x2496e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x2496e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 -- 172.21.15.185:0/2799522035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 msgr2=0x2497380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x2497380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 -- 172.21.15.185:0/2799522035 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22935a0 con 0x227a7b0 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5e4c014e80 tx=0x7f5e4c013000).ready entity=mon.1 client_cookie=9ed5ced1d2a5ad2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e377fe700 1 -- 172.21.15.185:0/2799522035 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c011a50 con 0x227a7b0 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e377fe700 1 -- 172.21.15.185:0/2799522035 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c011bb0 con 0x227a7b0 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e377fe700 1 -- 172.21.15.185:0/2799522035 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c020410 con 0x227a7b0 2020-11-16T22:18:34.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 msgr2=0x24988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5e4c014e80 tx=0x7f5e4c013000).stop 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- 172.21.15.185:0/2799522035 shutdown_connections 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x227a7b0 0x24988b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x2497380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/2799522035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x2496e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- 172.21.15.185:0/2799522035 >> 172.21.15.185:0/2799522035 conn(0x2448e40 msgr2=0x2495dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- 172.21.15.185:0/2799522035 shutdown_connections 2020-11-16T22:18:34.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- 172.21.15.185:0/2799522035 wait complete. 2020-11-16T22:18:34.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 -- start start 2020-11-16T22:18:34.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.135+0000 7f5e6f52a740 1 Processor -- start 2020-11-16T22:18:34.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x249be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x24c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c56e0 con 0x2498da0 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x23c5820 con 0x220cbb0 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x23c6a90 con 0x227a7b0 2020-11-16T22:18:34.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 10 client.0 ll_register_callbacks cb 0x23bbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x249be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54176/0 (socket says 172.21.15.185:54176) 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 -- 172.21.15.185:0/1534673754 learned_addr learned my addr 172.21.15.185:0/1534673754 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x24c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 -- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 msgr2=0x249be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x249be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 -- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 msgr2=0x24c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x24c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 -- 172.21.15.185:0/1534673754 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23b6280 con 0x227a7b0 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f5e50010000 tx=0x7f5e50015000).ready entity=mon.2 client_cookie=c434ceaebd0a63b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:34.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50024070 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e500069e0 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24796c0 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e50009cf0 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 10 client.6056 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x21ebfd0 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24796c0 con 0x227a7b0 2020-11-16T22:18:34.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f5e5001fa90 con 0x227a7b0 2020-11-16T22:18:34.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 10 client.6056 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:34.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 10 client.6056 resolve_mds: resolved ID 'c' to GID 6054 2020-11-16T22:18:34.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 20 client.6056 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:34.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x249c2f0 0x24dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 4 client.6056 mds_command: new command op to 6054 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 --> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x22b6c50 con 0x249c2f0 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x249c2f0 0x24dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e34ff9700 10 client.6056 ms_handle_connect on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.139+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x249c2f0 0x24dfd00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=bef3ac8763bc06ba server_cookie=c594d946c8393d09 in_seq=0 out_seq=0 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e34ff9700 1 -- 172.21.15.185:0/1534673754 <== mds.1 v2:172.21.15.96:6833/774952173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x22b6c50 con 0x249c2f0 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e34ff9700 10 client.6056 handle_command_reply: tid=0 2020-11-16T22:18:34.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 client.6056 shutdown 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 msgr2=0x24c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f5e50010000 tx=0x7f5e50015000).stop 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 shutdown_connections 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x249c2f0 0x24dfd00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x249be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2498da0 0x24c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e34ff9700 0 client.6056 ms_handle_reset on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1534673754 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x227a7b0 0x24c6d40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 >> 172.21.15.185:0/1534673754 conn(0x2448e40 msgr2=0x2496e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 shutdown_connections 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1534673754 wait complete. 2020-11-16T22:18:34.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.143+0000 7f5e6f52a740 20 client.6056 trim_cache size 0 max 16384 2020-11-16T22:18:34.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 Processor -- start 2020-11-16T22:18:34.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- start start 2020-11-16T22:18:34.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x2494200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x24cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x239de40 con 0x227a7b0 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24975a0 con 0x220cbb0 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24cbde0 con 0x2448e40 2020-11-16T22:18:34.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54182/0 (socket says 172.21.15.185:54182) 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 -- 172.21.15.185:0/1239966846 learned_addr learned my addr 172.21.15.185:0/1239966846 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x2494200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x24cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 -- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 msgr2=0x2494200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x2494200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 -- 172.21.15.185:0/1239966846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 msgr2=0x24cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x24cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 -- 172.21.15.185:0/1239966846 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24008160 con 0x2448e40 2020-11-16T22:18:34.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x24cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5e4c020000 tx=0x7f5e4c017000).ready entity=mon.2 client_cookie=a6621f834e7fdcb9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e37fff700 1 -- 172.21.15.185:0/1239966846 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c020410 con 0x2448e40 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e37fff700 1 -- 172.21.15.185:0/1239966846 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c02d0d0 con 0x2448e40 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e37fff700 1 -- 172.21.15.185:0/1239966846 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c011d20 con 0x2448e40 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 msgr2=0x24dcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f5e4c020000 tx=0x7f5e4c017000).stop 2020-11-16T22:18:34.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1239966846 shutdown_connections 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x24cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2448e40 0x24dcc60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/1239966846 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x220cbb0 0x2494200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1239966846 >> 172.21.15.185:0/1239966846 conn(0x7f5e24002320 msgr2=0x2496db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1239966846 shutdown_connections 2020-11-16T22:18:34.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.159+0000 7f5e6f52a740 1 -- 172.21.15.185:0/1239966846 wait complete. 2020-11-16T22:18:34.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- start start 2020-11-16T22:18:34.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 Processor -- start 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x252e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 0x252e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cced0 con 0x227a7b0 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x24db660 con 0x2448e40 2020-11-16T22:18:34.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d5df0 con 0x220cbb0 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 10 client.0 ll_register_callbacks cb 0x249b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 0x252e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54186/0 (socket says 172.21.15.185:54186) 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 -- 172.21.15.185:0/773351736 learned_addr learned my addr 172.21.15.185:0/773351736 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x252e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 -- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 msgr2=0x252e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 0x252e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 -- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 msgr2=0x252e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x252e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 -- 172.21.15.185:0/773351736 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x222a720 con 0x220cbb0 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5ad9d700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 0x252e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x252e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e5a59c700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5e4c014da0 tx=0x7f5e4c030000).ready entity=mon.2 client_cookie=872daca6728f71d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0118d0 con 0x220cbb0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e4c0207c0 con 0x220cbb0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e4c011bb0 con 0x220cbb0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e4c020b90 con 0x220cbb0 2020-11-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 10 client.6062 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.163+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2235e00 con 0x220cbb0 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c011bb0 con 0x220cbb0 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f5e4c034020 con 0x220cbb0 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 10 client.6062 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 10 client.6062 resolve_mds: resolved ID 'c' to GID 6054 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 20 client.6062 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2498da0 0x24999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 4 client.6062 mds_command: new command op to 6054 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 --> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x22bdf50 con 0x2498da0 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2498da0 0x24999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 10 client.6062 ms_handle_connect on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e59d9b700 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2498da0 0x24999e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=26716648a41f5530 server_cookie=f76e938f9b7ce5eb in_seq=0 out_seq=0 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 1 -- 172.21.15.185:0/773351736 <== mds.1 v2:172.21.15.96:6833/774952173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x22bdf50 con 0x2498da0 2020-11-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 10 client.6062 handle_command_reply: tid=0 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 client.6062 shutdown 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 msgr2=0x24cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5e4c014da0 tx=0x7f5e4c030000).stop 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 shutdown_connections 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:6833/774952173,v1:172.21.15.96:6835/774952173] conn(0x2498da0 0x24999e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2448e40 0x252e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x220cbb0 0x24cc710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 --2- 172.21.15.185:0/773351736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x227a7b0 0x252e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 >> 172.21.15.185:0/773351736 conn(0x7f5e24002320 msgr2=0x24dd280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e34ff9700 0 client.6062 ms_handle_reset on v2:172.21.15.96:6833/774952173 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 shutdown_connections 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 1 -- 172.21.15.185:0/773351736 wait complete. 2020-11-16T22:18:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e6f52a740 20 client.6062 trim_cache size 0 max 16384 2020-11-16T22:18:34.158 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:18:34.158 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:34.158 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:34.158 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 1618616061, 2020-11-16T22:18:34.158 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627781, 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 2162878635, 2020-11-16T22:18:34.159 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627780, 2020-11-16T22:18:34.160 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3/d4/d5" 2020-11-16T22:18:34.160 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:34.160 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:18:34.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 msgr2=0x7f5e30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 0x7f5e30057520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5e4c006fd0 tx=0x7f5e4c006df0).stop 2020-11-16T22:18:34.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 msgr2=0x7f5e5c0ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f5e5400cac0 tx=0x7f5e54007e40).stop 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 shutdown_connections 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5e30055060 0x7f5e30057520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5e5c063df0 0x7f5e5c0ac770 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e5c061240 0x7f5e5c142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 --2- 172.21.15.185:0/3303338958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5e5c062ff0 0x7f5e5c0ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 >> 172.21.15.185:0/3303338958 conn(0x7f5e5c067940 msgr2=0x7f5e5c0626d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 shutdown_connections 2020-11-16T22:18:34.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.167+0000 7f5e5b59e700 1 -- 172.21.15.185:0/3303338958 wait complete. 2020-11-16T22:18:34.165 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:34.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.303+0000 7fcef59cf700 1 Processor -- start 2020-11-16T22:18:34.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.303+0000 7fcef59cf700 1 -- start start 2020-11-16T22:18:34.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcef00b6aa0 0x7fcef00acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef0147f90 con 0x7fcef00b72e0 2020-11-16T22:18:34.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00af020 con 0x7fcef00b6aa0 2020-11-16T22:18:34.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00b4810 con 0x7fcef00b64b0 2020-11-16T22:18:34.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef49cd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fcef00b72e0 0x7fcef0144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54188/0 (socket says 172.21.15.185:54188) 2020-11-16T22:18:34.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef49cd700 1 -- 172.21.15.185:0/2099670738 learned_addr learned my addr 172.21.15.185:0/2099670738 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:34.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceeffff700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 --2- 172.21.15.185:0/2099670738 >> v2:172.21.15.185:3300/0 conn(0x7fcef00b6aa0 0x7fcef00acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceef7fe700 1 -- 172.21.15.185:0/2099670738 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350732727 0 0) 0x7fcef0147f90 con 0x7fcef00b72e0 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceef7fe700 1 -- 172.21.15.185:0/2099670738 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced4003240 con 0x7fcef00b72e0 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 -- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 msgr2=0x7fcef00b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 -- 172.21.15.185:0/2099670738 >> v1:172.21.15.185:6789/0 conn(0x7fcef00b72e0 legacy=0x7fcef0144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 -- 172.21.15.185:0/2099670738 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0148460 con 0x7fcef00b6aa0 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceeffff700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef51ce700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef00acee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fcee8009c90 tx=0x7fcee80091d0).ready entity=mon.1 client_cookie=40ab70026be0af82 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceef7fe700 1 -- 172.21.15.185:0/2099670738 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800b9c0 con 0x7fcef00b6aa0 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceef7fe700 1 -- 172.21.15.185:0/2099670738 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee800bb20 con 0x7fcef00b6aa0 2020-11-16T22:18:34.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fceef7fe700 1 -- 172.21.15.185:0/2099670738 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800bea0 con 0x7fcef00b6aa0 2020-11-16T22:18:34.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 msgr2=0x7fcef00acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef00acee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fcee8009c90 tx=0x7fcee80091d0).stop 2020-11-16T22:18:34.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- 172.21.15.185:0/2099670738 shutdown_connections 2020-11-16T22:18:34.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef00acee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2099670738 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- 172.21.15.185:0/2099670738 >> 172.21.15.185:0/2099670738 conn(0x7fcef0067860 msgr2=0x7fcef0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- 172.21.15.185:0/2099670738 shutdown_connections 2020-11-16T22:18:34.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.307+0000 7fcef59cf700 1 -- 172.21.15.185:0/2099670738 wait complete. 2020-11-16T22:18:34.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.311+0000 7fcef59cf700 1 Processor -- start 2020-11-16T22:18:34.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.311+0000 7fcef59cf700 1 -- start start 2020-11-16T22:18:34.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcef00b72e0 0x7fcef01464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0157650 con 0x7fcef00b72e0 2020-11-16T22:18:34.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00b3370 con 0x7fcef00b6aa0 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0143b60 con 0x7fcef00b64b0 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fceeffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54200/0 (socket says 172.21.15.185:54200) 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 -- 172.21.15.185:0/2101876589 learned_addr learned my addr 172.21.15.185:0/2101876589 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef51ce700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcef00b72e0 0x7fcef01464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 -- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 msgr2=0x7fcef0157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef0157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 -- 172.21.15.185:0/2101876589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcef00b72e0 msgr2=0x7fcef01464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcef00b72e0 0x7fcef01464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 -- 172.21.15.185:0/2101876589 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0156780 con 0x7fcef00b64b0 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fceeffff700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef49cd700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fcee400c6c0 tx=0x7fcee4002860).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40072e0 con 0x7fcef00b64b0 2020-11-16T22:18:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee40054c0 con 0x7fcef00b64b0 2020-11-16T22:18:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4005840 con 0x7fcef00b64b0 2020-11-16T22:18:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef0156a20 con 0x7fcef00b64b0 2020-11-16T22:18:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.315+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef0146a60 con 0x7fcef00b64b0 2020-11-16T22:18:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef00ae470 con 0x7fcef00b64b0 2020-11-16T22:18:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcee4010070 con 0x7fcef00b64b0 2020-11-16T22:18:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fceecff9700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 0x7fcec8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fceeffff700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 0x7fcec8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcee406d510 con 0x7fcef00b64b0 2020-11-16T22:18:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.319+0000 7fceeffff700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 0x7fcec8057520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fcef0141630 tx=0x7fcedc006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.327+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcee4043660 con 0x7fcef00b64b0 2020-11-16T22:18:34.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.523+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef005d200 con 0x7fcef00b64b0 2020-11-16T22:18:34.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.523+0000 7fceecff9700 1 -- 172.21.15.185:0/2101876589 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7fcee4046c80 con 0x7fcef00b64b0 2020-11-16T22:18:34.507 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:34.507 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:34.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 msgr2=0x7fcec8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 0x7fcec8057520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fcef0141630 tx=0x7fcedc006df0).stop 2020-11-16T22:18:34.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 msgr2=0x7fcef00b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fcee400c6c0 tx=0x7fcee4002860).stop 2020-11-16T22:18:34.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 shutdown_connections 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcec8055060 0x7fcec8057520 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcef00b72e0 0x7fcef01464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcef00b64b0 0x7fcef00b3590 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 --2- 172.21.15.185:0/2101876589 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcef00b6aa0 0x7fcef0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 >> 172.21.15.185:0/2101876589 conn(0x7fcef0067860 msgr2=0x7fcef0144790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 shutdown_connections 2020-11-16T22:18:34.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.527+0000 7fcef59cf700 1 -- 172.21.15.185:0/2101876589 wait complete. 2020-11-16T22:18:34.511 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:34.526 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls 2020-11-16T22:18:34.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 Processor -- start 2020-11-16T22:18:34.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 -- start start 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 0x7fda180b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1814c660 con 0x7fda180b3080 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda180ac120 con 0x7fda180b13d0 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.667+0000 7fda1dde7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda180ac3c0 con 0x7fda180b3920 2020-11-16T22:18:34.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda17fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 0x7fda180b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda17fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 0x7fda180b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60766/0 (socket says 172.21.15.185:60766) 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda17fff700 1 -- 172.21.15.185:0/2462396843 learned_addr learned my addr 172.21.15.185:0/2462396843 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2462396843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54212/0 (socket says 172.21.15.185:54212) 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2462396843 >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 msgr2=0x7fda180b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2462396843 >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 0x7fda180b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2462396843 >> v1:172.21.15.185:6789/0 conn(0x7fda180b3080 legacy=0x7fda180ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2462396843 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180b20e0 con 0x7fda180b3920 2020-11-16T22:18:34.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2462396843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fda10009c90 tx=0x7fda100091d0).ready entity=mon.2 client_cookie=afcf25c652384068 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda177fe700 1 -- 172.21.15.185:0/2462396843 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000ba70 con 0x7fda180b3920 2020-11-16T22:18:34.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda177fe700 1 -- 172.21.15.185:0/2462396843 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000bbd0 con 0x7fda180b3920 2020-11-16T22:18:34.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda177fe700 1 -- 172.21.15.185:0/2462396843 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10011400 con 0x7fda180b3920 2020-11-16T22:18:34.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- 172.21.15.185:0/2462396843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 msgr2=0x7fda18148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 --2- 172.21.15.185:0/2462396843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fda10009c90 tx=0x7fda100091d0).stop 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- 172.21.15.185:0/2462396843 shutdown_connections 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 --2- 172.21.15.185:0/2462396843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18148750 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 --2- 172.21.15.185:0/2462396843 >> v2:172.21.15.185:3300/0 conn(0x7fda180b13d0 0x7fda180b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- 172.21.15.185:0/2462396843 >> 172.21.15.185:0/2462396843 conn(0x7fda18060130 msgr2=0x7fda18060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- 172.21.15.185:0/2462396843 shutdown_connections 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- 172.21.15.185:0/2462396843 wait complete. 2020-11-16T22:18:34.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 Processor -- start 2020-11-16T22:18:34.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.671+0000 7fda1dde7700 1 -- start start 2020-11-16T22:18:34.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 0x7fda18146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda180b3080 0x7fda18145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1814a5e0 con 0x7fda180b3080 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1815b940 con 0x7fda180b13d0 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda181471a0 con 0x7fda180b3920 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 0x7fda18146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54218/0 (socket says 172.21.15.185:54218) 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 0x7fda18146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60770/0 (socket says 172.21.15.185:60770) 2020-11-16T22:18:34.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 -- 172.21.15.185:0/796704471 learned_addr learned my addr 172.21.15.185:0/796704471 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda17fff700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda180b3080 0x7fda18145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 -- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 msgr2=0x7fda18146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 0x7fda18146cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 -- 172.21.15.185:0/796704471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda180b3080 msgr2=0x7fda18145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda180b3080 0x7fda18145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 -- 172.21.15.185:0/796704471 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1814a7c0 con 0x7fda180b3920 2020-11-16T22:18:34.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fda10009210 tx=0x7fda10016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda14ff9700 1 -- 172.21.15.185:0/796704471 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1001c6c0 con 0x7fda180b3920 2020-11-16T22:18:34.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda14ff9700 1 -- 172.21.15.185:0/796704471 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1001c820 con 0x7fda180b3920 2020-11-16T22:18:34.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda18146370 con 0x7fda180b3920 2020-11-16T22:18:34.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda14ff9700 1 -- 172.21.15.185:0/796704471 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1001cbe0 con 0x7fda180b3920 2020-11-16T22:18:34.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.675+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda180af5b0 con 0x7fda180b3920 2020-11-16T22:18:34.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda14ff9700 1 -- 172.21.15.185:0/796704471 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fda10024030 con 0x7fda180b3920 2020-11-16T22:18:34.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda14ff9700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 0x7fd9f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda1cde5700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 0x7fd9f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda14ff9700 1 -- 172.21.15.185:0/796704471 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda1006f840 con 0x7fda180b3920 2020-11-16T22:18:34.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda1cde5700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 0x7fd9f0057520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fda08003300 tx=0x7fda08007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 Processor -- start 2020-11-16T22:18:34.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 -- start start 2020-11-16T22:18:34.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2ae2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x296f4b0 con 0x2857bb0 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a23930 con 0x2ae3da0 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.679+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x28e36f0 con 0x28c57b0 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2ae2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54222/0 (socket says 172.21.15.185:54222) 2020-11-16T22:18:34.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 -- 172.21.15.185:0/3451858339 learned_addr learned my addr 172.21.15.185:0/3451858339 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda17fff700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 -- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 msgr2=0x2ae2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2ae2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 -- 172.21.15.185:0/3451858339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 msgr2=0x2ae1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 -- 172.21.15.185:0/3451858339 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28de5a0 con 0x28c57b0 2020-11-16T22:18:34.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda17fff700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2ae2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:34.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fda08015b60 tx=0x7fda0800a000).ready entity=mon.2 client_cookie=acd4bce8d22c94d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fd9f5ffb700 1 -- 172.21.15.185:0/3451858339 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08015350 con 0x28c57b0 2020-11-16T22:18:34.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fd9f5ffb700 1 -- 172.21.15.185:0/3451858339 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08017860 con 0x28c57b0 2020-11-16T22:18:34.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fd9f5ffb700 1 -- 172.21.15.185:0/3451858339 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08017be0 con 0x28c57b0 2020-11-16T22:18:34.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 msgr2=0x2ae38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fda08015b60 tx=0x7fda0800a000).stop 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- 172.21.15.185:0/3451858339 shutdown_connections 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2ae2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2ae38b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- 172.21.15.185:0/3451858339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- 172.21.15.185:0/3451858339 >> 172.21.15.185:0/3451858339 conn(0x2a93e40 msgr2=0x2ae0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- 172.21.15.185:0/3451858339 shutdown_connections 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- 172.21.15.185:0/3451858339 wait complete. 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- start start 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 Processor -- start 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2b276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a106e0 con 0x2857bb0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a10820 con 0x2ae3da0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a11a90 con 0x28c57b0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda2dada740 10 client.0 ll_register_callbacks cb 0x2a06ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda17fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2b276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1d5e6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60788/0 (socket says 172.21.15.185:60788) 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2458635280 learned_addr learned my addr 172.21.15.185:0/2458635280 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.683+0000 7fda1cde5700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 msgr2=0x2b276f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2b276f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2458635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 msgr2=0x2ae03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 -- 172.21.15.185:0/2458635280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a01280 con 0x2ae3da0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1cde5700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fda10011fd0 tx=0x7fda10012000).ready entity=mon.1 client_cookie=8bb59639ce5f33f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100457b0 con 0x2ae3da0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda10048640 con 0x2ae3da0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2ac46c0 con 0x2ae3da0 2020-11-16T22:18:34.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda10048a10 con 0x2ae3da0 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 10 client.6679 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2836fd0 con 0x2ae3da0 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2ac46c0 con 0x2ae3da0 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fda10047d40 con 0x2ae3da0 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 10 client.6679 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 10 client.6679 resolve_mds: resolved ID 'a' to GID 5681 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 20 client.6679 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:34.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2b2b920 0x2b121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 4 client.6679 mds_command: new command op to 5681 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 --> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2901c50 con 0x2b2b920 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1cde5700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2b2b920 0x2b121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 10 client.6679 ms_handle_connect on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda1cde5700 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2b2b920 0x2b121a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=d08680ea1adc6806 server_cookie=54eaa1ff1cb9ae69 in_seq=0 out_seq=0 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/2458635280 <== mds.2 v2:172.21.15.185:6834/3773729030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2901c50 con 0x2b2b920 2020-11-16T22:18:34.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 10 client.6679 handle_command_reply: tid=0 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 client.6679 shutdown 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 msgr2=0x2b2b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fda10011fd0 tx=0x7fda10012000).stop 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 shutdown_connections 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2b2b920 0x2b121a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2ae3da0 0x2b2b430 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2857bb0 0x2ae03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 --2- 172.21.15.185:0/2458635280 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28c57b0 0x2b276f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fd9e6ffd700 0 client.6679 ms_handle_reset on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 >> 172.21.15.185:0/2458635280 conn(0x2a93e40 msgr2=0x2ae1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 shutdown_connections 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 1 -- 172.21.15.185:0/2458635280 wait complete. 2020-11-16T22:18:34.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.687+0000 7fda2dada740 20 client.6679 trim_cache size 0 max 16384 2020-11-16T22:18:34.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 Processor -- start 2020-11-16T22:18:34.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 -- start start 2020-11-16T22:18:34.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2ae1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2b27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29e8e40 con 0x28c57b0 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2addc10 con 0x2a93e40 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ae1b20 con 0x2857bb0 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54236/0 (socket says 172.21.15.185:54236) 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2ae1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 -- 172.21.15.185:0/2014740366 learned_addr learned my addr 172.21.15.185:0/2014740366 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2b27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 -- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 msgr2=0x2ae1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2ae1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 -- 172.21.15.185:0/2014740366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 msgr2=0x2b27cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2b27cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 -- 172.21.15.185:0/2014740366 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec008160 con 0x2857bb0 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2b27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:34.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.707+0000 7fda17fff700 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fda0c006fd0 tx=0x7fda0c008040).ready entity=mon.2 client_cookie=8dcbd81f4c86f499 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fd9f67fc700 1 -- 172.21.15.185:0/2014740366 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c017bd0 con 0x2857bb0 2020-11-16T22:18:34.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fd9f67fc700 1 -- 172.21.15.185:0/2014740366 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c017d30 con 0x2857bb0 2020-11-16T22:18:34.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fd9f67fc700 1 -- 172.21.15.185:0/2014740366 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c017ea0 con 0x2857bb0 2020-11-16T22:18:34.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 msgr2=0x2ae22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fda0c006fd0 tx=0x7fda0c008040).stop 2020-11-16T22:18:34.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- 172.21.15.185:0/2014740366 shutdown_connections 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2ae1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2b27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 --2- 172.21.15.185:0/2014740366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae22a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- 172.21.15.185:0/2014740366 >> 172.21.15.185:0/2014740366 conn(0x7fd9ec002320 msgr2=0x2b18500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- 172.21.15.185:0/2014740366 shutdown_connections 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- 172.21.15.185:0/2014740366 wait complete. 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 -- start start 2020-11-16T22:18:34.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.711+0000 7fda2dada740 1 Processor -- start 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2ae4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2b795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b20a90 con 0x28c57b0 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae1400 con 0x2a93e40 2020-11-16T22:18:34.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ae4b60 con 0x2857bb0 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 10 client.0 ll_register_callbacks cb 0x2b16d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54242/0 (socket says 172.21.15.185:54242) 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 -- 172.21.15.185:0/121739428 learned_addr learned my addr 172.21.15.185:0/121739428 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2b795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda17fff700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2ae4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 -- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 msgr2=0x2b795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2b795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 -- 172.21.15.185:0/121739428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 msgr2=0x2ae4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2ae4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 -- 172.21.15.185:0/121739428 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2875720 con 0x2857bb0 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda17fff700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2ae4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1d5e6700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2b795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda1cde5700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fda0801c000 tx=0x7fda08023040).ready entity=mon.2 client_cookie=7f632a82116cdc23 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:34.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08026040 con 0x2857bb0 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda080179d0 con 0x2857bb0 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda08017860 con 0x2857bb0 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0802a280 con 0x2857bb0 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 10 client.6086 fetch_fsmap learned FSMap version 61 2020-11-16T22:18:34.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x2880e00 con 0x2857bb0 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08017860 con 0x2857bb0 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 61) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fda0801e020 con 0x2857bb0 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 10 client.6086 fetch_fsmap finished waiting for FSMap version 61 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 10 client.6086 resolve_mds: resolved ID 'a' to GID 5681 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 20 client.6086 populate_metadata read hostname 'smithi185' 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2ae3da0 0x2b79ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:34.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 4 client.6086 mds_command: new command op to 5681 tid=0[{,",p,r,e,f,i,x,",:, ,",d,a,m,a,g,e, ,l,s,",}] 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 --> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] -- command(tid 0: { " p r e f i x " : " d a m a g e l s " }) v1 -- 0x2908f50 con 0x2ae3da0 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda17fff700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2ae3da0 0x2b79ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 10 client.6086 ms_handle_connect on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda17fff700 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2ae3da0 0x2b79ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=b32b14a9d609bc1d server_cookie=8e9cd2747908d101 in_seq=0 out_seq=0 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 1 -- 172.21.15.185:0/121739428 <== mds.2 v2:172.21.15.185:6834/3773729030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+296 (crc 0 0 0) 0x2908f50 con 0x2ae3da0 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 10 client.6086 handle_command_reply: tid=0 2020-11-16T22:18:34.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 client.6086 shutdown 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 msgr2=0x2ae4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fda0801c000 tx=0x7fda08023040).stop 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 shutdown_connections 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:6834/3773729030,v1:172.21.15.185:6835/3773729030] conn(0x2ae3da0 0x2b79ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a93e40 0x2b795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2857bb0 0x2ae4180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 --2- 172.21.15.185:0/121739428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x28c57b0 0x2ae4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fd9e6ffd700 0 client.6086 ms_handle_reset on v2:172.21.15.185:6834/3773729030 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 >> 172.21.15.185:0/121739428 conn(0x7fd9ec002320 msgr2=0x2ae1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 shutdown_connections 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.715+0000 7fda2dada740 1 -- 172.21.15.185:0/121739428 wait complete. 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda2dada740 20 client.6086 trim_cache size 0 max 16384 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 msgr2=0x7fd9f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 0x7fd9f0057520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fda08003300 tx=0x7fda08007b90).stop 2020-11-16T22:18:34.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 msgr2=0x7fda18145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fda10009210 tx=0x7fda10016000).stop 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 shutdown_connections 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd9f0055060 0x7fd9f0057520 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fda180b13d0 0x7fda18146cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fda180b3920 0x7fda18145e80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 --2- 172.21.15.185:0/796704471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda180b3080 0x7fda18145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:34.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 >> 172.21.15.185:0/796704471 conn(0x7fda18060130 msgr2=0x7fda180b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 shutdown_connections 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:34.719+0000 7fda1dde7700 1 -- 172.21.15.185:0/796704471 wait complete. 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 2660984539, 2020-11-16T22:18:34.709 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627782, 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: "damage_type": "backtrace", 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: "id": 3813735853, 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: "ino": 1099511627783, 2020-11-16T22:18:34.710 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2020-11-16T22:18:34.711 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:34.711 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:18:34.718 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:18:34.719 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:34.754 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:34.771 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:34.786 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:18:34.786 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:18:34.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:18:34.787 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:18:34.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:18:34.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:34.886 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:18:34.907 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:18:35.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 Processor -- start 2020-11-16T22:18:35.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 -- start start 2020-11-16T22:18:35.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc2e80b3080 0x7fc2e80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e814c660 con 0x7fc2e80b13d0 2020-11-16T22:18:35.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e80ac120 con 0x7fc2e80b3920 2020-11-16T22:18:35.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e7fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e80ac3c0 con 0x7fc2e80b3080 2020-11-16T22:18:35.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e77fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc2e80b3920 0x7fc2e8148750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54248/0 (socket says 172.21.15.185:54248) 2020-11-16T22:18:35.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e77fe700 1 -- 172.21.15.185:0/161959623 learned_addr learned my addr 172.21.15.185:0/161959623 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:35.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc2e80b3080 0x7fc2e80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213646381 0 0) 0x7fc2e80ac120 con 0x7fc2e80b3920 2020-11-16T22:18:35.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2cc003240 con 0x7fc2e80b3920 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305944900 0 0) 0x7fc2e814c660 con 0x7fc2e80b13d0 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e6ffd700 1 -- 172.21.15.185:0/161959623 >> v1:172.21.15.185:6789/0 conn(0x7fc2e80b3920 legacy=0x7fc2e8148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e6ffd700 1 -- 172.21.15.185:0/161959623 >> v1:172.21.15.96:6789/0 conn(0x7fc2e80b13d0 legacy=0x7fc2e80b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.059+0000 7fc2e6ffd700 1 -- 172.21.15.185:0/161959623 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80acd50 con 0x7fc2e80b3080 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e6ffd700 1 --2- 172.21.15.185:0/161959623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3080 0x7fc2e80ac860 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc2e000b9a0 tx=0x7fc2e0006df0).ready entity=mon.1 client_cookie=8c66f416c4ac6d90 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00097b0 con 0x7fc2e80b3080 2020-11-16T22:18:35.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e0009910 con 0x7fc2e80b3080 2020-11-16T22:18:35.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e5ffb700 1 -- 172.21.15.185:0/161959623 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0009c90 con 0x7fc2e80b3080 2020-11-16T22:18:35.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- 172.21.15.185:0/161959623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3080 msgr2=0x7fc2e80ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/161959623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3080 0x7fc2e80ac860 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fc2e000b9a0 tx=0x7fc2e0006df0).stop 2020-11-16T22:18:35.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- 172.21.15.185:0/161959623 shutdown_connections 2020-11-16T22:18:35.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/161959623 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3080 0x7fc2e80ac860 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- 172.21.15.185:0/161959623 >> 172.21.15.185:0/161959623 conn(0x7fc2e8060130 msgr2=0x7fc2e8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:35.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- 172.21.15.185:0/161959623 shutdown_connections 2020-11-16T22:18:35.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- 172.21.15.185:0/161959623 wait complete. 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 Processor -- start 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- start start 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2e80b13d0 0x7fc2e80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e80af420 0x7fc2e810fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e810f640 con 0x7fc2e80af420 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e815bb30 con 0x7fc2e80b3920 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.063+0000 7fc2e7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81122b0 con 0x7fc2e80b13d0 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2e80b13d0 0x7fc2e80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60814/0 (socket says 172.21.15.185:60814) 2020-11-16T22:18:35.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 -- 172.21.15.185:0/2928943920 learned_addr learned my addr 172.21.15.185:0/2928943920 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e77fe700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e80af420 0x7fc2e810fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 -- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2e80b13d0 msgr2=0x7fc2e80aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2e80b13d0 0x7fc2e80aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 -- 172.21.15.185:0/2928943920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e80af420 msgr2=0x7fc2e810fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e80af420 0x7fc2e810fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 -- 172.21.15.185:0/2928943920 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80b6a80 con 0x7fc2e80b3920 2020-11-16T22:18:35.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e67fc700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fc2d000cae0 tx=0x7fc2d0007da0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000a7c0 con 0x7fc2e80b3920 2020-11-16T22:18:35.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000a920 con 0x7fc2e80b3920 2020-11-16T22:18:35.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e80b6d20 con 0x7fc2e80b3920 2020-11-16T22:18:35.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000acc0 con 0x7fc2e80b3920 2020-11-16T22:18:35.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e815ac80 con 0x7fc2e80b3920 2020-11-16T22:18:35.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e810f7b0 con 0x7fc2e80b3920 2020-11-16T22:18:35.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.067+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc2d0010070 con 0x7fc2e80b3920 2020-11-16T22:18:35.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.071+0000 7fc2deffd700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 0x7fc2b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.071+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc2d006dc00 con 0x7fc2e80b3920 2020-11-16T22:18:35.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.071+0000 7fc2e6ffd700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 0x7fc2b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.071+0000 7fc2e6ffd700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 0x7fc2b8057520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fc2e80b38f0 tx=0x7fc2e0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.075+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc2e810f7b0 con 0x7fc2e80b3920 2020-11-16T22:18:35.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.231+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc2e810f7b0 con 0x7fc2e80b3920 2020-11-16T22:18:35.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.239+0000 7fc2deffd700 1 -- 172.21.15.185:0/2928943920 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc2e810f7b0 con 0x7fc2e80b3920 2020-11-16T22:18:35.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 msgr2=0x7fc2b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 0x7fc2b8057520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fc2e80b38f0 tx=0x7fc2e0015000).stop 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 msgr2=0x7fc2e8146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fc2d000cae0 tx=0x7fc2d0007da0).stop 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 shutdown_connections 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc2b8055060 0x7fc2b8057520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e80af420 0x7fc2e810fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc2e80b13d0 0x7fc2e80aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 --2- 172.21.15.185:0/2928943920 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc2e80b3920 0x7fc2e8146500 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 >> 172.21.15.185:0/2928943920 conn(0x7fc2e8060130 msgr2=0x7fc2e80675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:35.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 shutdown_connections 2020-11-16T22:18:35.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.243+0000 7fc2e7fff700 1 -- 172.21.15.185:0/2928943920 wait complete. 2020-11-16T22:18:35.239 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2020-11-16T22:18:35.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 Processor -- start 2020-11-16T22:18:35.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 -- start start 2020-11-16T22:18:35.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8aec0670c0 0x7f8aec067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec09d000 con 0x7f8aec0676a0 2020-11-16T22:18:35.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec05c0b0 con 0x7f8aec1488a0 2020-11-16T22:18:35.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af2b25700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0663d0 con 0x7f8aec0670c0 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af10c2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f8aec1488a0 0x7f8aec148d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54262/0 (socket says 172.21.15.185:54262) 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8af10c2700 1 -- 172.21.15.185:0/1336995233 learned_addr learned my addr 172.21.15.185:0/1336995233 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aebfff700 1 --2- 172.21.15.185:0/1336995233 >> v2:172.21.15.96:3300/0 conn(0x7f8aec0670c0 0x7f8aec067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aeb7fe700 1 -- 172.21.15.185:0/1336995233 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579016332 0 0) 0x7f8aec05c0b0 con 0x7f8aec1488a0 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aeb7fe700 1 -- 172.21.15.185:0/1336995233 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad0003240 con 0x7f8aec1488a0 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aebfff700 1 -- 172.21.15.185:0/1336995233 >> v1:172.21.15.185:6789/0 conn(0x7f8aec1488a0 legacy=0x7f8aec148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aebfff700 1 -- 172.21.15.185:0/1336995233 >> v1:172.21.15.96:6789/0 conn(0x7f8aec0676a0 legacy=0x7f8aec09c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.375+0000 7f8aebfff700 1 -- 172.21.15.185:0/1336995233 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec066670 con 0x7f8aec0670c0 2020-11-16T22:18:35.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8aebfff700 1 --2- 172.21.15.185:0/1336995233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0670c0 0x7f8aec067a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8ad800b9a0 tx=0x7f8ad8006df0).ready entity=mon.0 client_cookie=1e050206bd7c8b78 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8aeb7fe700 1 -- 172.21.15.185:0/1336995233 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80097b0 con 0x7f8aec0670c0 2020-11-16T22:18:35.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8aeb7fe700 1 -- 172.21.15.185:0/1336995233 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8009910 con 0x7f8aec0670c0 2020-11-16T22:18:35.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8aeb7fe700 1 -- 172.21.15.185:0/1336995233 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8009c90 con 0x7f8aec0670c0 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/1336995233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0670c0 msgr2=0x7f8aec067a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 --2- 172.21.15.185:0/1336995233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0670c0 0x7f8aec067a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8ad800b9a0 tx=0x7f8ad8006df0).stop 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/1336995233 shutdown_connections 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 --2- 172.21.15.185:0/1336995233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0670c0 0x7f8aec067a80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/1336995233 >> 172.21.15.185:0/1336995233 conn(0x7f8aec0b4410 msgr2=0x7f8aec060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/1336995233 shutdown_connections 2020-11-16T22:18:35.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/1336995233 wait complete. 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 Processor -- start 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- start start 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8aec0670c0 0x7f8aec09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0676a0 0x7f8aec09b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1499a0 con 0x7f8aec0676a0 2020-11-16T22:18:35.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec149bd0 con 0x7f8aec0670c0 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec099750 con 0x7f8aec1488a0 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af08c1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8aec0670c0 0x7f8aec09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54274/0 (socket says 172.21.15.185:54274) 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8aebfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0676a0 0x7f8aec09b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 -- 172.21.15.185:0/2705632220 learned_addr learned my addr 172.21.15.185:0/2705632220 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:35.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 -- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8aec0670c0 msgr2=0x7f8aec09b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8aec0670c0 0x7f8aec09b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 -- 172.21.15.185:0/2705632220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0676a0 msgr2=0x7f8aec09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:35.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0676a0 0x7f8aec09b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:35.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 -- 172.21.15.185:0/2705632220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec099ac0 con 0x7f8aec1488a0 2020-11-16T22:18:35.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af10c2700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f8ae400ca60 tx=0x7f8ae4007db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4005bd0 con 0x7f8aec1488a0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.379+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec099d60 con 0x7f8aec1488a0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4005d30 con 0x7f8aec1488a0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec09a190 con 0x7f8aec1488a0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4005ea0 con 0x7f8aec1488a0 2020-11-16T22:18:35.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec097c40 con 0x7f8aec1488a0 2020-11-16T22:18:35.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8ae4010030 con 0x7f8aec1488a0 2020-11-16T22:18:35.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8ae8ff9700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 0x7f8ac4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:35.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ae406dd90 con 0x7f8aec1488a0 2020-11-16T22:18:35.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8af08c1700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 0x7f8ac4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:35.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.383+0000 7f8af08c1700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 0x7f8ac4057520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8ae0003300 tx=0x7f8ae0007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:35.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.391+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8aec097c40 con 0x7f8aec1488a0 2020-11-16T22:18:35.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:35.595+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) v1 -- 0x7f8aec097c40 con 0x7f8aec1488a0 2020-11-16T22:18:36.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8ae8ff9700 1 -- 172.21.15.185:0/2705632220 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v234) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8aec097c40 con 0x7f8aec1488a0 2020-11-16T22:18:36.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 msgr2=0x7f8ac4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 0x7f8ac4057520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8ae0003300 tx=0x7f8ae0007b90).stop 2020-11-16T22:18:36.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 msgr2=0x7f8aec141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f8ae400ca60 tx=0x7f8ae4007db0).stop 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 shutdown_connections 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8ac4055060 0x7f8ac4057520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8aec0670c0 0x7f8aec09b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8aec1488a0 0x7f8aec141840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 --2- 172.21.15.185:0/2705632220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8aec0676a0 0x7f8aec09b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 >> 172.21.15.185:0/2705632220 conn(0x7f8aec0b4410 msgr2=0x7f8aec0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:36.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 shutdown_connections 2020-11-16T22:18:36.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.223+0000 7f8af2b25700 1 -- 172.21.15.185:0/2705632220 wait complete. 2020-11-16T22:18:36.220 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:18:36.221 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:18:36.221 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2020-11-16T22:18:36.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 Processor -- start 2020-11-16T22:18:36.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 -- start start 2020-11-16T22:18:36.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff4e40682b0 0x7ff4e4143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e4067510 con 0x7ff4e4067070 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e40677b0 con 0x7ff4e40682b0 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e8dde700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40b17a0 con 0x7ff4e40b7470 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54280/0 (socket says 172.21.15.185:54280) 2020-11-16T22:18:36.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 -- 172.21.15.185:0/1198281151 learned_addr learned my addr 172.21.15.185:0/1198281151 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e3fff700 1 --2- 172.21.15.185:0/1198281151 >> v2:172.21.15.96:3300/0 conn(0x7ff4e40682b0 0x7ff4e4143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 -- 172.21.15.185:0/1198281151 >> v2:172.21.15.96:3300/0 conn(0x7ff4e40682b0 msgr2=0x7ff4e4143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 --2- 172.21.15.185:0/1198281151 >> v2:172.21.15.96:3300/0 conn(0x7ff4e40682b0 0x7ff4e4143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 -- 172.21.15.185:0/1198281151 >> v1:172.21.15.96:6789/0 conn(0x7ff4e4067070 legacy=0x7ff4e40b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.367+0000 7ff4e37fe700 1 -- 172.21.15.185:0/1198281151 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40b1a40 con 0x7ff4e40b7470 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e37fe700 1 --2- 172.21.15.185:0/1198281151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7ff4d4006fd0 tx=0x7ff4d4006eb0).ready entity=mon.2 client_cookie=f0b31ea73c7a1a4e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:36.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e2ffd700 1 -- 172.21.15.185:0/1198281151 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40098d0 con 0x7ff4e40b7470 2020-11-16T22:18:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e2ffd700 1 -- 172.21.15.185:0/1198281151 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d4009a30 con 0x7ff4e40b7470 2020-11-16T22:18:36.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e2ffd700 1 -- 172.21.15.185:0/1198281151 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4009db0 con 0x7ff4e40b7470 2020-11-16T22:18:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- 172.21.15.185:0/1198281151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 msgr2=0x7ff4e40b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/1198281151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7ff4d4006fd0 tx=0x7ff4d4006eb0).stop 2020-11-16T22:18:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- 172.21.15.185:0/1198281151 shutdown_connections 2020-11-16T22:18:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/1198281151 >> v2:172.21.15.96:3300/0 conn(0x7ff4e40682b0 0x7ff4e4143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/1198281151 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40b7470 0x7ff4e40b7850 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- 172.21.15.185:0/1198281151 >> 172.21.15.185:0/1198281151 conn(0x7ff4e40637b0 msgr2=0x7ff4e40b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- 172.21.15.185:0/1198281151 shutdown_connections 2020-11-16T22:18:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- 172.21.15.185:0/1198281151 wait complete. 2020-11-16T22:18:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 Processor -- start 2020-11-16T22:18:36.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.371+0000 7ff4e8dde700 1 -- start start 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e4067070 0x7ff4e4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 0x7ff4e40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e4153600 con 0x7ff4e4067070 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e413aaa0 con 0x7ff4e40b7470 2020-11-16T22:18:36.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e413d720 con 0x7ff4e40682b0 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 0x7ff4e40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54284/0 (socket says 172.21.15.185:54284) 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 0x7ff4e40acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60844/0 (socket says 172.21.15.185:60844) 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 -- 172.21.15.185:0/4015647792 learned_addr learned my addr 172.21.15.185:0/4015647792 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e37fe700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e4067070 0x7ff4e4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 -- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 msgr2=0x7ff4e40acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 0x7ff4e40acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 -- 172.21.15.185:0/4015647792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e4067070 msgr2=0x7ff4e4153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e4067070 0x7ff4e4153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 -- 172.21.15.185:0/4015647792 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e413d9c0 con 0x7ff4e40682b0 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4daffd700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7ff4cc00c710 tx=0x7ff4cc007b60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc007020 con 0x7ff4e40682b0 2020-11-16T22:18:36.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc007180 con 0x7ff4e40682b0 2020-11-16T22:18:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e413bf50 con 0x7ff4e40682b0 2020-11-16T22:18:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0064f0 con 0x7ff4e40682b0 2020-11-16T22:18:36.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.375+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e413c3f0 con 0x7ff4e40682b0 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e4143e30 con 0x7ff4e40682b0 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff4cc010070 con 0x7ff4e40682b0 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4dbfff700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 0x7ff4c0057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4cc06d0a0 con 0x7ff4e40682b0 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4e37fe700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 0x7ff4c0057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:36.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.379+0000 7ff4e37fe700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 0x7ff4c0057570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7ff4d400b1b0 tx=0x7ff4d4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:36.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.387+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff4cc043270 con 0x7ff4e40682b0 2020-11-16T22:18:36.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:36.591+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7ff4e413c0e0 con 0x7ff4e40682b0 2020-11-16T22:18:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.223+0000 7ff4dbfff700 1 -- 172.21.15.185:0/4015647792 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v235) v1 ==== 144+0+0 (secure 0 0 0) 0x7ff4cc046890 con 0x7ff4e40682b0 2020-11-16T22:18:37.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 msgr2=0x7ff4c0057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 0x7ff4c0057570 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7ff4d400b1b0 tx=0x7ff4d4015000).stop 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 msgr2=0x7ff4e4142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7ff4cc00c710 tx=0x7ff4cc007b60).stop 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 shutdown_connections 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff4c00550b0 0x7ff4c0057570 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff4e40b7470 0x7ff4e40acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e4067070 0x7ff4e4153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 --2- 172.21.15.185:0/4015647792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff4e40682b0 0x7ff4e4142780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 >> 172.21.15.185:0/4015647792 conn(0x7ff4e40637b0 msgr2=0x7ff4e4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:37.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 shutdown_connections 2020-11-16T22:18:37.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.227+0000 7ff4e8dde700 1 -- 172.21.15.185:0/4015647792 wait complete. 2020-11-16T22:18:37.229 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:18:37.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 Processor -- start 2020-11-16T22:18:37.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 -- start start 2020-11-16T22:18:37.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 0x7f61540b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6154143f80 con 0x7f6154060bf0 2020-11-16T22:18:37.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6154067ca0 con 0x7f61540b7510 2020-11-16T22:18:37.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6153fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6154067f30 con 0x7f61540b3500 2020-11-16T22:18:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f61527fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6152ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 0x7f61540b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f61527fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54294/0 (socket says 172.21.15.185:54294) 2020-11-16T22:18:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f6152ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 0x7f61540b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60848/0 (socket says 172.21.15.185:60848) 2020-11-16T22:18:37.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.379+0000 7f61527fc700 1 -- 172.21.15.185:0/599194670 learned_addr learned my addr 172.21.15.185:0/599194670 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:37.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f61527fc700 1 -- 172.21.15.185:0/599194670 >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 msgr2=0x7f61540b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f61527fc700 1 --2- 172.21.15.185:0/599194670 >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 0x7f61540b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f61527fc700 1 -- 172.21.15.185:0/599194670 >> v1:172.21.15.96:6789/0 conn(0x7f6154060bf0 legacy=0x7f61540667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f61527fc700 1 -- 172.21.15.185:0/599194670 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540681d0 con 0x7f61540b3500 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f61527fc700 1 --2- 172.21.15.185:0/599194670 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f6148006fd0 tx=0x7f6148006eb0).ready entity=mon.2 client_cookie=e4aea66779a6cd20 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6151ffb700 1 -- 172.21.15.185:0/599194670 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148009810 con 0x7f61540b3500 2020-11-16T22:18:37.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6151ffb700 1 -- 172.21.15.185:0/599194670 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148009970 con 0x7f61540b3500 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6151ffb700 1 -- 172.21.15.185:0/599194670 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148009cf0 con 0x7f61540b3500 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- 172.21.15.185:0/599194670 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 msgr2=0x7f61540677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- 172.21.15.185:0/599194670 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f6148006fd0 tx=0x7f6148006eb0).stop 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- 172.21.15.185:0/599194670 shutdown_connections 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- 172.21.15.185:0/599194670 >> v2:172.21.15.185:3300/0 conn(0x7f61540b7510 0x7f61540b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- 172.21.15.185:0/599194670 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f61540b3500 0x7f61540677b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- 172.21.15.185:0/599194670 >> 172.21.15.185:0/599194670 conn(0x7f6154064b90 msgr2=0x7f6154063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- 172.21.15.185:0/599194670 shutdown_connections 2020-11-16T22:18:37.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- 172.21.15.185:0/599194670 wait complete. 2020-11-16T22:18:37.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 Processor -- start 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- start start 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 0x7f61540ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 0x7f6154153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 0x7f6154147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6154153180 con 0x7f6154153d30 2020-11-16T22:18:37.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61541541b0 con 0x7f61540b7510 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.383+0000 7f6153fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540ae3e0 con 0x7f6154060bf0 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f61527fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 0x7f6154153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f61527fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 0x7f6154153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60856/0 (socket says 172.21.15.185:60856) 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f61527fc700 1 -- 172.21.15.185:0/1895044185 learned_addr learned my addr 172.21.15.185:0/1895044185 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 0x7f61540ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f61537fe700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 0x7f6154147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 -- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 msgr2=0x7f6154153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 0x7f6154153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 -- 172.21.15.185:0/1895044185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 msgr2=0x7f6154147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 0x7f6154147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 -- 172.21.15.185:0/1895044185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540adfe0 con 0x7f6154060bf0 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f61537fe700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 0x7f6154147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6152ffd700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 0x7f61540ad060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f614400ec00 tx=0x7f6144006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400b2c0 con 0x7f6154060bf0 2020-11-16T22:18:37.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614400c8e0 con 0x7f6154060bf0 2020-11-16T22:18:37.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61541533a0 con 0x7f6154060bf0 2020-11-16T22:18:37.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400cc80 con 0x7f6154060bf0 2020-11-16T22:18:37.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61540ad610 con 0x7f6154060bf0 2020-11-16T22:18:37.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6154153580 con 0x7f6154060bf0 2020-11-16T22:18:37.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.387+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f614401c460 con 0x7f6154060bf0 2020-11-16T22:18:37.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.391+0000 7f613affd700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 0x7f6130057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.391+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f614406e0a0 con 0x7f6154060bf0 2020-11-16T22:18:37.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.391+0000 7f61527fc700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 0x7f6130057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.391+0000 7f61527fc700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 0x7f6130057520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f614800b1b0 tx=0x7f6148015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.395+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6154153580 con 0x7f6154060bf0 2020-11-16T22:18:37.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.555+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f6154153530 con 0x7f6154060bf0 2020-11-16T22:18:37.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.563+0000 7f613affd700 1 -- 172.21.15.185:0/1895044185 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61440442a0 con 0x7f6154060bf0 2020-11-16T22:18:37.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.563+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 msgr2=0x7f6130057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.563+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 0x7f6130057520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f614800b1b0 tx=0x7f6148015000).stop 2020-11-16T22:18:37.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.563+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 msgr2=0x7f61540ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.563+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 0x7f61540ad060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f614400ec00 tx=0x7f6144006df0).stop 2020-11-16T22:18:37.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 shutdown_connections 2020-11-16T22:18:37.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6130055060 0x7f6130057520 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6154153d30 0x7f6154147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6154060bf0 0x7f61540ad060 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 --2- 172.21.15.185:0/1895044185 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f61540b7510 0x7f6154153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 >> 172.21.15.185:0/1895044185 conn(0x7f6154064b90 msgr2=0x7f615413d400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:37.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 shutdown_connections 2020-11-16T22:18:37.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.567+0000 7f6153fff700 1 -- 172.21.15.185:0/1895044185 wait complete. 2020-11-16T22:18:37.563 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:18:37.564 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:37.602 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:37.620 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:18:37.635 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:18:37.636 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:18:37.636 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:37.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.823+0000 7f06ae104700 1 Processor -- start 2020-11-16T22:18:37.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.823+0000 7f06ae104700 1 -- start start 2020-11-16T22:18:37.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f06a8062900 0x7f06a8062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 0x7f06a80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a8067930 con 0x7f06a81446c0 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80632e0 con 0x7f06a8062900 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8066ae0 con 0x7f06a8067de0 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a6ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f06a81446c0 0x7f06a8067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54300/0 (socket says 172.21.15.185:54300) 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a6ffd700 1 -- 172.21.15.185:0/1280501230 learned_addr learned my addr 172.21.15.185:0/1280501230 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1280501230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 0x7f06a80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a77fe700 1 --2- 172.21.15.185:0/1280501230 >> v2:172.21.15.96:3300/0 conn(0x7f06a8062900 0x7f06a8062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 -- 172.21.15.185:0/1280501230 >> v2:172.21.15.96:3300/0 conn(0x7f06a8062900 msgr2=0x7f06a8062ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:37.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1280501230 >> v2:172.21.15.96:3300/0 conn(0x7f06a8062900 0x7f06a8062ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 -- 172.21.15.185:0/1280501230 >> v1:172.21.15.185:6789/0 conn(0x7f06a81446c0 legacy=0x7f06a8067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 -- 172.21.15.185:0/1280501230 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8066d80 con 0x7f06a8067de0 2020-11-16T22:18:37.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a67fc700 1 -- 172.21.15.185:0/1280501230 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498572971 0 0) 0x7f06a8067930 con 0x7f06a81446c0 2020-11-16T22:18:37.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1280501230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 0x7f06a80b1840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f06a0009c90 tx=0x7f06a00091d0).ready entity=mon.2 client_cookie=9b992a5cdaf64108 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a67fc700 1 -- 172.21.15.185:0/1280501230 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000bbc0 con 0x7f06a8067de0 2020-11-16T22:18:37.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a67fc700 1 -- 172.21.15.185:0/1280501230 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000bd20 con 0x7f06a8067de0 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06a67fc700 1 -- 172.21.15.185:0/1280501230 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0005cb0 con 0x7f06a8067de0 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- 172.21.15.185:0/1280501230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 msgr2=0x7f06a80b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 --2- 172.21.15.185:0/1280501230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 0x7f06a80b1840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f06a0009c90 tx=0x7f06a00091d0).stop 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- 172.21.15.185:0/1280501230 shutdown_connections 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 --2- 172.21.15.185:0/1280501230 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8067de0 0x7f06a80b1840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 --2- 172.21.15.185:0/1280501230 >> v2:172.21.15.96:3300/0 conn(0x7f06a8062900 0x7f06a8062ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- 172.21.15.185:0/1280501230 >> 172.21.15.185:0/1280501230 conn(0x7f06a80b4410 msgr2=0x7f06a8060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:37.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- 172.21.15.185:0/1280501230 shutdown_connections 2020-11-16T22:18:37.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.827+0000 7f06ae104700 1 -- 172.21.15.185:0/1280501230 wait complete. 2020-11-16T22:18:37.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 Processor -- start 2020-11-16T22:18:37.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- start start 2020-11-16T22:18:37.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 0x7f06a81415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06a8067de0 0x7f06a80ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80ac840 con 0x7f06a8067de0 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80acd10 con 0x7f06a81446c0 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80ad040 con 0x7f06a8062900 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 0x7f06a81415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60870/0 (socket says 172.21.15.185:60870) 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 -- 172.21.15.185:0/1904413923 learned_addr learned my addr 172.21.15.185:0/1904413923 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a6ffd700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06a8067de0 0x7f06a80ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 -- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 msgr2=0x7f06a81415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 0x7f06a81415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 -- 172.21.15.185:0/1904413923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06a8067de0 msgr2=0x7f06a80ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:37.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06a8067de0 0x7f06a80ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 -- 172.21.15.185:0/1904413923 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80aca70 con 0x7f06a81446c0 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a77fe700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 0x7f06a81415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06a7fff700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f06a0006000 tx=0x7f06a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000bbc0 con 0x7f06a81446c0 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000be90 con 0x7f06a81446c0 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a80ac410 con 0x7f06a81446c0 2020-11-16T22:18:37.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011860 con 0x7f06a81446c0 2020-11-16T22:18:37.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.831+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a80aeb00 con 0x7f06a81446c0 2020-11-16T22:18:37.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a805d4e0 con 0x7f06a81446c0 2020-11-16T22:18:37.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f06a0024030 con 0x7f06a81446c0 2020-11-16T22:18:37.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06877fe700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 0x7f0680057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:37.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f06a006f9b0 con 0x7f06a81446c0 2020-11-16T22:18:37.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06a77fe700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 0x7f0680057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:37.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.835+0000 7f06a77fe700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 0x7f0680057520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f069800da90 tx=0x7f0698008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:37.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:37.843+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f06a0045bb0 con 0x7f06a81446c0 2020-11-16T22:18:38.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.047+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a8061e90 con 0x7f06a81446c0 2020-11-16T22:18:38.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.047+0000 7f06877fe700 1 -- 172.21.15.185:0/1904413923 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f06a0022030 con 0x7f06a81446c0 2020-11-16T22:18:38.029 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:38.029 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:38.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 msgr2=0x7f0680057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 0x7f0680057520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f069800da90 tx=0x7f0698008040).stop 2020-11-16T22:18:38.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 msgr2=0x7f06a80b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f06a0006000 tx=0x7f06a0015000).stop 2020-11-16T22:18:38.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 shutdown_connections 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0680055060 0x7f0680057520 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06a81446c0 0x7f06a80b0240 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06a8062900 0x7f06a81415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 --2- 172.21.15.185:0/1904413923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06a8067de0 0x7f06a80ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 >> 172.21.15.185:0/1904413923 conn(0x7f06a80b4410 msgr2=0x7f06a8136710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 shutdown_connections 2020-11-16T22:18:38.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.051+0000 7f06ae104700 1 -- 172.21.15.185:0/1904413923 wait complete. 2020-11-16T22:18:38.034 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:38.048 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:38.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 Processor -- start 2020-11-16T22:18:38.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 -- start start 2020-11-16T22:18:38.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f67c00b76c0 0x7f67c00b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c0147f90 con 0x7f67c00b72e0 2020-11-16T22:18:38.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c0148460 con 0x7f67c00b64b0 2020-11-16T22:18:38.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00af020 con 0x7f67c00b76c0 2020-11-16T22:18:38.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67be7fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f67c00b64b0 0x7f67c013dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54318/0 (socket says 172.21.15.185:54318) 2020-11-16T22:18:38.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67be7fc700 1 -- 172.21.15.185:0/2227658556 learned_addr learned my addr 172.21.15.185:0/2227658556 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:38.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bf7fe700 1 --2- 172.21.15.185:0/2227658556 >> v2:172.21.15.96:3300/0 conn(0x7f67c00b76c0 0x7f67c00b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bdffb700 1 -- 172.21.15.185:0/2227658556 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229364650 0 0) 0x7f67c0148460 con 0x7f67c00b64b0 2020-11-16T22:18:38.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bdffb700 1 -- 172.21.15.185:0/2227658556 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003240 con 0x7f67c00b64b0 2020-11-16T22:18:38.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bf7fe700 1 -- 172.21.15.185:0/2227658556 >> v1:172.21.15.185:6789/0 conn(0x7f67c00b64b0 legacy=0x7f67c013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bf7fe700 1 -- 172.21.15.185:0/2227658556 >> v1:172.21.15.96:6789/0 conn(0x7f67c00b72e0 legacy=0x7f67c0144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bf7fe700 1 -- 172.21.15.185:0/2227658556 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00b4810 con 0x7f67c00b76c0 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bf7fe700 1 --2- 172.21.15.185:0/2227658556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b76c0 0x7f67c00b6aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f67b800b9a0 tx=0x7f67b8006df0).ready entity=mon.0 client_cookie=36b3234f7ada9ee3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.183+0000 7f67bdffb700 1 -- 172.21.15.185:0/2227658556 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80097b0 con 0x7f67c00b76c0 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bdffb700 1 -- 172.21.15.185:0/2227658556 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8009910 con 0x7f67c00b76c0 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bdffb700 1 -- 172.21.15.185:0/2227658556 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8009c90 con 0x7f67c00b76c0 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- 172.21.15.185:0/2227658556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b76c0 msgr2=0x7f67c00b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 --2- 172.21.15.185:0/2227658556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b76c0 0x7f67c00b6aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f67b800b9a0 tx=0x7f67b8006df0).stop 2020-11-16T22:18:38.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- 172.21.15.185:0/2227658556 shutdown_connections 2020-11-16T22:18:38.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 --2- 172.21.15.185:0/2227658556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b76c0 0x7f67c00b6aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- 172.21.15.185:0/2227658556 >> 172.21.15.185:0/2227658556 conn(0x7f67c0067860 msgr2=0x7f67c0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- 172.21.15.185:0/2227658556 shutdown_connections 2020-11-16T22:18:38.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- 172.21.15.185:0/2227658556 wait complete. 2020-11-16T22:18:38.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 Processor -- start 2020-11-16T22:18:38.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.187+0000 7f67bffff700 1 -- start start 2020-11-16T22:18:38.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b72e0 0x7f67c01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67c00b76c0 0x7f67c00b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00b0d50 con 0x7f67c00b72e0 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c013ee80 con 0x7f67c00b76c0 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c013f990 con 0x7f67c00b64b0 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54326/0 (socket says 172.21.15.185:54326) 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 -- 172.21.15.185:0/1460474646 learned_addr learned my addr 172.21.15.185:0/1460474646 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bf7fe700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67c00b76c0 0x7f67c00b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67be7fc700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b72e0 0x7f67c01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 -- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67c00b76c0 msgr2=0x7f67c00b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67c00b76c0 0x7f67c00b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 -- 172.21.15.185:0/1460474646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b72e0 msgr2=0x7f67c01571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b72e0 0x7f67c01571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 -- 172.21.15.185:0/1460474646 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c013f0b0 con 0x7f67c00b64b0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67beffd700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f67b400c710 tx=0x7f67b4007c00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400e490 con 0x7f67c00b64b0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400e5f0 con 0x7f67c00b64b0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c013f350 con 0x7f67c00b64b0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400e990 con 0x7f67c00b64b0 2020-11-16T22:18:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c00b0910 con 0x7f67c00b64b0 2020-11-16T22:18:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f67b4019030 con 0x7f67c00b64b0 2020-11-16T22:18:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c013fb20 con 0x7f67c00b64b0 2020-11-16T22:18:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.191+0000 7f67a6ffd700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 0x7f679c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.195+0000 7f67be7fc700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 0x7f679c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.195+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67b406d190 con 0x7f67c00b64b0 2020-11-16T22:18:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.195+0000 7f67be7fc700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 0x7f679c057520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f67c0147820 tx=0x7f67b0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.199+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f67c013fb20 con 0x7f67c00b64b0 2020-11-16T22:18:38.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c005d200 con 0x7f67c00b64b0 2020-11-16T22:18:38.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67a6ffd700 1 -- 172.21.15.185:0/1460474646 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16828 (secure 0 0 0) 0x7f67b40251d0 con 0x7f67c00b64b0 2020-11-16T22:18:38.333 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:38.334 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:39.874438+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:38.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 msgr2=0x7f679c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 0x7f679c057520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f67c0147820 tx=0x7f67b0009040).stop 2020-11-16T22:18:38.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 msgr2=0x7f67c013e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f67b400c710 tx=0x7f67b4007c00).stop 2020-11-16T22:18:38.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 shutdown_connections 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f679c055060 0x7f679c057520 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f67c00b76c0 0x7f67c00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f67c00b64b0 0x7f67c013e6c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 --2- 172.21.15.185:0/1460474646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c00b72e0 0x7f67c01571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 >> 172.21.15.185:0/1460474646 conn(0x7f67c0067860 msgr2=0x7f67c0067c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.351+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 shutdown_connections 2020-11-16T22:18:38.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.355+0000 7f67bffff700 1 -- 172.21.15.185:0/1460474646 wait complete. 2020-11-16T22:18:38.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:17:36.945221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:17:37.680544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:18:38.349 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:18:38.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.495+0000 7f3a1bfff700 1 Processor -- start 2020-11-16T22:18:38.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.495+0000 7f3a1bfff700 1 -- start start 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 0x7f3a1c067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c0b3f50 con 0x7f3a1c0b7690 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b41f0 con 0x7f3a1c0b1f20 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b4490 con 0x7f3a1c141690 2020-11-16T22:18:38.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 0x7f3a1c067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1affd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3a1c0b7690 0x7f3a1c144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54328/0 (socket says 172.21.15.185:54328) 2020-11-16T22:18:38.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60892/0 (socket says 172.21.15.185:60892) 2020-11-16T22:18:38.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1affd700 1 -- 172.21.15.185:0/2512999191 learned_addr learned my addr 172.21.15.185:0/2512999191 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:38.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/2512999191 >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 msgr2=0x7f3a1c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:38.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 --2- 172.21.15.185:0/2512999191 >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/2512999191 >> v1:172.21.15.185:6789/0 conn(0x7f3a1c0b7690 legacy=0x7f3a1c144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/2512999191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0b4e20 con 0x7f3a1c141690 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1b7fe700 1 --2- 172.21.15.185:0/2512999191 >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1a7fc700 1 --2- 172.21.15.185:0/2512999191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 0x7f3a1c067420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f3a0400b9a0 tx=0x7f3a04006df0).ready entity=mon.2 client_cookie=150b41d257f01596 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a19ffb700 1 -- 172.21.15.185:0/2512999191 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040097b0 con 0x7f3a1c141690 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a19ffb700 1 -- 172.21.15.185:0/2512999191 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04009910 con 0x7f3a1c141690 2020-11-16T22:18:38.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a19ffb700 1 -- 172.21.15.185:0/2512999191 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04009c90 con 0x7f3a1c141690 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- 172.21.15.185:0/2512999191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 msgr2=0x7f3a1c067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/2512999191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 0x7f3a1c067420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f3a0400b9a0 tx=0x7f3a04006df0).stop 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- 172.21.15.185:0/2512999191 shutdown_connections 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/2512999191 >> v2:172.21.15.185:3300/0 conn(0x7f3a1c0b1f20 0x7f3a1c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/2512999191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c141690 0x7f3a1c067420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- 172.21.15.185:0/2512999191 >> 172.21.15.185:0/2512999191 conn(0x7f3a1c064b90 msgr2=0x7f3a1c063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- 172.21.15.185:0/2512999191 shutdown_connections 2020-11-16T22:18:38.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.499+0000 7f3a1bfff700 1 -- 172.21.15.185:0/2512999191 wait complete. 2020-11-16T22:18:38.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.503+0000 7f3a1bfff700 1 Processor -- start 2020-11-16T22:18:38.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.503+0000 7f3a1bfff700 1 -- start start 2020-11-16T22:18:38.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a1c0b1f20 0x7f3a1c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3a1c141690 0x7f3a1c0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0af180 con 0x7f3a1c0b1f20 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b1540 con 0x7f3a1c141690 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b0640 con 0x7f3a1c0b7690 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54342/0 (socket says 172.21.15.185:54342) 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/792403344 learned_addr learned my addr 172.21.15.185:0/792403344 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1b7fe700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3a1c141690 0x7f3a1c0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1affd700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a1c0b1f20 0x7f3a1c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3a1c141690 msgr2=0x7f3a1c0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3a1c141690 0x7f3a1c0ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/792403344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a1c0b1f20 msgr2=0x7f3a1c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a1c0b1f20 0x7f3a1c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 -- 172.21.15.185:0/792403344 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0b08e0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a1a7fc700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3a04006e30 tx=0x7f3a04015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.507+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04011d50 con 0x7f3a1c0b7690 2020-11-16T22:18:38.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0401e460 con 0x7f3a1c0b7690 2020-11-16T22:18:38.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c0b1270 con 0x7f3a1c0b7690 2020-11-16T22:18:38.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0401e800 con 0x7f3a1c0b7690 2020-11-16T22:18:38.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c0b66a0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c0601c0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3a04025030 con 0x7f3a1c0b7690 2020-11-16T22:18:38.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a12ffd700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 0x7f39ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a1affd700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 0x7f39ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a0406f2c0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.511+0000 7f3a1affd700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 0x7f39ec057520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3a0c007fd0 tx=0x7f3a0c007f00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.519+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3a1c0601c0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.671+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3a1c0601c0 con 0x7f3a1c0b7690 2020-11-16T22:18:38.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.671+0000 7f3a12ffd700 1 -- 172.21.15.185:0/792403344 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3a04023020 con 0x7f3a1c0b7690 2020-11-16T22:18:38.655 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:18:38.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 msgr2=0x7f39ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 0x7f39ec057520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3a0c007fd0 tx=0x7f3a0c007f00).stop 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 msgr2=0x7f3a1c1314d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3a04006e30 tx=0x7f3a04015000).stop 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 shutdown_connections 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f39ec055060 0x7f39ec057520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3a1c141690 0x7f3a1c0ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a1c0b1f20 0x7f3a1c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 --2- 172.21.15.185:0/792403344 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3a1c0b7690 0x7f3a1c1314d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 >> 172.21.15.185:0/792403344 conn(0x7f3a1c064b90 msgr2=0x7f3a1c140cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 shutdown_connections 2020-11-16T22:18:38.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.675+0000 7f3a1bfff700 1 -- 172.21.15.185:0/792403344 wait complete. 2020-11-16T22:18:38.669 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:18:38.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.815+0000 7f0c1aafe700 1 Processor -- start 2020-11-16T22:18:38.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.815+0000 7f0c1aafe700 1 -- start start 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c140b3020 0x7f0c140b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0c140abef0 0x7f0c14145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140acdf0 con 0x7f0c14067890 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140ad2c0 con 0x7f0c140abef0 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140ad020 con 0x7f0c140b3020 2020-11-16T22:18:38.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c140b3020 0x7f0c140b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c140b3020 0x7f0c140b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60906/0 (socket says 172.21.15.185:60906) 2020-11-16T22:18:38.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 -- 172.21.15.185:0/2767584616 learned_addr learned my addr 172.21.15.185:0/2767584616 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:38.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c192fb700 1 --2- 172.21.15.185:0/2767584616 >> v2:172.21.15.96:3300/0 conn(0x7f0c140abef0 0x7f0c14145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 -- 172.21.15.185:0/2767584616 >> v2:172.21.15.96:3300/0 conn(0x7f0c140abef0 msgr2=0x7f0c14145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:38.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 --2- 172.21.15.185:0/2767584616 >> v2:172.21.15.96:3300/0 conn(0x7f0c140abef0 0x7f0c14145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 -- 172.21.15.185:0/2767584616 >> v1:172.21.15.96:6789/0 conn(0x7f0c14067890 legacy=0x7f0c14067ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 -- 172.21.15.185:0/2767584616 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14145460 con 0x7f0c140b3020 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c19afc700 1 --2- 172.21.15.185:0/2767584616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c140b17c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0c14064930 tx=0x7f0c04006eb0).ready entity=mon.1 client_cookie=19d0364b01f86ecf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c18afa700 1 -- 172.21.15.185:0/2767584616 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04009810 con 0x7f0c140b3020 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c18afa700 1 -- 172.21.15.185:0/2767584616 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04009970 con 0x7f0c140b3020 2020-11-16T22:18:38.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c18afa700 1 -- 172.21.15.185:0/2767584616 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04009cf0 con 0x7f0c140b3020 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- 172.21.15.185:0/2767584616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 msgr2=0x7f0c140b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/2767584616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c140b17c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0c14064930 tx=0x7f0c04006eb0).stop 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- 172.21.15.185:0/2767584616 shutdown_connections 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/2767584616 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c140b17c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/2767584616 >> v2:172.21.15.96:3300/0 conn(0x7f0c140abef0 0x7f0c14145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- 172.21.15.185:0/2767584616 >> 172.21.15.185:0/2767584616 conn(0x7f0c14060130 msgr2=0x7f0c14060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- 172.21.15.185:0/2767584616 shutdown_connections 2020-11-16T22:18:38.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.819+0000 7f0c1aafe700 1 -- 172.21.15.185:0/2767584616 wait complete. 2020-11-16T22:18:38.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.823+0000 7f0c1aafe700 1 Processor -- start 2020-11-16T22:18:38.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.823+0000 7f0c1aafe700 1 -- start start 2020-11-16T22:18:38.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.823+0000 7f0c1aafe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.823+0000 7f0c1aafe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c140abef0 0x7f0c1413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.823+0000 7f0c1aafe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c14140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141511b0 con 0x7f0c140abef0 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14151450 con 0x7f0c140b3020 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141516f0 con 0x7f0c14067890 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54354/0 (socket says 172.21.15.185:54354) 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 -- 172.21.15.185:0/3118739024 learned_addr learned my addr 172.21.15.185:0/3118739024 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1a2fd700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c14140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c192fb700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c140abef0 0x7f0c1413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 -- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 msgr2=0x7f0c14140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c14140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 -- 172.21.15.185:0/3118739024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c140abef0 msgr2=0x7f0c1413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c140abef0 0x7f0c1413fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 -- 172.21.15.185:0/3118739024 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14151990 con 0x7f0c14067890 2020-11-16T22:18:38.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c19afc700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f0c0400beb0 tx=0x7f0c04015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04011d50 con 0x7f0c14067890 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0401e460 con 0x7f0c14067890 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c14147ba0 con 0x7f0c14067890 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0401e800 con 0x7f0c14067890 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c140ae110 con 0x7f0c14067890 2020-11-16T22:18:38.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c140ae2a0 con 0x7f0c14067890 2020-11-16T22:18:38.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.827+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0c04025030 con 0x7f0c14067890 2020-11-16T22:18:38.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.831+0000 7f0c01ffb700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 0x7f0bec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:38.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.831+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c0406f7e0 con 0x7f0c14067890 2020-11-16T22:18:38.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.831+0000 7f0c192fb700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 0x7f0bec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:38.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.831+0000 7f0c192fb700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 0x7f0bec057520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f0c140607f0 tx=0x7f0c10006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:38.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.835+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0c040459b0 con 0x7f0c14067890 2020-11-16T22:18:38.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.983+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c1405d200 con 0x7f0c14067890 2020-11-16T22:18:38.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.983+0000 7f0c01ffb700 1 -- 172.21.15.185:0/3118739024 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c04023020 con 0x7f0c14067890 2020-11-16T22:18:38.967 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:38.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 msgr2=0x7f0bec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 0x7f0bec057520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f0c140607f0 tx=0x7f0c10006df0).stop 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 msgr2=0x7f0c141476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f0c0400beb0 tx=0x7f0c04015000).stop 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 shutdown_connections 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0bec055060 0x7f0bec057520 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c140b3020 0x7f0c14140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c14067890 0x7f0c141476b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 --2- 172.21.15.185:0/3118739024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c140abef0 0x7f0c1413fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:38.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 >> 172.21.15.185:0/3118739024 conn(0x7f0c14060130 msgr2=0x7f0c140b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:38.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 shutdown_connections 2020-11-16T22:18:38.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:38.987+0000 7f0c1aafe700 1 -- 172.21.15.185:0/3118739024 wait complete. 2020-11-16T22:18:38.981 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:18:39.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 Processor -- start 2020-11-16T22:18:39.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 -- start start 2020-11-16T22:18:39.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06080acee0 0x7f06081408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0608068010 0x7f0608062f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06080684c0 con 0x7f06080ab5a0 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06081410e0 con 0x7f0608068010 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060f3fd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0608141380 con 0x7f06080acee0 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060c998700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f06080ab5a0 0x7f0608067900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54356/0 (socket says 172.21.15.185:54356) 2020-11-16T22:18:39.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060c998700 1 -- 172.21.15.185:0/1626971894 learned_addr learned my addr 172.21.15.185:0/1626971894 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 --2- 172.21.15.185:0/1626971894 >> v2:172.21.15.185:3300/0 conn(0x7f0608068010 0x7f0608062f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d199700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06080acee0 0x7f06081408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f05f7fff700 1 -- 172.21.15.185:0/1626971894 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903162814 0 0) 0x7f06040010d0 con 0x7f06080ab5a0 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f05f7fff700 1 -- 172.21.15.185:0/1626971894 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f0003240 con 0x7f06080ab5a0 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 -- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06080acee0 msgr2=0x7f06081408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:39.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06080acee0 0x7f06081408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 -- 172.21.15.185:0/1626971894 >> v1:172.21.15.185:6789/0 conn(0x7f06080ab5a0 legacy=0x7f0608067900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 -- 172.21.15.185:0/1626971894 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608141620 con 0x7f0608068010 2020-11-16T22:18:39.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.123+0000 7f060d99a700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0608068010 0x7f0608062f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0600009c90 tx=0x7f06000091d0).ready entity=mon.1 client_cookie=5e8b808fb71e4134 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f05f7fff700 1 -- 172.21.15.185:0/1626971894 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000b9c0 con 0x7f0608068010 2020-11-16T22:18:39.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f05f7fff700 1 -- 172.21.15.185:0/1626971894 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060000bb20 con 0x7f0608068010 2020-11-16T22:18:39.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f05f7fff700 1 -- 172.21.15.185:0/1626971894 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000bea0 con 0x7f0608068010 2020-11-16T22:18:39.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0608068010 msgr2=0x7f0608062f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0608068010 0x7f0608062f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0600009c90 tx=0x7f06000091d0).stop 2020-11-16T22:18:39.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- 172.21.15.185:0/1626971894 shutdown_connections 2020-11-16T22:18:39.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0608068010 0x7f0608062f40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- 172.21.15.185:0/1626971894 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f06080acee0 0x7f06081408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- 172.21.15.185:0/1626971894 >> 172.21.15.185:0/1626971894 conn(0x7f06080657b0 msgr2=0x7f0608065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:39.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- 172.21.15.185:0/1626971894 shutdown_connections 2020-11-16T22:18:39.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- 172.21.15.185:0/1626971894 wait complete. 2020-11-16T22:18:39.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 Processor -- start 2020-11-16T22:18:39.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- start start 2020-11-16T22:18:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06080ab5a0 0x7f0608063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 0x7f060813d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060813b120 con 0x7f06080acee0 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060813d0c0 con 0x7f06080ab5a0 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060f3fd700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0608151970 con 0x7f0608068010 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060d199700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060d199700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54368/0 (socket says 172.21.15.185:54368) 2020-11-16T22:18:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060d199700 1 -- 172.21.15.185:0/3558214638 learned_addr learned my addr 172.21.15.185:0/3558214638 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.127+0000 7f060c998700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06080ab5a0 0x7f0608063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d99a700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 0x7f060813d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 -- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06080ab5a0 msgr2=0x7f0608063000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06080ab5a0 0x7f0608063000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 -- 172.21.15.185:0/3558214638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 msgr2=0x7f060813d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 0x7f060813d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 -- 172.21.15.185:0/3558214638 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060809c020 con 0x7f0608068010 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d99a700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 0x7f060813d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060d199700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f05f800ebd0 tx=0x7f05f8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800c9b0 con 0x7f0608068010 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800cb10 con 0x7f0608068010 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060809c2c0 con 0x7f0608068010 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800ce90 con 0x7f0608068010 2020-11-16T22:18:39.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060809b4d0 con 0x7f0608068010 2020-11-16T22:18:39.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060805d4e0 con 0x7f0608068010 2020-11-16T22:18:39.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f05f8012600 con 0x7f0608068010 2020-11-16T22:18:39.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 0x7f05e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05f806e160 con 0x7f0608068010 2020-11-16T22:18:39.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060c998700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 0x7f05e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.131+0000 7f060c998700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 0x7f05e0057520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f060809b380 tx=0x7f0604000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.139+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f05f8044360 con 0x7f0608068010 2020-11-16T22:18:39.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.287+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0608067ca0 con 0x7f0608068010 2020-11-16T22:18:39.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f05f57fa700 1 -- 172.21.15.185:0/3558214638 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f05f8047980 con 0x7f0608068010 2020-11-16T22:18:39.272 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:39.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 msgr2=0x7f05e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 0x7f05e0057520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f060809b380 tx=0x7f0604000b80).stop 2020-11-16T22:18:39.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 msgr2=0x7f0608151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f05f800ebd0 tx=0x7f05f8006df0).stop 2020-11-16T22:18:39.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 shutdown_connections 2020-11-16T22:18:39.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05e0055060 0x7f05e0057520 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06080acee0 0x7f060813d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0608068010 0x7f0608151480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 --2- 172.21.15.185:0/3558214638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f06080ab5a0 0x7f0608063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 >> 172.21.15.185:0/3558214638 conn(0x7f06080657b0 msgr2=0x7f0608140670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:39.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 shutdown_connections 2020-11-16T22:18:39.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.291+0000 7f060f3fd700 1 -- 172.21.15.185:0/3558214638 wait complete. 2020-11-16T22:18:39.289 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:18:39.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 Processor -- start 2020-11-16T22:18:39.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 -- start start 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fce98141690 0x7fce98067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b1f20 0x7fce980b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce980b3f50 con 0x7fce980b7690 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce980b41f0 con 0x7fce98141690 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9d7ff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce980b4490 con 0x7fce980b1f20 2020-11-16T22:18:39.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce977fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fce98141690 0x7fce98067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce977fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fce98141690 0x7fce98067420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60932/0 (socket says 172.21.15.185:60932) 2020-11-16T22:18:39.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce977fe700 1 -- 172.21.15.185:0/187443294 learned_addr learned my addr 172.21.15.185:0/187443294 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:39.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce97fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fce980b7690 0x7fce98144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54372/0 (socket says 172.21.15.185:54372) 2020-11-16T22:18:39.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.439+0000 7fce9cffe700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b1f20 0x7fce980b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce977fe700 1 -- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b1f20 msgr2=0x7fce980b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce977fe700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b1f20 0x7fce980b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce977fe700 1 -- 172.21.15.185:0/187443294 >> v1:172.21.15.185:6789/0 conn(0x7fce980b7690 legacy=0x7fce98144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce96ffd700 1 -- 172.21.15.185:0/187443294 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200529862 0 0) 0x7fce980b3f50 con 0x7fce980b7690 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce977fe700 1 -- 172.21.15.185:0/187443294 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980b4e20 con 0x7fce98141690 2020-11-16T22:18:39.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce977fe700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98067420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fce8c006fd0 tx=0x7fce8c006f40).ready entity=mon.1 client_cookie=1681b9dc639be7fa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce96ffd700 1 -- 172.21.15.185:0/187443294 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00dc60 con 0x7fce98141690 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce96ffd700 1 -- 172.21.15.185:0/187443294 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c00ddc0 con 0x7fce98141690 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce96ffd700 1 -- 172.21.15.185:0/187443294 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c010680 con 0x7fce98141690 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 -- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 msgr2=0x7fce98067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98067420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fce8c006fd0 tx=0x7fce8c006f40).stop 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 -- 172.21.15.185:0/187443294 shutdown_connections 2020-11-16T22:18:39.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b1f20 0x7fce980b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/187443294 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98067420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 -- 172.21.15.185:0/187443294 >> 172.21.15.185:0/187443294 conn(0x7fce98064b90 msgr2=0x7fce98063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:39.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 -- 172.21.15.185:0/187443294 shutdown_connections 2020-11-16T22:18:39.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.443+0000 7fce9d7ff700 1 -- 172.21.15.185:0/187443294 wait complete. 2020-11-16T22:18:39.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 Processor -- start 2020-11-16T22:18:39.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- start start 2020-11-16T22:18:39.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 0x7fce980adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9813e4b0 con 0x7fce980b1f20 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce980acf60 con 0x7fce98141690 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9813ea80 con 0x7fce980b7690 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54384/0 (socket says 172.21.15.185:54384) 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 -- 172.21.15.185:0/1677537934 learned_addr learned my addr 172.21.15.185:0/1677537934 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:39.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9cffe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce97fff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 0x7fce980adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 -- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 msgr2=0x7fce98156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 -- 172.21.15.185:0/1677537934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 msgr2=0x7fce980adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 0x7fce980adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 -- 172.21.15.185:0/1677537934 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980ad1c0 con 0x7fce980b7690 2020-11-16T22:18:39.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce97fff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 0x7fce980adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce977fe700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fce8c006a10 tx=0x7fce980ac470).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c01d070 con 0x7fce980b7690 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c0186e0 con 0x7fce980b7690 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce980ad460 con 0x7fce980b7690 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c018a60 con 0x7fce980b7690 2020-11-16T22:18:39.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.447+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce98157480 con 0x7fce980b7690 2020-11-16T22:18:39.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fce8c029070 con 0x7fce980b7690 2020-11-16T22:18:39.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce77fff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 0x7fce70057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce8c06f370 con 0x7fce980b7690 2020-11-16T22:18:39.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce97fff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 0x7fce70057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce980ad820 con 0x7fce980b7690 2020-11-16T22:18:39.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.451+0000 7fce97fff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 0x7fce70057520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fce9813bff0 tx=0x7fce88007140).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.455+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fce980ad820 con 0x7fce980b7690 2020-11-16T22:18:39.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce980ad820 con 0x7fce980b7690 2020-11-16T22:18:39.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce77fff700 1 -- 172.21.15.185:0/1677537934 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7fce980ad820 con 0x7fce980b7690 2020-11-16T22:18:39.592 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:39.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 msgr2=0x7fce70057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 0x7fce70057520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fce9813bff0 tx=0x7fce88007140).stop 2020-11-16T22:18:39.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 msgr2=0x7fce98156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fce8c006a10 tx=0x7fce980ac470).stop 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 shutdown_connections 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fce70055060 0x7fce70057520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fce98141690 0x7fce98156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce980b1f20 0x7fce980adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 --2- 172.21.15.185:0/1677537934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fce980b7690 0x7fce98156aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 >> 172.21.15.185:0/1677537934 conn(0x7fce98064b90 msgr2=0x7fce980604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:39.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 shutdown_connections 2020-11-16T22:18:39.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.611+0000 7fce9d7ff700 1 -- 172.21.15.185:0/1677537934 wait complete. 2020-11-16T22:18:39.607 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:39.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 Processor -- start 2020-11-16T22:18:39.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 -- start start 2020-11-16T22:18:39.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ffa001458a0 0x7ffa00145d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000b34f0 con 0x7ffa00148880 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000b7660 con 0x7ffa001458a0 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ffa05111700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000b7900 con 0x7ffa000602d0 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9ff7fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ffa001458a0 0x7ffa00145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54390/0 (socket says 172.21.15.185:54390) 2020-11-16T22:18:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9fffff700 1 -- 172.21.15.185:0/1994524993 learned_addr learned my addr 172.21.15.185:0/1994524993 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:39.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9fffff700 1 -- 172.21.15.185:0/1994524993 >> v2:172.21.15.96:3300/0 conn(0x7ffa001458a0 msgr2=0x7ffa00145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:39.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.747+0000 7ff9fffff700 1 --2- 172.21.15.185:0/1994524993 >> v2:172.21.15.96:3300/0 conn(0x7ffa001458a0 0x7ffa00145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9fffff700 1 -- 172.21.15.185:0/1994524993 >> v1:172.21.15.96:6789/0 conn(0x7ffa00148880 legacy=0x7ffa00148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9fffff700 1 -- 172.21.15.185:0/1994524993 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000b7ba0 con 0x7ffa000602d0 2020-11-16T22:18:39.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9fffff700 1 --2- 172.21.15.185:0/1994524993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7ff9f0006fd0 tx=0x7ff9f0006eb0).ready entity=mon.2 client_cookie=e17eb2cd84229157 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9feffd700 1 -- 172.21.15.185:0/1994524993 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0009810 con 0x7ffa000602d0 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9feffd700 1 -- 172.21.15.185:0/1994524993 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0009970 con 0x7ffa000602d0 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ff9feffd700 1 -- 172.21.15.185:0/1994524993 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0009cf0 con 0x7ffa000602d0 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 -- 172.21.15.185:0/1994524993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 msgr2=0x7ffa000673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 --2- 172.21.15.185:0/1994524993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7ff9f0006fd0 tx=0x7ff9f0006eb0).stop 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 -- 172.21.15.185:0/1994524993 shutdown_connections 2020-11-16T22:18:39.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 --2- 172.21.15.185:0/1994524993 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa000602d0 0x7ffa000673d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 --2- 172.21.15.185:0/1994524993 >> v2:172.21.15.96:3300/0 conn(0x7ffa001458a0 0x7ffa00145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 -- 172.21.15.185:0/1994524993 >> 172.21.15.185:0/1994524993 conn(0x7ffa0005ba80 msgr2=0x7ffa0005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:39.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 -- 172.21.15.185:0/1994524993 shutdown_connections 2020-11-16T22:18:39.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.751+0000 7ffa05111700 1 -- 172.21.15.185:0/1994524993 wait complete. 2020-11-16T22:18:39.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 Processor -- start 2020-11-16T22:18:39.736 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- start start 2020-11-16T22:18:39.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 0x7ffa00146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa00146770 0x7ffa00157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000b14f0 con 0x7ffa00148880 2020-11-16T22:18:39.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015b270 con 0x7ffa001458a0 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015b510 con 0x7ffa00146770 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60952/0 (socket says 172.21.15.185:60952) 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 -- 172.21.15.185:0/871359045 learned_addr learned my addr 172.21.15.185:0/871359045 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa04910700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa00146770 0x7ffa00157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9ff7fe700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 0x7ffa00146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 -- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa00146770 msgr2=0x7ffa00157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa00146770 0x7ffa00157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 -- 172.21.15.185:0/871359045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 msgr2=0x7ffa00146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 0x7ffa00146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 -- 172.21.15.185:0/871359045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0015b7b0 con 0x7ffa001458a0 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9ff7fe700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 0x7ffa00146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:39.740 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9fffff700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff9f000b1b0 tx=0x7ff9f0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0011c80 con 0x7ffa001458a0 2020-11-16T22:18:39.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0011de0 con 0x7ffa001458a0 2020-11-16T22:18:39.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa00111330 con 0x7ffa001458a0 2020-11-16T22:18:39.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f001e670 con 0x7ffa001458a0 2020-11-16T22:18:39.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.755+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa000b11f0 con 0x7ffa001458a0 2020-11-16T22:18:39.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0005d200 con 0x7ffa001458a0 2020-11-16T22:18:39.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff9f0025030 con 0x7ffa001458a0 2020-11-16T22:18:39.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ff9dffff700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 0x7ff9d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:39.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ff9ff7fe700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 0x7ff9d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:39.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff9f006f540 con 0x7ffa001458a0 2020-11-16T22:18:39.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.759+0000 7ff9ff7fe700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 0x7ff9d8057520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ffa00148fd0 tx=0x7ff9f4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:39.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.767+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff9f0045780 con 0x7ffa001458a0 2020-11-16T22:18:39.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.967+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa000ae440 con 0x7ffa001458a0 2020-11-16T22:18:39.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.967+0000 7ff9dffff700 1 -- 172.21.15.185:0/871359045 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+2526 (secure 0 0 0) 0x7ff9f0023020 con 0x7ffa001458a0 2020-11-16T22:18:39.950 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:39.950 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:17:38.870443+0000","modified":"2020-11-16T22:18:26.336948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6057,"mds_1":6054,"mds_2":5681},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5681":{"gid":5681,"name":"a","rank":2,"incarnation":54,"state":"up:active","state_seq":15,"addr":"172.21.15.185:6835/3773729030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3773729030},{"type":"v1","addr":"172.21.15.185:6835","nonce":3773729030}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6054":{"gid":6054,"name":"c","rank":1,"incarnation":52,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/774952173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6833","nonce":774952173},{"type":"v1","addr":"172.21.15.96:6835","nonce":774952173}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6057":{"gid":6057,"name":"b","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6834/2034361528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2034361528},{"type":"v1","addr":"172.21.15.96:6834","nonce":2034361528}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-16T22:18:40.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 msgr2=0x7ff9d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 0x7ff9d8057520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ffa00148fd0 tx=0x7ff9f4008040).stop 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 msgr2=0x7ffa0015bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff9f000b1b0 tx=0x7ff9f0015000).stop 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 shutdown_connections 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff9d8055060 0x7ff9d8057520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffa001458a0 0x7ffa0015bc50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffa00146770 0x7ffa00157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 --2- 172.21.15.185:0/871359045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa00148880 0x7ffa00146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 >> 172.21.15.185:0/871359045 conn(0x7ffa0005ba80 msgr2=0x7ffa001473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:40.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 shutdown_connections 2020-11-16T22:18:40.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:39.971+0000 7ffa05111700 1 -- 172.21.15.185:0/871359045 wait complete. 2020-11-16T22:18:40.157 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 61 2020-11-16T22:18:40.158 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 Processor -- start 2020-11-16T22:18:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- start start 2020-11-16T22:18:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb9d80b37e0 0x7fb9d80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d8066f80 con 0x7fb9d80b0f90 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d81478d0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8147b70 con 0x7fb9d8148550 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d77fe700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb9d80b37e0 0x7fb9d80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54406/0 (socket says 172.21.15.185:54406) 2020-11-16T22:18:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d7fff700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb9d80b0f90 0x7fb9d8144300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:40466/0 (socket says 172.21.15.185:40466) 2020-11-16T22:18:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d7fff700 1 -- 172.21.15.185:0/3708062310 learned_addr learned my addr 172.21.15.185:0/3708062310 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:40.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 -- 172.21.15.185:0/3708062310 >> v2:172.21.15.96:3300/0 conn(0x7fb9d80b37e0 msgr2=0x7fb9d80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:40.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 --2- 172.21.15.185:0/3708062310 >> v2:172.21.15.96:3300/0 conn(0x7fb9d80b37e0 0x7fb9d80b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 -- 172.21.15.185:0/3708062310 >> v1:172.21.15.96:6789/0 conn(0x7fb9d80b0f90 legacy=0x7fb9d8144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 -- 172.21.15.185:0/3708062310 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d8147e10 con 0x7fb9d8148550 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ceffd700 1 --2- 172.21.15.185:0/3708062310 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fb9c000b9a0 tx=0x7fb9c0006df0).ready entity=mon.2 client_cookie=52de059e110ed545 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d6ffd700 1 -- 172.21.15.185:0/3708062310 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00097b0 con 0x7fb9d8148550 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d6ffd700 1 -- 172.21.15.185:0/3708062310 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0009910 con 0x7fb9d8148550 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9d6ffd700 1 -- 172.21.15.185:0/3708062310 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0009c90 con 0x7fb9d8148550 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- 172.21.15.185:0/3708062310 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 msgr2=0x7fb9d8141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/3708062310 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fb9c000b9a0 tx=0x7fb9c0006df0).stop 2020-11-16T22:18:40.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- 172.21.15.185:0/3708062310 shutdown_connections 2020-11-16T22:18:40.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/3708062310 >> v2:172.21.15.96:3300/0 conn(0x7fb9d80b37e0 0x7fb9d80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/3708062310 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d8148550 0x7fb9d8141470 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- 172.21.15.185:0/3708062310 >> 172.21.15.185:0/3708062310 conn(0x7fb9d80604a0 msgr2=0x7fb9d80608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:40.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- 172.21.15.185:0/3708062310 shutdown_connections 2020-11-16T22:18:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.307+0000 7fb9ddf13700 1 -- 172.21.15.185:0/3708062310 wait complete. 2020-11-16T22:18:40.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 Processor -- start 2020-11-16T22:18:40.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 -- start start 2020-11-16T22:18:40.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 0x7fb9d8140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9d8148550 0x7fb9d8095fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81406a0 con 0x7fb9d8148550 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d814a700 con 0x7fb9d80b37e0 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ddf13700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d814a460 con 0x7fb9d80b0f90 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9d77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 0x7fb9d8140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60968/0 (socket says 172.21.15.185:60968) 2020-11-16T22:18:40.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9d77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 0x7fb9d8140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54410/0 (socket says 172.21.15.185:54410) 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 -- 172.21.15.185:0/2052132241 learned_addr learned my addr 172.21.15.185:0/2052132241 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9d7fff700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9d8148550 0x7fb9d8095fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 -- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 msgr2=0x7fb9d8140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 0x7fb9d8140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 -- 172.21.15.185:0/2052132241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9d8148550 msgr2=0x7fb9d8095fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9d8148550 0x7fb9d8095fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 -- 172.21.15.185:0/2052132241 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d8140900 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.311+0000 7fb9ceffd700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fb9c0006e30 tx=0x7fb9c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0011c80 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0011de0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c001e670 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d814a0a0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d809af20 con 0x7fb9d80b37e0 2020-11-16T22:18:40.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d805d4e0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb9c0025030 con 0x7fb9d80b37e0 2020-11-16T22:18:40.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 0x7fb9b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:40.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb9c006f460 con 0x7fb9d80b37e0 2020-11-16T22:18:40.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9d77fe700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 0x7fb9b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:40.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.315+0000 7fb9d77fe700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 0x7fb9b4057520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb9c8006fd0 tx=0x7fb9c8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:40.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.319+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb9c0045770 con 0x7fb9d80b37e0 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.471+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d81476b0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9cffff700 1 -- 172.21.15.185:0/2052132241 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16828 (secure 0 0 0) 0x7fb9c00311d0 con 0x7fb9d80b37e0 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 msgr2=0x7fb9b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 0x7fb9b4057520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fb9c8006fd0 tx=0x7fb9c8006df0).stop 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 msgr2=0x7fb9d8063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:40.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fb9c0006e30 tx=0x7fb9c0015000).stop 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 shutdown_connections 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb9b4055060 0x7fb9b4057520 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9d8148550 0x7fb9d8095fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb9d80b0f90 0x7fb9d8140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 --2- 172.21.15.185:0/2052132241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb9d80b37e0 0x7fb9d8063090 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 >> 172.21.15.185:0/2052132241 conn(0x7fb9d80604a0 msgr2=0x7fb9d80671a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:40.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 shutdown_connections 2020-11-16T22:18:40.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:40.475+0000 7fb9ddf13700 1 -- 172.21.15.185:0/2052132241 wait complete. 2020-11-16T22:18:40.709 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:40.709 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":55,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:17:39.874438+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:17:36.945221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-16T22:17:37.680544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:41.118 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:18:41.119 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:18:41.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 Processor -- start 2020-11-16T22:18:41.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 -- start start 2020-11-16T22:18:41.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fab8409b270 0x7fab8409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fab84144840 0x7fab840b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8409d410 con 0x7fab8409c900 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab84144cf0 con 0x7fab84144840 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab8a27a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840b6b20 con 0x7fab8409b270 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fab8409b270 0x7fab8409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab83fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fab8409c900 0x7fab841476e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54412/0 (socket says 172.21.15.185:54412) 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fab8409b270 0x7fab8409cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60974/0 (socket says 172.21.15.185:60974) 2020-11-16T22:18:41.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab83fff700 1 -- 172.21.15.185:0/1804148234 learned_addr learned my addr 172.21.15.185:0/1804148234 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:41.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab88817700 1 --2- 172.21.15.185:0/1804148234 >> v2:172.21.15.96:3300/0 conn(0x7fab84144840 0x7fab840b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 -- 172.21.15.185:0/1804148234 >> v2:172.21.15.96:3300/0 conn(0x7fab84144840 msgr2=0x7fab840b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:41.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 --2- 172.21.15.185:0/1804148234 >> v2:172.21.15.96:3300/0 conn(0x7fab84144840 0x7fab840b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:41.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 -- 172.21.15.185:0/1804148234 >> v1:172.21.15.185:6789/0 conn(0x7fab8409c900 legacy=0x7fab841476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:41.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 -- 172.21.15.185:0/1804148234 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8409b010 con 0x7fab8409b270 2020-11-16T22:18:41.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab837fe700 1 --2- 172.21.15.185:0/1804148234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab8409cce0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fab840811a0 tx=0x7fab74006df0).ready entity=mon.1 client_cookie=fda1cdea1d285b96 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:41.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab82ffd700 1 -- 172.21.15.185:0/1804148234 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab740097b0 con 0x7fab8409b270 2020-11-16T22:18:41.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab82ffd700 1 -- 172.21.15.185:0/1804148234 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab74009910 con 0x7fab8409b270 2020-11-16T22:18:41.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.271+0000 7fab82ffd700 1 -- 172.21.15.185:0/1804148234 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74009c90 con 0x7fab8409b270 2020-11-16T22:18:41.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- 172.21.15.185:0/1804148234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 msgr2=0x7fab8409cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:41.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 --2- 172.21.15.185:0/1804148234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab8409cce0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fab840811a0 tx=0x7fab74006df0).stop 2020-11-16T22:18:41.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- 172.21.15.185:0/1804148234 shutdown_connections 2020-11-16T22:18:41.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 --2- 172.21.15.185:0/1804148234 >> v2:172.21.15.96:3300/0 conn(0x7fab84144840 0x7fab840b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:41.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 --2- 172.21.15.185:0/1804148234 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab8409cce0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:41.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- 172.21.15.185:0/1804148234 >> 172.21.15.185:0/1804148234 conn(0x7fab84068330 msgr2=0x7fab840ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:41.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- 172.21.15.185:0/1804148234 shutdown_connections 2020-11-16T22:18:41.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- 172.21.15.185:0/1804148234 wait complete. 2020-11-16T22:18:41.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 Processor -- start 2020-11-16T22:18:41.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.275+0000 7fab8a27a700 1 -- start start 2020-11-16T22:18:41.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 0x7fab84148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab84144840 0x7fab840a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab84153a40 con 0x7fab84144840 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b1880 con 0x7fab8409b270 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b1b20 con 0x7fab8409c900 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab837fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 0x7fab84148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:60980/0 (socket says 172.21.15.185:60980) 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 -- 172.21.15.185:0/751475279 learned_addr learned my addr 172.21.15.185:0/751475279 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab88817700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab84144840 0x7fab840a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 -- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 msgr2=0x7fab84148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 0x7fab84148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:41.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 -- 172.21.15.185:0/751475279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab84144840 msgr2=0x7fab840a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:41.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab84144840 0x7fab840a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:41.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 -- 172.21.15.185:0/751475279 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840b1dc0 con 0x7fab8409b270 2020-11-16T22:18:41.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab837fe700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 0x7fab84148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:41.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab83fff700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fab6c00ca90 tx=0x7fab6c00cda0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:41.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c005b80 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c005ce0 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8409fc10 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c007320 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab84149240 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8409fdb0 con 0x7fab8409b270 2020-11-16T22:18:41.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.279+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fab6c010030 con 0x7fab8409b270 2020-11-16T22:18:41.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.283+0000 7fab7bfff700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 0x7fab54057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:41.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.283+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab6c06dd20 con 0x7fab8409b270 2020-11-16T22:18:41.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.283+0000 7fab837fe700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 0x7fab54057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:41.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.283+0000 7fab837fe700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 0x7fab54057520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fab7400be30 tx=0x7fab74015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:41.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.287+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fab8409fdb0 con 0x7fab8409b270 2020-11-16T22:18:41.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:41.491+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fab8405d4e0 con 0x7fab8409b270 2020-11-16T22:18:42.225 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:18:42.246+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:18:42.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.251+0000 7fab7bfff700 1 -- 172.21.15.185:0/751475279 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7fab6c043f80 con 0x7fab8409b270 2020-11-16T22:18:42.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 msgr2=0x7fab54057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 0x7fab54057520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fab7400be30 tx=0x7fab74015000).stop 2020-11-16T22:18:42.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 msgr2=0x7fab84153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fab6c00ca90 tx=0x7fab6c00cda0).stop 2020-11-16T22:18:42.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 shutdown_connections 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab54055060 0x7fab54057520 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab8409b270 0x7fab84153550 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab84144840 0x7fab840a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 --2- 172.21.15.185:0/751475279 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab8409c900 0x7fab84148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 >> 172.21.15.185:0/751475279 conn(0x7fab84068330 msgr2=0x7fab84149830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:42.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 shutdown_connections 2020-11-16T22:18:42.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.255+0000 7fab8a27a700 1 -- 172.21.15.185:0/751475279 wait complete. 2020-11-16T22:18:42.240 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:18:42.241 INFO:tasks.ceph.mds.b.smithi096.stderr: -7996> 2020-11-16T22:17:52.186+0000 7f4f6a19b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:18:42.242 INFO:tasks.ceph.mds.c.smithi096.stderr: -933> 2020-11-16T22:18:20.586+0000 7f88859b1700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1) n(v2 rc2020-11-16T22:17:52.371298+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@4},l=6346 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55dd1ad82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:17:52.371298+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.242 INFO:tasks.ceph.mds.c.smithi096.stderr: -814> 2020-11-16T22:18:20.590+0000 7f888b9bd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1) n(v1 rc2020-11-16T22:17:52.371298+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={6346=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55dd1ad82c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:17:52.371298+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.246 INFO:tasks.ceph.mds.a.smithi185.stderr: -895> 2020-11-16T22:18:20.583+0000 7f233a740700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1) n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@4},l=6346 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a2715fa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.599298+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:17:51.599298+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.246 INFO:tasks.ceph.mds.a.smithi185.stderr: -866> 2020-11-16T22:18:20.587+0000 7f233a740700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/AsLsXsFsx@2},l=6346 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a2715fa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.249 INFO:tasks.ceph.mds.b.smithi096.stderr: -1724> 2020-11-16T22:18:20.586+0000 7f4f66994700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1) n(v4 rc2020-11-16T22:17:52.371298+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={6346=pAsLsXsFsx/pAsLsXsFsx@5},l=6346 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5594ad0f2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.591298+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:17:52.371298+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.249 INFO:tasks.ceph.mds.b.smithi096.stderr: -1630> 2020-11-16T22:18:20.590+0000 7f4f6c9a0700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1) n(v3 rc2020-11-16T22:17:52.371298+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={6346=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x5594ad0f2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:17:52.371298+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:17:51.595298+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:17:52.371298+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:18:42.257 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:18:42.266 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:18:42.268 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:18:42.268 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:18:42.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 Processor -- start 2020-11-16T22:18:42.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- start start 2020-11-16T22:18:42.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcde40b3920 0x7fcde40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcde4143ef0 0x7fcde40b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde40b1310 con 0x7fcde40b2e80 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde413e400 con 0x7fcde4143ef0 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde40ac070 con 0x7fcde40b3920 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcde40b3920 0x7fcde40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde8c2b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fcde40b2e80 0x7fcde413dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54438/0 (socket says 172.21.15.185:54438) 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcde40b3920 0x7fcde40b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32770/0 (socket says 172.21.15.185:32770) 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde8c2b700 1 -- 172.21.15.185:0/4138117455 learned_addr learned my addr 172.21.15.185:0/4138117455 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:42.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde942c700 1 --2- 172.21.15.185:0/4138117455 >> v2:172.21.15.96:3300/0 conn(0x7fcde4143ef0 0x7fcde40b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 -- 172.21.15.185:0/4138117455 >> v2:172.21.15.96:3300/0 conn(0x7fcde4143ef0 msgr2=0x7fcde40b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 --2- 172.21.15.185:0/4138117455 >> v2:172.21.15.96:3300/0 conn(0x7fcde4143ef0 0x7fcde40b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 -- 172.21.15.185:0/4138117455 >> v1:172.21.15.185:6789/0 conn(0x7fcde40b2e80 legacy=0x7fcde413dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 -- 172.21.15.185:0/4138117455 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40ac310 con 0x7fcde40b3920 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde3fff700 1 --2- 172.21.15.185:0/4138117455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde40b3920 0x7fcde40b3260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fcde40b38c0 tx=0x7fcdcc006df0).ready entity=mon.1 client_cookie=c6268d9f01cfd9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:42.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde37fe700 1 -- 172.21.15.185:0/4138117455 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0097b0 con 0x7fcde40b3920 2020-11-16T22:18:42.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde37fe700 1 -- 172.21.15.185:0/4138117455 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc009910 con 0x7fcde40b3920 2020-11-16T22:18:42.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde37fe700 1 -- 172.21.15.185:0/4138117455 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc009c90 con 0x7fcde40b3920 2020-11-16T22:18:42.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- 172.21.15.185:0/4138117455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde40b3920 msgr2=0x7fcde40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/4138117455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde40b3920 0x7fcde40b3260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fcde40b38c0 tx=0x7fcdcc006df0).stop 2020-11-16T22:18:42.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- 172.21.15.185:0/4138117455 shutdown_connections 2020-11-16T22:18:42.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/4138117455 >> v2:172.21.15.96:3300/0 conn(0x7fcde4143ef0 0x7fcde40b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/4138117455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde40b3920 0x7fcde40b3260 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- 172.21.15.185:0/4138117455 >> 172.21.15.185:0/4138117455 conn(0x7fcde4060130 msgr2=0x7fcde4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- 172.21.15.185:0/4138117455 shutdown_connections 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.395+0000 7fcde9c2d700 1 -- 172.21.15.185:0/4138117455 wait complete. 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 Processor -- start 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- start start 2020-11-16T22:18:42.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 0x7fcde4143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcde40b3920 0x7fcde4153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4153320 con 0x7fcde40b2e80 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4143ac0 con 0x7fcde4143ef0 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4153570 con 0x7fcde40b3920 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcde40b3920 0x7fcde4153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32776/0 (socket says 172.21.15.185:32776) 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 -- 172.21.15.185:0/2559003256 learned_addr learned my addr 172.21.15.185:0/2559003256 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde8c2b700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 0x7fcde4143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 -- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcde40b3920 msgr2=0x7fcde4153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcde40b3920 0x7fcde4153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 -- 172.21.15.185:0/2559003256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 msgr2=0x7fcde4143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:42.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 0x7fcde4143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 -- 172.21.15.185:0/2559003256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4153810 con 0x7fcde4143ef0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde8c2b700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 0x7fcde4143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde942c700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fcddc00eba0 tx=0x7fcddc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00b2e0 con 0x7fcde4143ef0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00c820 con 0x7fcde4143ef0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde4143100 con 0x7fcde4143ef0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00cbc0 con 0x7fcde4143ef0 2020-11-16T22:18:42.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.399+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde41541a0 con 0x7fcde4143ef0 2020-11-16T22:18:42.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde4143c80 con 0x7fcde4143ef0 2020-11-16T22:18:42.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcddc01c460 con 0x7fcde4143ef0 2020-11-16T22:18:42.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde0ff9700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 0x7fcdb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:42.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fcddc06e210 con 0x7fcde4143ef0 2020-11-16T22:18:42.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde8c2b700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 0x7fcdb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:42.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.403+0000 7fcde8c2b700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 0x7fcdb4057520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fcdd4007fd0 tx=0x7fcdd4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:42.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.407+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcde4143c80 con 0x7fcde4143ef0 2020-11-16T22:18:42.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:42.607+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcde4143c50 con 0x7fcde4143ef0 2020-11-16T22:18:43.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.267+0000 7fcde0ff9700 1 -- 172.21.15.185:0/2559003256 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcddc044310 con 0x7fcde4143ef0 2020-11-16T22:18:43.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 msgr2=0x7fcdb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 0x7fcdb4057520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fcdd4007fd0 tx=0x7fcdd4009040).stop 2020-11-16T22:18:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 msgr2=0x7fcde4067b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fcddc00eba0 tx=0x7fcddc006df0).stop 2020-11-16T22:18:43.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 shutdown_connections 2020-11-16T22:18:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcdb4055060 0x7fcdb4057520 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcde4143ef0 0x7fcde4067b10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcde40b2e80 0x7fcde4143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 --2- 172.21.15.185:0/2559003256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcde40b3920 0x7fcde4153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 >> 172.21.15.185:0/2559003256 conn(0x7fcde4060130 msgr2=0x7fcde40668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 shutdown_connections 2020-11-16T22:18:43.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.271+0000 7fcde9c2d700 1 -- 172.21.15.185:0/2559003256 wait complete. 2020-11-16T22:18:43.269 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:18:43.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.463+0000 7f0cf89de700 1 Processor -- start 2020-11-16T22:18:43.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.463+0000 7f0cf89de700 1 -- start start 2020-11-16T22:18:43.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:43.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40b51e0 con 0x7f0cf40b75a0 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40b4ca0 con 0x7f0cf40b6120 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40b4f40 con 0x7f0cf40b6890 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54462/0 (socket says 172.21.15.185:54462) 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf1d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f0cf40b6120 0x7f0cf412d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54452/0 (socket says 172.21.15.185:54452) 2020-11-16T22:18:43.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 -- 172.21.15.185:0/3337730816 learned_addr learned my addr 172.21.15.185:0/3337730816 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf159a700 1 -- 172.21.15.185:0/3337730816 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318005119 0 0) 0x7f0cf40b4ca0 con 0x7f0cf40b6120 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 -- 172.21.15.185:0/3337730816 >> v1:172.21.15.185:6789/0 conn(0x7f0cf40b6120 legacy=0x7f0cf412d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 -- 172.21.15.185:0/3337730816 >> v1:172.21.15.96:6789/0 conn(0x7f0cf40b75a0 legacy=0x7f0cf40b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 -- 172.21.15.185:0/3337730816 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf40b5570 con 0x7f0cf40b6890 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf2d9d700 1 --2- 172.21.15.185:0/3337730816 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f0cec009d80 tx=0x7f0cec009290).ready entity=mon.2 client_cookie=bb44417f0f41b8fd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:43.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf159a700 1 -- 172.21.15.185:0/3337730816 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00bc80 con 0x7f0cf40b6890 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf159a700 1 -- 172.21.15.185:0/3337730816 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00bde0 con 0x7f0cf40b6890 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf159a700 1 -- 172.21.15.185:0/3337730816 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec011610 con 0x7f0cf40b6890 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- 172.21.15.185:0/3337730816 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 msgr2=0x7f0cf412d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 --2- 172.21.15.185:0/3337730816 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f0cec009d80 tx=0x7f0cec009290).stop 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- 172.21.15.185:0/3337730816 shutdown_connections 2020-11-16T22:18:43.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 --2- 172.21.15.185:0/3337730816 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6890 0x7f0cf412d8b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- 172.21.15.185:0/3337730816 >> 172.21.15.185:0/3337730816 conn(0x7f0cf4067860 msgr2=0x7f0cf4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:43.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- 172.21.15.185:0/3337730816 shutdown_connections 2020-11-16T22:18:43.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 -- 172.21.15.185:0/3337730816 wait complete. 2020-11-16T22:18:43.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.467+0000 7f0cf89de700 1 Processor -- start 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- start start 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0cf40b6890 0x7f0cf4147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 0x7f0cf40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf414ae30 con 0x7f0cf40b75a0 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4149760 con 0x7f0cf40b6890 2020-11-16T22:18:43.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41474b0 con 0x7f0cf40b6120 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf1d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0cf40b6890 0x7f0cf4147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54464/0 (socket says 172.21.15.185:54464) 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 -- 172.21.15.185:0/670686260 learned_addr learned my addr 172.21.15.185:0/670686260 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf2d9d700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 0x7f0cf40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 -- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0cf40b6890 msgr2=0x7f0cf4147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0cf40b6890 0x7f0cf4147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 -- 172.21.15.185:0/670686260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 msgr2=0x7f0cf40ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 0x7f0cf40ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 -- 172.21.15.185:0/670686260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4147c30 con 0x7f0cf40b6120 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf2d9d700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 0x7f0cf40ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf259c700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f0cdc00c740 tx=0x7f0cdc007bc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:43.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc007020 con 0x7f0cf40b6120 2020-11-16T22:18:43.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc007180 con 0x7f0cf40b6120 2020-11-16T22:18:43.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf41494c0 con 0x7f0cf40b6120 2020-11-16T22:18:43.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00e560 con 0x7f0cf40b6120 2020-11-16T22:18:43.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.471+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4148fc0 con 0x7f0cf40b6120 2020-11-16T22:18:43.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf40b1490 con 0x7f0cf40b6120 2020-11-16T22:18:43.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0cdc010070 con 0x7f0cf40b6120 2020-11-16T22:18:43.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cea7fc700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 0x7f0cd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:43.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f0cdc06d1d0 con 0x7f0cf40b6120 2020-11-16T22:18:43.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cf1d9b700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 0x7f0cd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:43.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.475+0000 7f0cf1d9b700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 0x7f0cd0057520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0cf414b8c0 tx=0x7f0ce4007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:43.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.479+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0cdc043330 con 0x7f0cf40b6120 2020-11-16T22:18:43.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:43.635+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cf405d4e0 con 0x7f0cf40b6120 2020-11-16T22:18:44.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.315+0000 7f0cea7fc700 1 -- 172.21.15.185:0/670686260 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0cdc046950 con 0x7f0cf40b6120 2020-11-16T22:18:44.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 msgr2=0x7f0cd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 0x7f0cd0057520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f0cf414b8c0 tx=0x7f0ce4007a00).stop 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 msgr2=0x7f0cf41456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f0cdc00c740 tx=0x7f0cdc007bc0).stop 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 shutdown_connections 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cd0055060 0x7f0cd0057520 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf40b75a0 0x7f0cf40ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0cf40b6120 0x7f0cf41456e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 --2- 172.21.15.185:0/670686260 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0cf40b6890 0x7f0cf4147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 >> 172.21.15.185:0/670686260 conn(0x7f0cf4067860 msgr2=0x7f0cf412b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:44.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 shutdown_connections 2020-11-16T22:18:44.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.323+0000 7f0cf89de700 1 -- 172.21.15.185:0/670686260 wait complete. 2020-11-16T22:18:44.306 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:18:44.320 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:18:44.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 Processor -- start 2020-11-16T22:18:44.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 -- start start 2020-11-16T22:18:44.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ce80ad110 0x7f6ce81477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce8061fe0 con 0x7f6ce80b3090 2020-11-16T22:18:44.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce80b2b60 con 0x7f6ce80ad110 2020-11-16T22:18:44.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce814c060 con 0x7f6ce80b1d30 2020-11-16T22:18:44.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54474/0 (socket says 172.21.15.185:54474) 2020-11-16T22:18:44.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 -- 172.21.15.185:0/514602896 learned_addr learned my addr 172.21.15.185:0/514602896 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:44.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce77fe700 1 --2- 172.21.15.185:0/514602896 >> v2:172.21.15.96:3300/0 conn(0x7f6ce80ad110 0x7f6ce81477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 -- 172.21.15.185:0/514602896 >> v2:172.21.15.96:3300/0 conn(0x7f6ce80ad110 msgr2=0x7f6ce81477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:44.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 --2- 172.21.15.185:0/514602896 >> v2:172.21.15.96:3300/0 conn(0x7f6ce80ad110 0x7f6ce81477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 -- 172.21.15.185:0/514602896 >> v1:172.21.15.96:6789/0 conn(0x7f6ce80b3090 legacy=0x7f6ce814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:44.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 -- 172.21.15.185:0/514602896 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce80b0fe0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce67fc700 1 --2- 172.21.15.185:0/514602896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f6cd0006fd0 tx=0x7f6cd0006eb0).ready entity=mon.2 client_cookie=127fd735c4152c3d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:44.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce5ffb700 1 -- 172.21.15.185:0/514602896 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0009890 con 0x7f6ce80b1d30 2020-11-16T22:18:44.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce5ffb700 1 -- 172.21.15.185:0/514602896 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd00099f0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.467+0000 7f6ce5ffb700 1 -- 172.21.15.185:0/514602896 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0009d70 con 0x7f6ce80b1d30 2020-11-16T22:18:44.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- 172.21.15.185:0/514602896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 msgr2=0x7f6ce80b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:44.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/514602896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f6cd0006fd0 tx=0x7f6cd0006eb0).stop 2020-11-16T22:18:44.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- 172.21.15.185:0/514602896 shutdown_connections 2020-11-16T22:18:44.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/514602896 >> v2:172.21.15.96:3300/0 conn(0x7f6ce80ad110 0x7f6ce81477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/514602896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b1d30 0x7f6ce80b1720 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- 172.21.15.185:0/514602896 >> 172.21.15.185:0/514602896 conn(0x7f6ce8060130 msgr2=0x7f6ce8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:44.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- 172.21.15.185:0/514602896 shutdown_connections 2020-11-16T22:18:44.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- 172.21.15.185:0/514602896 wait complete. 2020-11-16T22:18:44.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 Processor -- start 2020-11-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- start start 2020-11-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce80ad110 0x7f6ce81473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b3090 0x7f6ce815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8157830 con 0x7f6ce80ad110 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80b6ab0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce810f8e0 con 0x7f6ce80b3090 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b3090 0x7f6ce815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32806/0 (socket says 172.21.15.185:32806) 2020-11-16T22:18:44.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 -- 172.21.15.185:0/3881534837 learned_addr learned my addr 172.21.15.185:0/3881534837 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:44.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce6ffd700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce80ad110 0x7f6ce81473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 -- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b3090 msgr2=0x7f6ce815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:44.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b3090 0x7f6ce815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 -- 172.21.15.185:0/3881534837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce80ad110 msgr2=0x7f6ce81473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:44.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce80ad110 0x7f6ce81473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:44.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 -- 172.21.15.185:0/3881534837 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce810fb80 con 0x7f6ce80b1d30 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.471+0000 7f6ce67fc700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f6ce814c1a0 tx=0x7f6cd0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0011d50 con 0x7f6ce80b1d30 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd001e460 con 0x7f6ce80b1d30 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce81478c0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd001e7e0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce80b67c0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce80b11c0 con 0x7f6ce80b1d30 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6cd0025030 con 0x7f6ce80b1d30 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 0x7f6cb8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f6cd006f410 con 0x7f6ce80b1d30 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6ce6ffd700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 0x7f6cb8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.475+0000 7f6ce6ffd700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 0x7f6cb8057520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6ce00039b0 tx=0x7f6ce0007ef0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:44.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.483+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6cd0045850 con 0x7f6ce80b1d30 2020-11-16T22:18:44.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:44.635+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ce810fd10 con 0x7f6ce80b1d30 2020-11-16T22:18:45.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.323+0000 7f6cdeffd700 1 -- 172.21.15.185:0/3881534837 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6cd0023070 con 0x7f6ce80b1d30 2020-11-16T22:18:45.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 msgr2=0x7f6cb8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 0x7f6cb8057520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6ce00039b0 tx=0x7f6ce0007ef0).stop 2020-11-16T22:18:45.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 msgr2=0x7f6ce815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f6ce814c1a0 tx=0x7f6cd0015000).stop 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 shutdown_connections 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6cb8055060 0x7f6cb8057520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ce80b3090 0x7f6ce815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce80ad110 0x7f6ce81473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 --2- 172.21.15.185:0/3881534837 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ce80b1d30 0x7f6ce815af80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 >> 172.21.15.185:0/3881534837 conn(0x7f6ce8060130 msgr2=0x7f6ce80608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 shutdown_connections 2020-11-16T22:18:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.327+0000 7f6ce7fff700 1 -- 172.21.15.185:0/3881534837 wait complete. 2020-11-16T22:18:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:18:45.324 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:18:45.324 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:18:45.325 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:18:45.325 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:18:45.325 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:18:45.325 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:18:45.326 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:18:45.326 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:18:45.326 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:18:45.326 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:18:45.326 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:18:45.327 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:18:45.338 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:18:45.338 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:18:45.339 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:18:45.340 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:18:45.340 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:18:45.340 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:18:45.380 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:18:45.381 DEBUG:teuthology.parallel:result is None 2020-11-16T22:18:45.383 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:18:45.383 DEBUG:teuthology.parallel:result is None 2020-11-16T22:18:45.384 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:18:45.384 DEBUG:teuthology.parallel:result is None 2020-11-16T22:18:45.384 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:18:45.454 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:18:45.456 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:18:45.476 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:18:45.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 Processor -- start 2020-11-16T22:18:45.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- start start 2020-11-16T22:18:45.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49c013e820 0x7f49c0141050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 0x7f49c0096e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c0141620 con 0x7f49c013e440 2020-11-16T22:18:45.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c01418c0 con 0x7f49c013e820 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0141b60 con 0x7f49c00b2af0 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49c013e820 0x7f49c0141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c5049700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 0x7f49c0096e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49c013e820 0x7f49c0141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32826/0 (socket says 172.21.15.185:32826) 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 -- 172.21.15.185:0/696683489 learned_addr learned my addr 172.21.15.185:0/696683489 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:45.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 -- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 msgr2=0x7f49c0096e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 0x7f49c0096e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 -- 172.21.15.185:0/696683489 >> v1:172.21.15.96:6789/0 conn(0x7f49c013e440 legacy=0x7f49c00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 -- 172.21.15.185:0/696683489 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b3ea0 con 0x7f49c013e820 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c5049700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 0x7f49c0096e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bffff700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c013e820 0x7f49c0141050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f49b4006fd0 tx=0x7f49b4006eb0).ready entity=mon.1 client_cookie=a47c1f368fdc9f2a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bf7fe700 1 -- 172.21.15.185:0/696683489 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4009810 con 0x7f49c013e820 2020-11-16T22:18:45.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bf7fe700 1 -- 172.21.15.185:0/696683489 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4009970 con 0x7f49c013e820 2020-11-16T22:18:45.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49bf7fe700 1 -- 172.21.15.185:0/696683489 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4009cf0 con 0x7f49c013e820 2020-11-16T22:18:45.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c013e820 msgr2=0x7f49c0141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c013e820 0x7f49c0141050 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f49b4006fd0 tx=0x7f49b4006eb0).stop 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- 172.21.15.185:0/696683489 shutdown_connections 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c00b2af0 0x7f49c0096e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 --2- 172.21.15.185:0/696683489 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c013e820 0x7f49c0141050 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- 172.21.15.185:0/696683489 >> 172.21.15.185:0/696683489 conn(0x7f49c00604a0 msgr2=0x7f49c00608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:45.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- 172.21.15.185:0/696683489 shutdown_connections 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.535+0000 7f49c6aac700 1 -- 172.21.15.185:0/696683489 wait complete. 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 Processor -- start 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- start start 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 0x7f49c0155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49c013e820 0x7f49c009b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c013eeb0 con 0x7f49c013e820 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c013f150 con 0x7f49c00b2af0 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c013f3f0 con 0x7f49c013e440 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c4848700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 0x7f49c0155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54508/0 (socket says 172.21.15.185:54508) 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c4848700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 0x7f49c0155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32830/0 (socket says 172.21.15.185:32830) 2020-11-16T22:18:45.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 -- 172.21.15.185:0/1374760957 learned_addr learned my addr 172.21.15.185:0/1374760957 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c5049700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49c013e820 0x7f49c009b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 -- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 msgr2=0x7f49c0155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 0x7f49c0155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 -- 172.21.15.185:0/1374760957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49c013e820 msgr2=0x7f49c009b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49c013e820 0x7f49c009b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 -- 172.21.15.185:0/1374760957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c013f890 con 0x7f49c013e440 2020-11-16T22:18:45.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bffff700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f49b4010000 tx=0x7f49b4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:45.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011d50 con 0x7f49c013e440 2020-11-16T22:18:45.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b401e460 con 0x7f49c013e440 2020-11-16T22:18:45.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b401e820 con 0x7f49c013e440 2020-11-16T22:18:45.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c0098d20 con 0x7f49c013e440 2020-11-16T22:18:45.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c013fb20 con 0x7f49c013e440 2020-11-16T22:18:45.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.539+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c005d4e0 con 0x7f49c013e440 2020-11-16T22:18:45.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.543+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f49b4025030 con 0x7f49c013e440 2020-11-16T22:18:45.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.543+0000 7f49bcff9700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 0x7f4998057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:45.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.543+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f49b406ef90 con 0x7f49c013e440 2020-11-16T22:18:45.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.543+0000 7f49c4848700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 0x7f4998057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:45.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.543+0000 7f49c4848700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 0x7f4998057520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f49ac007fd0 tx=0x7f49ac000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:45.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.547+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f49b40454d0 con 0x7f49c013e440 2020-11-16T22:18:45.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:45.710+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f49c0145550 con 0x7f49c013e440 2020-11-16T22:18:46.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49bcff9700 1 -- 172.21.15.185:0/1374760957 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f49b4023020 con 0x7f49c013e440 2020-11-16T22:18:46.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 msgr2=0x7f4998057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 0x7f4998057520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f49ac007fd0 tx=0x7f49ac000b80).stop 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 msgr2=0x7f49c0098830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f49b4010000 tx=0x7f49b4015000).stop 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 shutdown_connections 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4998055060 0x7f4998057520 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49c00b2af0 0x7f49c0155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49c013e820 0x7f49c009b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 --2- 172.21.15.185:0/1374760957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49c013e440 0x7f49c0098830 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.538+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 >> 172.21.15.185:0/1374760957 conn(0x7f49c00604a0 msgr2=0x7f49c01449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.542+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 shutdown_connections 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.542+0000 7f49c6aac700 1 -- 172.21.15.185:0/1374760957 wait complete. 2020-11-16T22:18:46.525 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:18:46.538 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:18:46.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 Processor -- start 2020-11-16T22:18:46.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 -- start start 2020-11-16T22:18:46.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d740b7470 0x7f1d740b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:46.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d74067510 con 0x7f1d74067070 2020-11-16T22:18:46.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d740677b0 con 0x7f1d740682b0 2020-11-16T22:18:46.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d7ad69700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740b17a0 con 0x7f1d740b7470 2020-11-16T22:18:46.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d79d67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d740b7470 0x7f1d740b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:46.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d79d67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1d740b7470 0x7f1d740b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32840/0 (socket says 172.21.15.185:32840) 2020-11-16T22:18:46.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.682+0000 7f1d79d67700 1 -- 172.21.15.185:0/1117577051 learned_addr learned my addr 172.21.15.185:0/1117577051 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:46.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d79d67700 1 -- 172.21.15.185:0/1117577051 >> v1:172.21.15.185:6789/0 conn(0x7f1d740682b0 legacy=0x7f1d740ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d79d67700 1 -- 172.21.15.185:0/1117577051 >> v1:172.21.15.96:6789/0 conn(0x7f1d74067070 legacy=0x7f1d740b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:46.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d79d67700 1 -- 172.21.15.185:0/1117577051 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740b1a40 con 0x7f1d740b7470 2020-11-16T22:18:46.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d79d67700 1 --2- 172.21.15.185:0/1117577051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d740b7850 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f1d70009d50 tx=0x7f1d70009290).ready entity=mon.1 client_cookie=c7105fb96bbaf4cd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:46.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d78d65700 1 -- 172.21.15.185:0/1117577051 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000bc40 con 0x7f1d740b7470 2020-11-16T22:18:46.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d78d65700 1 -- 172.21.15.185:0/1117577051 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000bda0 con 0x7f1d740b7470 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d78d65700 1 -- 172.21.15.185:0/1117577051 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70010620 con 0x7f1d740b7470 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- 172.21.15.185:0/1117577051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 msgr2=0x7f1d740b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/1117577051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d740b7850 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f1d70009d50 tx=0x7f1d70009290).stop 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- 172.21.15.185:0/1117577051 shutdown_connections 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/1117577051 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d740b7850 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- 172.21.15.185:0/1117577051 >> 172.21.15.185:0/1117577051 conn(0x7f1d740637b0 msgr2=0x7f1d740b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:46.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- 172.21.15.185:0/1117577051 shutdown_connections 2020-11-16T22:18:46.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- 172.21.15.185:0/1117577051 wait complete. 2020-11-16T22:18:46.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 Processor -- start 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.686+0000 7f1d7ad69700 1 -- start start 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 0x7f1d741535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d740682b0 0x7f1d740af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7413d8d0 con 0x7f1d740682b0 2020-11-16T22:18:46.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7413db70 con 0x7f1d740b7470 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7413de10 con 0x7f1d74067070 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32850/0 (socket says 172.21.15.185:32850) 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 -- 172.21.15.185:0/983992550 learned_addr learned my addr 172.21.15.185:0/983992550 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d79d67700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 0x7f1d741535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:46.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d79566700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d740682b0 0x7f1d740af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 -- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 msgr2=0x7f1d741535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 0x7f1d741535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 -- 172.21.15.185:0/983992550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d740682b0 msgr2=0x7f1d740af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d740682b0 0x7f1d740af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 -- 172.21.15.185:0/983992550 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7413e130 con 0x7f1d740b7470 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d79d67700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 0x7f1d741535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:46.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7a568700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1d6c00c710 tx=0x7f1d6c007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:46.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00e560 con 0x7f1d740b7470 2020-11-16T22:18:46.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00e6c0 con 0x7f1d740b7470 2020-11-16T22:18:46.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d740b0ba0 con 0x7f1d740b7470 2020-11-16T22:18:46.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00ea40 con 0x7f1d740b7470 2020-11-16T22:18:46.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740aff10 con 0x7f1d740b7470 2020-11-16T22:18:46.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d740b0f20 con 0x7f1d740b7470 2020-11-16T22:18:46.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1d6c019030 con 0x7f1d740b7470 2020-11-16T22:18:46.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 0x7f1d4c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:46.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d79d67700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 0x7f1d4c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:46.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.690+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1d6c06cef0 con 0x7f1d740b7470 2020-11-16T22:18:46.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.694+0000 7f1d79d67700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 0x7f1d4c057520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f1d700092d0 tx=0x7f1d70017000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:46.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.698+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1d6c043530 con 0x7f1d740b7470 2020-11-16T22:18:46.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.910+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1d7413f780 con 0x7f1d740b7470 2020-11-16T22:18:46.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d61ffb700 1 -- 172.21.15.185:0/983992550 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+4274 (secure 0 0 0) 0x7f1d6c046b50 con 0x7f1d740b7470 2020-11-16T22:18:46.896 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:46.896 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:18:46.896 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:18:46.897 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.898 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.899 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:18:46.900 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:18:46.901 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.902 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.903 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.904 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.905 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:18:46.906 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.907 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.908 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.909 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.910 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:18:46.911 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow r", 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.912 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.913 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:18:46.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:18:46.915 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:46.916 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:46.917 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:18:46.918 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:18:46.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 msgr2=0x7f1d4c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 0x7f1d4c057520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f1d700092d0 tx=0x7f1d70017000).stop 2020-11-16T22:18:46.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 msgr2=0x7f1d7413f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:46.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1d6c00c710 tx=0x7f1d6c007b30).stop 2020-11-16T22:18:46.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 shutdown_connections 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1d4c055060 0x7f1d4c057520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1d740b7470 0x7f1d7413f980 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1d74067070 0x7f1d741535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 --2- 172.21.15.185:0/983992550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d740682b0 0x7f1d740af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 >> 172.21.15.185:0/983992550 conn(0x7f1d740637b0 msgr2=0x7f1d740b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:46.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 shutdown_connections 2020-11-16T22:18:46.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:46.914+0000 7f1d7ad69700 1 -- 172.21.15.185:0/983992550 wait complete. 2020-11-16T22:18:46.921 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:47.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 Processor -- start 2020-11-16T22:18:47.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 -- start start 2020-11-16T22:18:47.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8684061e60 0x7f868414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840abf90 con 0x7f86840acc10 2020-11-16T22:18:47.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840ac230 con 0x7f86840669a0 2020-11-16T22:18:47.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868b3b3700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840ac4d0 con 0x7f8684061e60 2020-11-16T22:18:47.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868a3b1700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f86840669a0 0x7f8684066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54526/0 (socket says 172.21.15.185:54526) 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868a3b1700 1 -- 172.21.15.185:0/2405266475 learned_addr learned my addr 172.21.15.185:0/2405266475 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f868abb2700 1 --2- 172.21.15.185:0/2405266475 >> v2:172.21.15.96:3300/0 conn(0x7f8684061e60 0x7f868414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846215761 0 0) 0x7f86840ac230 con 0x7f86840669a0 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8670003240 con 0x7f86840669a0 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396286385 0 0) 0x7f86840abf90 con 0x7f86840acc10 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86840ac230 con 0x7f86840acc10 2020-11-16T22:18:47.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.058+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1600113036 0 0) 0x7f8670003240 con 0x7f86840669a0 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 >> v2:172.21.15.96:3300/0 conn(0x7f8684061e60 msgr2=0x7f868414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 --2- 172.21.15.185:0/2405266475 >> v2:172.21.15.96:3300/0 conn(0x7f8684061e60 0x7f868414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 >> v1:172.21.15.96:6789/0 conn(0x7f86840acc10 legacy=0x7f86841432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840ac770 con 0x7f86840669a0 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f86740014e0 con 0x7f86840669a0 2020-11-16T22:18:47.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 >> v1:172.21.15.185:6789/0 conn(0x7f86840669a0 legacy=0x7f8684066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86700038c0 0x7f8670003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86700044e0 0x7f8670004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f867000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840ac4d0 con 0x7f86700038c0 2020-11-16T22:18:47.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700053b0 con 0x7f86700044e0 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867000bca0 con 0x7f8670004e50 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f867000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f8689bb0700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86700044e0 0x7f8670004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868a3b1700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86700038c0 0x7f8670003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 -- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86700044e0 msgr2=0x7f8670004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86700044e0 0x7f8670004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 -- 172.21.15.185:0/2405266475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86700038c0 msgr2=0x7f8670003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86700038c0 0x7f8670003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 -- 172.21.15.185:0/2405266475 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670003240 con 0x7f8670004e50 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 -- 172.21.15.185:0/2405266475 reap_dead start 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868abb2700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f867000b730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f867c009ea0 tx=0x7f867c0091d0).ready entity=mon.2 client_cookie=bdc1f1ea18ddbd1f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00ec40 con 0x7f8670004e50 2020-11-16T22:18:47.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00eda0 con 0x7f8670004e50 2020-11-16T22:18:47.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f86893af700 1 -- 172.21.15.185:0/2405266475 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c012640 con 0x7f8670004e50 2020-11-16T22:18:47.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 -- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 msgr2=0x7f867000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f867000b730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f867c009ea0 tx=0x7f867c0091d0).stop 2020-11-16T22:18:47.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 -- 172.21.15.185:0/2405266475 shutdown_connections 2020-11-16T22:18:47.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 --2- 172.21.15.185:0/2405266475 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f867000b730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 -- 172.21.15.185:0/2405266475 >> 172.21.15.185:0/2405266475 conn(0x7f8684060130 msgr2=0x7f8684060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 -- 172.21.15.185:0/2405266475 shutdown_connections 2020-11-16T22:18:47.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 -- 172.21.15.185:0/2405266475 wait complete. 2020-11-16T22:18:47.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.062+0000 7f868b3b3700 1 Processor -- start 2020-11-16T22:18:47.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- start start 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86840ad9f0 0x7f86840ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86840b0280 0x7f86840aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840afa60 con 0x7f86840b0280 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840ade60 con 0x7f86840ad9f0 2020-11-16T22:18:47.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840aef10 con 0x7f8670004e50 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54542/0 (socket says 172.21.15.185:54542) 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 -- 172.21.15.185:0/491370731 learned_addr learned my addr 172.21.15.185:0/491370731 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f8689bb0700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86840ad9f0 0x7f86840ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868abb2700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86840b0280 0x7f86840aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 -- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86840ad9f0 msgr2=0x7f86840ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86840ad9f0 0x7f86840ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 -- 172.21.15.185:0/491370731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86840b0280 msgr2=0x7f86840aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86840b0280 0x7f86840aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 -- 172.21.15.185:0/491370731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684140ae0 con 0x7f8670004e50 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868a3b1700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8674004b60 tx=0x7f86840b21b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674007e40 con 0x7f8670004e50 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400f8a0 con 0x7f8670004e50 2020-11-16T22:18:47.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400fc60 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8684154130 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684147d00 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.066+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868405d200 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.070+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8674015410 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.070+0000 7f867a7fc700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 0x7f865c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.070+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f867406da10 con 0x7f8670004e50 2020-11-16T22:18:47.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.070+0000 7f8689bb0700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 0x7f865c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.070+0000 7f8689bb0700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 0x7f865c057520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f86840b4f60 tx=0x7f8680007b00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.074+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8674043f40 con 0x7f8670004e50 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8684144f30 con 0x7f8670004e50 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f867a7fc700 1 -- 172.21.15.185:0/491370731 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13676 (secure 0 0 0) 0x7f8674047560 con 0x7f8670004e50 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 msgr2=0x7f865c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 0x7f865c057520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f86840b4f60 tx=0x7f8680007b00).stop 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 msgr2=0x7f8684147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f8674004b60 tx=0x7f86840b21b0).stop 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 shutdown_connections 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f865c055060 0x7f865c057520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86840b0280 0x7f86840aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8670004e50 0x7f8684147810 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 --2- 172.21.15.185:0/491370731 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f86840ad9f0 0x7f86840ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.230+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 >> 172.21.15.185:0/491370731 conn(0x7f8684060130 msgr2=0x7f86840b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.234+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 shutdown_connections 2020-11-16T22:18:47.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.234+0000 7f868b3b3700 1 -- 172.21.15.185:0/491370731 wait complete. 2020-11-16T22:18:47.265 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:47.265 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":61,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:46.534059+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:47.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:18:47.266 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:18:47.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.414+0000 7f9d310e9700 1 Processor -- start 2020-11-16T22:18:47.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.414+0000 7f9d310e9700 1 -- start start 2020-11-16T22:18:47.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9d2c063f90 0x7f9d2c145530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0af570 con 0x7f9d2c145f10 2020-11-16T22:18:47.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b3d00 con 0x7f9d2c063f90 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0af2d0 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9d2c063f90 0x7f9d2c145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54552/0 (socket says 172.21.15.185:54552) 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 -- 172.21.15.185:0/1694364547 learned_addr learned my addr 172.21.15.185:0/1694364547 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 -- 172.21.15.185:0/1694364547 >> v2:172.21.15.185:3300/0 conn(0x7f9d2c063f90 msgr2=0x7f9d2c145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 --2- 172.21.15.185:0/1694364547 >> v2:172.21.15.185:3300/0 conn(0x7f9d2c063f90 0x7f9d2c145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 -- 172.21.15.185:0/1694364547 >> v1:172.21.15.185:6789/0 conn(0x7f9d2c145f10 legacy=0x7f9d2c14a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 -- 172.21.15.185:0/1694364547 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b3190 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2b7fe700 1 --2- 172.21.15.185:0/1694364547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f9d14006fd0 tx=0x7f9d14006eb0).ready entity=mon.2 client_cookie=1b7fd1e35b5a741f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2affd700 1 -- 172.21.15.185:0/1694364547 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14009810 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2affd700 1 -- 172.21.15.185:0/1694364547 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14009970 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d2affd700 1 -- 172.21.15.185:0/1694364547 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14009cf0 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- 172.21.15.185:0/1694364547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 msgr2=0x7f9d2c145a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 --2- 172.21.15.185:0/1694364547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f9d14006fd0 tx=0x7f9d14006eb0).stop 2020-11-16T22:18:47.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- 172.21.15.185:0/1694364547 shutdown_connections 2020-11-16T22:18:47.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 --2- 172.21.15.185:0/1694364547 >> v2:172.21.15.185:3300/0 conn(0x7f9d2c063f90 0x7f9d2c145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 --2- 172.21.15.185:0/1694364547 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c0b4c30 0x7f9d2c145a20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- 172.21.15.185:0/1694364547 >> 172.21.15.185:0/1694364547 conn(0x7f9d2c067940 msgr2=0x7f9d2c067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- 172.21.15.185:0/1694364547 shutdown_connections 2020-11-16T22:18:47.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.418+0000 7f9d310e9700 1 -- 172.21.15.185:0/1694364547 wait complete. 2020-11-16T22:18:47.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 Processor -- start 2020-11-16T22:18:47.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- start start 2020-11-16T22:18:47.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 0x7f9d2c0b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 0x7f9d2c13db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b7300 con 0x7f9d2c0b4c30 2020-11-16T22:18:47.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c155630 con 0x7f9d2c063f90 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b7530 con 0x7f9d2c145f10 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 0x7f9d2c0b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54560/0 (socket says 172.21.15.185:54560) 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 -- 172.21.15.185:0/349050422 learned_addr learned my addr 172.21.15.185:0/349050422 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 0x7f9d2c0b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32880/0 (socket says 172.21.15.185:32880) 2020-11-16T22:18:47.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 -- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 msgr2=0x7f9d2c0b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 0x7f9d2c0b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 -- 172.21.15.185:0/349050422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 msgr2=0x7f9d2c13db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d2b7fe700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 0x7f9d2c13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 0x7f9d2c13db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 -- 172.21.15.185:0/349050422 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b0f20 con 0x7f9d2c145f10 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d2b7fe700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 0x7f9d2c13db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:47.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d308e8700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f9d1c00c740 tx=0x7f9d1c007b30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00e560 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c00e6c0 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c0b11c0 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00ea40 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.422+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c154730 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9d1c019030 con 0x7f9d2c145f10 2020-11-16T22:18:47.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c05d200 con 0x7f9d2c145f10 2020-11-16T22:18:47.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d23fff700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 0x7f9d08057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9d1c06cda0 con 0x7f9d2c145f10 2020-11-16T22:18:47.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d2bfff700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 0x7f9d08057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.426+0000 7f9d2bfff700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 0x7f9d08057520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9d2c0ae8c0 tx=0x7f9d240091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.434+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9d1c0433e0 con 0x7f9d2c145f10 2020-11-16T22:18:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9d2c143080 con 0x7f9d2c145f10 2020-11-16T22:18:47.565 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d23fff700 1 -- 172.21.15.185:0/349050422 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9d1c046a00 con 0x7f9d2c145f10 2020-11-16T22:18:47.566 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:18:47.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 msgr2=0x7f9d08057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 0x7f9d08057520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f9d2c0ae8c0 tx=0x7f9d240091d0).stop 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 msgr2=0x7f9d2c0b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f9d1c00c740 tx=0x7f9d1c007b30).stop 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 shutdown_connections 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9d08055060 0x7f9d08057520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9d2c063f90 0x7f9d2c0b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9d2c145f10 0x7f9d2c0b7b70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 --2- 172.21.15.185:0/349050422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d2c0b4c30 0x7f9d2c13db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 >> 172.21.15.185:0/349050422 conn(0x7f9d2c067940 msgr2=0x7f9d2c0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 shutdown_connections 2020-11-16T22:18:47.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.582+0000 7f9d310e9700 1 -- 172.21.15.185:0/349050422 wait complete. 2020-11-16T22:18:47.580 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:18:47.709 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 Processor -- start 2020-11-16T22:18:47.710 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 -- start start 2020-11-16T22:18:47.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb010063fd0 0x7fb010061240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.711 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb0100b3c40 0x7fb0100b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100b4350 con 0x7fb010063fd0 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100b40d0 con 0x7fb0100b3c40 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100629a0 con 0x7fb0100b4a50 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.712 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb007fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb0100b3c40 0x7fb0100b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54568/0 (socket says 172.21.15.185:54568) 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 -- 172.21.15.185:0/2531045477 learned_addr learned my addr 172.21.15.185:0/2531045477 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00effd700 1 --2- 172.21.15.185:0/2531045477 >> v2:172.21.15.96:3300/0 conn(0x7fb010063fd0 0x7fb010061240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 -- 172.21.15.185:0/2531045477 >> v2:172.21.15.185:3300/0 conn(0x7fb0100b3c40 msgr2=0x7fb0100b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/2531045477 >> v2:172.21.15.185:3300/0 conn(0x7fb0100b3c40 0x7fb0100b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 -- 172.21.15.185:0/2531045477 >> v2:172.21.15.96:3300/0 conn(0x7fb010063fd0 msgr2=0x7fb010061240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:47.713 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/2531045477 >> v2:172.21.15.96:3300/0 conn(0x7fb010063fd0 0x7fb010061240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 -- 172.21.15.185:0/2531045477 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010062c40 con 0x7fb0100b4a50 2020-11-16T22:18:47.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.726+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/2531045477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fb010062610 tx=0x7fb008009290).ready entity=mon.2 client_cookie=7e50e498e85e290a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00e7fc700 1 -- 172.21.15.185:0/2531045477 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800bb10 con 0x7fb0100b4a50 2020-11-16T22:18:47.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00e7fc700 1 -- 172.21.15.185:0/2531045477 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00800bc70 con 0x7fb0100b4a50 2020-11-16T22:18:47.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00e7fc700 1 -- 172.21.15.185:0/2531045477 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008011400 con 0x7fb0100b4a50 2020-11-16T22:18:47.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- 172.21.15.185:0/2531045477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 msgr2=0x7fb010143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- 172.21.15.185:0/2531045477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fb010062610 tx=0x7fb008009290).stop 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- 172.21.15.185:0/2531045477 shutdown_connections 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- 172.21.15.185:0/2531045477 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010143310 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- 172.21.15.185:0/2531045477 >> v2:172.21.15.96:3300/0 conn(0x7fb010063fd0 0x7fb010061240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- 172.21.15.185:0/2531045477 >> v2:172.21.15.185:3300/0 conn(0x7fb0100b3c40 0x7fb0100b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- 172.21.15.185:0/2531045477 >> 172.21.15.185:0/2531045477 conn(0x7fb010067940 msgr2=0x7fb010067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- 172.21.15.185:0/2531045477 shutdown_connections 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- 172.21.15.185:0/2531045477 wait complete. 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 Processor -- start 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- start start 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 0x7fb0100ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100ae6f0 con 0x7fb010063fd0 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100acf70 con 0x7fb0100b3c40 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100ae290 con 0x7fb0100b4a50 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb007fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb007fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32898/0 (socket says 172.21.15.185:32898) 2020-11-16T22:18:47.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54572/0 (socket says 172.21.15.185:54572) 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb007fff700 1 -- 172.21.15.185:0/1251582130 learned_addr learned my addr 172.21.15.185:0/1251582130 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00effd700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 0x7fb0100ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 -- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 msgr2=0x7fb010140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 -- 172.21.15.185:0/1251582130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 msgr2=0x7fb0100ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 0x7fb0100ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 -- 172.21.15.185:0/1251582130 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100ad440 con 0x7fb0100b4a50 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00effd700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 0x7fb0100ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb007fff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.730+0000 7fb00f7fe700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fb008013000 tx=0x7fb008016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00801c680 con 0x7fb0100b4a50 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00801c7e0 con 0x7fb0100b4a50 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0100ad6e0 con 0x7fb0100b4a50 2020-11-16T22:18:47.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00801cb80 con 0x7fb0100b4a50 2020-11-16T22:18:47.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100acbf0 con 0x7fb0100b4a50 2020-11-16T22:18:47.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01005d200 con 0x7fb0100b4a50 2020-11-16T22:18:47.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb008017070 con 0x7fb0100b4a50 2020-11-16T22:18:47.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 0x7fafec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:47.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb00effd700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 0x7fafec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:47.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb00806f8e0 con 0x7fb0100b4a50 2020-11-16T22:18:47.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.734+0000 7fb00effd700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 0x7fafec057520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7faff8006fd0 tx=0x7faff8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:47.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.742+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb008045d60 con 0x7fb0100b4a50 2020-11-16T22:18:47.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb01005c720 con 0x7fb0100b4a50 2020-11-16T22:18:47.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb006ffd700 1 -- 172.21.15.185:0/1251582130 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb008015070 con 0x7fb0100b4a50 2020-11-16T22:18:47.877 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 msgr2=0x7fafec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 0x7fafec057520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7faff8006fd0 tx=0x7faff8006df0).stop 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 msgr2=0x7fb010141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fb008013000 tx=0x7fb008016000).stop 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 shutdown_connections 2020-11-16T22:18:47.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fafec055060 0x7fafec057520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb0100b4a50 0x7fb010141480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb010063fd0 0x7fb0100ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 --2- 172.21.15.185:0/1251582130 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb0100b3c40 0x7fb010140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 >> 172.21.15.185:0/1251582130 conn(0x7fb010067940 msgr2=0x7fb010147330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 shutdown_connections 2020-11-16T22:18:47.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:47.894+0000 7fb00ffff700 1 -- 172.21.15.185:0/1251582130 wait complete. 2020-11-16T22:18:47.891 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:18:48.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.030+0000 7f6ffdbd3700 1 Processor -- start 2020-11-16T22:18:48.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.030+0000 7f6ffdbd3700 1 -- start start 2020-11-16T22:18:48.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff80af130 con 0x7f6ff80b8820 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff80af600 con 0x7f6ff80b30e0 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80af360 con 0x7f6ff8141b30 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54580/0 (socket says 172.21.15.185:54580) 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffd3d2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6ff80b30e0 0x7f6ff81454e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54576/0 (socket says 172.21.15.185:54576) 2020-11-16T22:18:48.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 -- 172.21.15.185:0/3050254560 learned_addr learned my addr 172.21.15.185:0/3050254560 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:48.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967989789 0 0) 0x7f6ff80af600 con 0x7f6ff80b30e0 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe4003240 con 0x7f6ff80b30e0 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276009389 0 0) 0x7f6ff80af130 con 0x7f6ff80b8820 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 -- 172.21.15.185:0/3050254560 >> v1:172.21.15.185:6789/0 conn(0x7f6ff80b30e0 legacy=0x7f6ff81454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 -- 172.21.15.185:0/3050254560 >> v1:172.21.15.96:6789/0 conn(0x7f6ff80b8820 legacy=0x7f6ff8141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 -- 172.21.15.185:0/3050254560 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80b3dc0 con 0x7f6ff8141b30 2020-11-16T22:18:48.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff7fff700 1 --2- 172.21.15.185:0/3050254560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6fec00ba20 tx=0x7f6fec006eb0).ready entity=mon.2 client_cookie=99a08ec1917a3b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec009810 con 0x7f6ff8141b30 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec009970 con 0x7f6ff8141b30 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ff77fe700 1 -- 172.21.15.185:0/3050254560 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec009cf0 con 0x7f6ff8141b30 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/3050254560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 msgr2=0x7f6ff80af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/3050254560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6fec00ba20 tx=0x7f6fec006eb0).stop 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/3050254560 shutdown_connections 2020-11-16T22:18:48.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/3050254560 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff8141b30 0x7f6ff80af7e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/3050254560 >> 172.21.15.185:0/3050254560 conn(0x7f6ff80645d0 msgr2=0x7f6ff80649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:48.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/3050254560 shutdown_connections 2020-11-16T22:18:48.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.034+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/3050254560 wait complete. 2020-11-16T22:18:48.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 Processor -- start 2020-11-16T22:18:48.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- start start 2020-11-16T22:18:48.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 0x7f6ff81445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ff8140b60 0x7f6ff80ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80ae160 con 0x7f6ff8140b60 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80ae400 con 0x7f6ff80b8820 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80ae6a0 con 0x7f6ff80b30e0 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32910/0 (socket says 172.21.15.185:32910) 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 -- 172.21.15.185:0/2307382421 learned_addr learned my addr 172.21.15.185:0/2307382421 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:48.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffcbd1700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 0x7f6ff81445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffd3d2700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ff8140b60 0x7f6ff80ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 -- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 msgr2=0x7f6ff81445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 0x7f6ff81445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 -- 172.21.15.185:0/2307382421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ff8140b60 msgr2=0x7f6ff80ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ff8140b60 0x7f6ff80ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 -- 172.21.15.185:0/2307382421 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80ad180 con 0x7f6ff80b8820 2020-11-16T22:18:48.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffcbd1700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 0x7f6ff81445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff7fff700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f6ff80b8650 tx=0x7f6fec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec006890 con 0x7f6ff80b8820 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec011c80 con 0x7f6ff80b8820 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff813cd10 con 0x7f6ff80b8820 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec01d450 con 0x7f6ff80b8820 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff80adb10 con 0x7f6ff80b8820 2020-11-16T22:18:48.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff80ad360 con 0x7f6ff80b8820 2020-11-16T22:18:48.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.038+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6fec024030 con 0x7f6ff80b8820 2020-11-16T22:18:48.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.042+0000 7f6ff4ff9700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 0x7f6fd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.042+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6fec06ee90 con 0x7f6ff80b8820 2020-11-16T22:18:48.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.042+0000 7f6ffcbd1700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 0x7f6fd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.042+0000 7f6ffcbd1700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 0x7f6fd0057520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f6fe8001710 tx=0x7f6fe8009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.046+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6ff80ad360 con 0x7f6ff80b8820 2020-11-16T22:18:48.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ff80ad360 con 0x7f6ff80b8820 2020-11-16T22:18:48.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ff4ff9700 1 -- 172.21.15.185:0/2307382421 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6ff80ad360 con 0x7f6ff80b8820 2020-11-16T22:18:48.186 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:18:48.187 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:18:48.188 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:18:48.188 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:18:48.188 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 msgr2=0x7f6fd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 0x7f6fd0057520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f6fe8001710 tx=0x7f6fe8009040).stop 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 msgr2=0x7f6ff813c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.202+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f6ff80b8650 tx=0x7f6fec015000).stop 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 shutdown_connections 2020-11-16T22:18:48.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fd0055060 0x7f6fd0057520 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ff8140b60 0x7f6ff80ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ff80b30e0 0x7f6ff81445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 --2- 172.21.15.185:0/2307382421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ff80b8820 0x7f6ff813c820 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 >> 172.21.15.185:0/2307382421 conn(0x7f6ff80645d0 msgr2=0x7f6ff81411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 shutdown_connections 2020-11-16T22:18:48.190 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.206+0000 7f6ffdbd3700 1 -- 172.21.15.185:0/2307382421 wait complete. 2020-11-16T22:18:48.200 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:18:48.201 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:18:48.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.338+0000 7fd765c52700 1 Processor -- start 2020-11-16T22:18:48.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.338+0000 7fd765c52700 1 -- start start 2020-11-16T22:18:48.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.338+0000 7fd765c52700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd7600ac3c0 0x7fd76013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.338+0000 7fd765c52700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76013ddd0 con 0x7fd760066fa0 2020-11-16T22:18:48.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7601404e0 con 0x7fd760060130 2020-11-16T22:18:48.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7600b3350 con 0x7fd7600ac3c0 2020-11-16T22:18:48.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd764c50700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd7600ac3c0 0x7fd76013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765451700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd760060130 0x7fd760060550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54590/0 (socket says 172.21.15.185:54590) 2020-11-16T22:18:48.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765451700 1 -- 172.21.15.185:0/3012194460 learned_addr learned my addr 172.21.15.185:0/3012194460 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:48.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd75f7fe700 1 -- 172.21.15.185:0/3012194460 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501422016 0 0) 0x7fd7601404e0 con 0x7fd760060130 2020-11-16T22:18:48.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd75f7fe700 1 -- 172.21.15.185:0/3012194460 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c003240 con 0x7fd760060130 2020-11-16T22:18:48.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd764c50700 1 -- 172.21.15.185:0/3012194460 >> v1:172.21.15.185:6789/0 conn(0x7fd760060130 legacy=0x7fd760060550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd764c50700 1 -- 172.21.15.185:0/3012194460 >> v1:172.21.15.96:6789/0 conn(0x7fd760066fa0 legacy=0x7fd76013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd764c50700 1 -- 172.21.15.185:0/3012194460 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760140780 con 0x7fd7600ac3c0 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd764c50700 1 --2- 172.21.15.185:0/3012194460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7600ac3c0 0x7fd76013d1d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fd75000b9a0 tx=0x7fd750006df0).ready entity=mon.1 client_cookie=9cf3c0c41a1cb2d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd75f7fe700 1 -- 172.21.15.185:0/3012194460 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500097b0 con 0x7fd7600ac3c0 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd75f7fe700 1 -- 172.21.15.185:0/3012194460 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd750009910 con 0x7fd7600ac3c0 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd75f7fe700 1 -- 172.21.15.185:0/3012194460 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750009c90 con 0x7fd7600ac3c0 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- 172.21.15.185:0/3012194460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7600ac3c0 msgr2=0x7fd76013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 --2- 172.21.15.185:0/3012194460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7600ac3c0 0x7fd76013d1d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fd75000b9a0 tx=0x7fd750006df0).stop 2020-11-16T22:18:48.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- 172.21.15.185:0/3012194460 shutdown_connections 2020-11-16T22:18:48.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 --2- 172.21.15.185:0/3012194460 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7600ac3c0 0x7fd76013d1d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- 172.21.15.185:0/3012194460 >> 172.21.15.185:0/3012194460 conn(0x7fd7600b37b0 msgr2=0x7fd7600b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:48.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- 172.21.15.185:0/3012194460 shutdown_connections 2020-11-16T22:18:48.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- 172.21.15.185:0/3012194460 wait complete. 2020-11-16T22:18:48.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 Processor -- start 2020-11-16T22:18:48.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.342+0000 7fd765c52700 1 -- start start 2020-11-16T22:18:48.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 0x7fd760113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7600ac3c0 0x7fd760150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760113f60 con 0x7fd7600ac3c0 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760110e70 con 0x7fd760060130 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd760110080 con 0x7fd760066fa0 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 0x7fd760113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32922/0 (socket says 172.21.15.185:32922) 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 -- 172.21.15.185:0/3044168776 learned_addr learned my addr 172.21.15.185:0/3044168776 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765451700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7600ac3c0 0x7fd760150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 -- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 msgr2=0x7fd760113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 0x7fd760113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 -- 172.21.15.185:0/3044168776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7600ac3c0 msgr2=0x7fd760150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7600ac3c0 0x7fd760150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:48.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 -- 172.21.15.185:0/3044168776 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7601434f0 con 0x7fd760060130 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75ffff700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 0x7fd760113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd764c50700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd750006e30 tx=0x7fd750015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750011d50 con 0x7fd760060130 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75001e560 con 0x7fd760060130 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd760110520 con 0x7fd760060130 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75001e8e0 con 0x7fd760060130 2020-11-16T22:18:48.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7600b1870 con 0x7fd760060130 2020-11-16T22:18:48.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd760110240 con 0x7fd760060130 2020-11-16T22:18:48.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd75001a040 con 0x7fd760060130 2020-11-16T22:18:48.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 0x7fd738057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:48.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd7500714f0 con 0x7fd760060130 2020-11-16T22:18:48.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75ffff700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 0x7fd738057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:48.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.346+0000 7fd75ffff700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 0x7fd738057520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fd760066c90 tx=0x7fd754009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:48.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.358+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd750047b00 con 0x7fd760060130 2020-11-16T22:18:48.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:48.510+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd760044f60 con 0x7fd760060130 2020-11-16T22:18:48.684 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:48.690 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:48.692 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:18:49.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.350+0000 7fd75cff9700 1 -- 172.21.15.185:0/3044168776 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v62) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd75004b120 con 0x7fd760060130 2020-11-16T22:18:49.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 msgr2=0x7fd738057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 0x7fd738057520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fd760066c90 tx=0x7fd754009040).stop 2020-11-16T22:18:49.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 msgr2=0x7fd760113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd750006e30 tx=0x7fd750015000).stop 2020-11-16T22:18:49.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 shutdown_connections 2020-11-16T22:18:49.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd738055060 0x7fd738057520 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd760060130 0x7fd760113580 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7600ac3c0 0x7fd760150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 --2- 172.21.15.185:0/3044168776 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd760066fa0 0x7fd760113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 >> 172.21.15.185:0/3044168776 conn(0x7fd7600b37b0 msgr2=0x7fd760144080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:49.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 shutdown_connections 2020-11-16T22:18:49.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.354+0000 7fd765c52700 1 -- 172.21.15.185:0/3044168776 wait complete. 2020-11-16T22:18:49.341 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:18:49.359 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:18:49.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.502+0000 7f9fbb16f700 1 Processor -- start 2020-11-16T22:18:49.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.502+0000 7f9fbb16f700 1 -- start start 2020-11-16T22:18:49.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 0x7f9fb40666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:49.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40b7c20 con 0x7f9fb413de10 2020-11-16T22:18:49.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40b3a80 con 0x7f9fb413e1f0 2020-11-16T22:18:49.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40b3d20 con 0x7f9fb40b3570 2020-11-16T22:18:49.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fba96e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 0x7f9fb40666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:49.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb996c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f9fb413e1f0 0x7f9fb40b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54602/0 (socket says 172.21.15.185:54602) 2020-11-16T22:18:49.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb996c700 1 -- 172.21.15.185:0/3401153559 learned_addr learned my addr 172.21.15.185:0/3401153559 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:49.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fba96e700 1 -- 172.21.15.185:0/3401153559 >> v1:172.21.15.185:6789/0 conn(0x7f9fb413e1f0 legacy=0x7f9fb40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fba96e700 1 -- 172.21.15.185:0/3401153559 >> v1:172.21.15.96:6789/0 conn(0x7f9fb413de10 legacy=0x7f9fb40b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fba96e700 1 -- 172.21.15.185:0/3401153559 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40b3fc0 con 0x7f9fb40b3570 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb916b700 1 -- 172.21.15.185:0/3401153559 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199036861 0 0) 0x7f9fb40b3a80 con 0x7f9fb413e1f0 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fba96e700 1 --2- 172.21.15.185:0/3401153559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 0x7f9fb40666b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f9fac009cc0 tx=0x7f9fac0091d0).ready entity=mon.2 client_cookie=47bdca1edb4b5880 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb916b700 1 -- 172.21.15.185:0/3401153559 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00bb80 con 0x7f9fb40b3570 2020-11-16T22:18:49.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb916b700 1 -- 172.21.15.185:0/3401153559 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac00bce0 con 0x7f9fb40b3570 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fb916b700 1 -- 172.21.15.185:0/3401153559 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac011610 con 0x7f9fb40b3570 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- 172.21.15.185:0/3401153559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 msgr2=0x7f9fb40666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/3401153559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 0x7f9fb40666b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f9fac009cc0 tx=0x7f9fac0091d0).stop 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- 172.21.15.185:0/3401153559 shutdown_connections 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/3401153559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb40b3570 0x7f9fb40666b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- 172.21.15.185:0/3401153559 >> 172.21.15.185:0/3401153559 conn(0x7f9fb4064b90 msgr2=0x7f9fb4063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:49.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- 172.21.15.185:0/3401153559 shutdown_connections 2020-11-16T22:18:49.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.506+0000 7f9fbb16f700 1 -- 172.21.15.185:0/3401153559 wait complete. 2020-11-16T22:18:49.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 Processor -- start 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- start start 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9fb40b3570 0x7f9fb4146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 0x7f9fb4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4140a00 con 0x7f9fb413de10 2020-11-16T22:18:49.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4140020 con 0x7f9fb40b3570 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41402c0 con 0x7f9fb413eb30 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54610/0 (socket says 172.21.15.185:54610) 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 -- 172.21.15.185:0/1099357750 learned_addr learned my addr 172.21.15.185:0/1099357750 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba16d700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9fb40b3570 0x7f9fb4146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fb996c700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 0x7f9fb4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:49.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 -- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9fb40b3570 msgr2=0x7f9fb4146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9fb40b3570 0x7f9fb4146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 -- 172.21.15.185:0/1099357750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 msgr2=0x7f9fb4143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 0x7f9fb4143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 -- 172.21.15.185:0/1099357750 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4140560 con 0x7f9fb413eb30 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fb996c700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 0x7f9fb4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fba96e700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f9fac009ea0 tx=0x7f9fac016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac006ea0 con 0x7f9fb413eb30 2020-11-16T22:18:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac00be50 con 0x7f9fb413eb30 2020-11-16T22:18:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb41445b0 con 0x7f9fb413eb30 2020-11-16T22:18:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac010dc0 con 0x7f9fb413eb30 2020-11-16T22:18:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.510+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4147de0 con 0x7f9fb413eb30 2020-11-16T22:18:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb41406f0 con 0x7f9fb413eb30 2020-11-16T22:18:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9fac017070 con 0x7f9fb413eb30 2020-11-16T22:18:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fa27fc700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 0x7f9f8c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f9fac06f540 con 0x7f9fb413eb30 2020-11-16T22:18:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fba16d700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 0x7f9f8c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:49.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.514+0000 7f9fba16d700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 0x7f9f8c057520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9fb413f930 tx=0x7f9fa4007b70).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:49.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.518+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9fac0459f0 con 0x7f9fb413eb30 2020-11-16T22:18:49.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:49.682+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9fb405d200 con 0x7f9fb413eb30 2020-11-16T22:18:50.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.353+0000 7f9fa27fc700 1 -- 172.21.15.185:0/1099357750 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v63) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9fac0220a0 con 0x7f9fb413eb30 2020-11-16T22:18:50.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.361+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 msgr2=0x7f9f8c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.361+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 0x7f9f8c057520 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9fb413f930 tx=0x7f9fa4007b70).stop 2020-11-16T22:18:50.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.361+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 msgr2=0x7f9fb41440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.361+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f9fac009ea0 tx=0x7f9fac016000).stop 2020-11-16T22:18:50.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 shutdown_connections 2020-11-16T22:18:50.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f8c055060 0x7f9f8c057520 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9fb40b3570 0x7f9fb4146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9fb413eb30 0x7f9fb41440c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 --2- 172.21.15.185:0/1099357750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fb413de10 0x7f9fb4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 >> 172.21.15.185:0/1099357750 conn(0x7f9fb4064b90 msgr2=0x7f9fb40b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:50.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 shutdown_connections 2020-11-16T22:18:50.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.365+0000 7f9fbb16f700 1 -- 172.21.15.185:0/1099357750 wait complete. 2020-11-16T22:18:50.353 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:18:50.365 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:18:50.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.505+0000 7fb57ec28700 1 Processor -- start 2020-11-16T22:18:50.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.505+0000 7fb57ec28700 1 -- start start 2020-11-16T22:18:50.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.505+0000 7fb57ec28700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 0x7fb5780b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.505+0000 7fb57ec28700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb5780acee0 0x7fb5780b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5780ad370 con 0x7fb578148160 2020-11-16T22:18:50.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb578144f80 con 0x7fb5780b7170 2020-11-16T22:18:50.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb578144170 con 0x7fb5780acee0 2020-11-16T22:18:50.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb5780acee0 0x7fb5780b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb5780acee0 0x7fb5780b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32946/0 (socket says 172.21.15.185:32946) 2020-11-16T22:18:50.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 -- 172.21.15.185:0/3282720786 learned_addr learned my addr 172.21.15.185:0/3282720786 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:50.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57dc26700 1 --2- 172.21.15.185:0/3282720786 >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 0x7fb5780b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 -- 172.21.15.185:0/3282720786 >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 msgr2=0x7fb5780b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:50.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 --2- 172.21.15.185:0/3282720786 >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 0x7fb5780b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 -- 172.21.15.185:0/3282720786 >> v1:172.21.15.96:6789/0 conn(0x7fb578148160 legacy=0x7fb578144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 -- 172.21.15.185:0/3282720786 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780b6ed0 con 0x7fb5780acee0 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57dc26700 1 --2- 172.21.15.185:0/3282720786 >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 0x7fb5780b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 --2- 172.21.15.185:0/3282720786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb5780b64b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fb578141bf0 tx=0x7fb570009290).ready entity=mon.1 client_cookie=20f439f1803ff969 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57cc24700 1 -- 172.21.15.185:0/3282720786 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000bbc0 con 0x7fb5780acee0 2020-11-16T22:18:50.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57cc24700 1 -- 172.21.15.185:0/3282720786 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57000bd20 con 0x7fb5780acee0 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57cc24700 1 -- 172.21.15.185:0/3282720786 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570011610 con 0x7fb5780acee0 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- 172.21.15.185:0/3282720786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 msgr2=0x7fb5780b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- 172.21.15.185:0/3282720786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb5780b64b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fb578141bf0 tx=0x7fb570009290).stop 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- 172.21.15.185:0/3282720786 shutdown_connections 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- 172.21.15.185:0/3282720786 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb5780b64b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- 172.21.15.185:0/3282720786 >> v2:172.21.15.96:3300/0 conn(0x7fb5780b7170 0x7fb5780b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- 172.21.15.185:0/3282720786 >> 172.21.15.185:0/3282720786 conn(0x7fb578067860 msgr2=0x7fb578067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:50.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- 172.21.15.185:0/3282720786 shutdown_connections 2020-11-16T22:18:50.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- 172.21.15.185:0/3282720786 wait complete. 2020-11-16T22:18:50.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 Processor -- start 2020-11-16T22:18:50.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- start start 2020-11-16T22:18:50.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb578153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 0x7fb5781463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780b0540 con 0x7fb578148160 2020-11-16T22:18:50.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780b9c00 con 0x7fb5780acee0 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57ec28700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb578153480 con 0x7fb5780b7170 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57dc26700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb578153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57d425700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57d425700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54626/0 (socket says 172.21.15.185:54626) 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57dc26700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb578153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32948/0 (socket says 172.21.15.185:32948) 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57d425700 1 -- 172.21.15.185:0/342397873 learned_addr learned my addr 172.21.15.185:0/342397873 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:50.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.509+0000 7fb57e427700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 0x7fb5781463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 -- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 msgr2=0x7fb578153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb578153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 -- 172.21.15.185:0/342397873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 msgr2=0x7fb5781463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 0x7fb5781463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 -- 172.21.15.185:0/342397873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578153720 con 0x7fb5780b7170 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57e427700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 0x7fb5781463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:50.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57d425700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fb57400c710 tx=0x7fb574007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574007020 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574007180 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5781539c0 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400e560 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5780b98f0 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5780b4360 con 0x7fb5780b7170 2020-11-16T22:18:50.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb574010070 con 0x7fb5780b7170 2020-11-16T22:18:50.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.513+0000 7fb565ffb700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 0x7fb550057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.517+0000 7fb57dc26700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 0x7fb550057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.517+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fb57406d060 con 0x7fb5780b7170 2020-11-16T22:18:50.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.517+0000 7fb57dc26700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 0x7fb550057520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fb56800b660 tx=0x7fb568008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.521+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb5780b4360 con 0x7fb5780b7170 2020-11-16T22:18:50.705 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.721+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fb5780b4360 con 0x7fb5780b7170 2020-11-16T22:18:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.733+0000 7fb565ffb700 1 -- 172.21.15.185:0/342397873 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v66) v1 ==== 189+0+0 (secure 0 0 0) 0x7fb5780b4360 con 0x7fb5780b7170 2020-11-16T22:18:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.741+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 msgr2=0x7fb550057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.741+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 0x7fb550057520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fb56800b660 tx=0x7fb568008040).stop 2020-11-16T22:18:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.741+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 msgr2=0x7fb5780bac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.741+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fb57400c710 tx=0x7fb574007b90).stop 2020-11-16T22:18:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 shutdown_connections 2020-11-16T22:18:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb550055060 0x7fb550057520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5780acee0 0x7fb578153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb578148160 0x7fb5781463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 --2- 172.21.15.185:0/342397873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5780b7170 0x7fb5780bac50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 >> 172.21.15.185:0/342397873 conn(0x7fb578067860 msgr2=0x7fb578143800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 shutdown_connections 2020-11-16T22:18:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.745+0000 7fb57ec28700 1 -- 172.21.15.185:0/342397873 wait complete. 2020-11-16T22:18:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 11 and data pool 12 2020-11-16T22:18:50.747 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:50.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 Processor -- start 2020-11-16T22:18:50.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 -- start start 2020-11-16T22:18:50.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f55f00b74a0 0x7f55f0148000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 0x7f55f00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f0147690 con 0x7f55f0144440 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f01478c0 con 0x7f55f0143fb0 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.873+0000 7f55f7e9c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f0147b60 con 0x7f55f00b74a0 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f55f00b74a0 0x7f55f0148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f55f00b74a0 0x7f55f0148000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32956/0 (socket says 172.21.15.185:32956) 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 -- 172.21.15.185:0/593142056 learned_addr learned my addr 172.21.15.185:0/593142056 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5437700 1 --2- 172.21.15.185:0/593142056 >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 0x7f55f00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 -- 172.21.15.185:0/593142056 >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 msgr2=0x7f55f00b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 --2- 172.21.15.185:0/593142056 >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 0x7f55f00b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 -- 172.21.15.185:0/593142056 >> v1:172.21.15.96:6789/0 conn(0x7f55f0144440 legacy=0x7f55f00b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 -- 172.21.15.185:0/593142056 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00b0640 con 0x7f55f00b74a0 2020-11-16T22:18:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5437700 1 --2- 172.21.15.185:0/593142056 >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 0x7f55f00b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f5c38700 1 --2- 172.21.15.185:0/593142056 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f00b74a0 0x7f55f0148000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f55f0062200 tx=0x7f55ec006eb0).ready entity=mon.1 client_cookie=89eb20eb2fd21df1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f4c36700 1 -- 172.21.15.185:0/593142056 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec009890 con 0x7f55f00b74a0 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f4c36700 1 -- 172.21.15.185:0/593142056 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec0099f0 con 0x7f55f00b74a0 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f4c36700 1 -- 172.21.15.185:0/593142056 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec009d70 con 0x7f55f00b74a0 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- 172.21.15.185:0/593142056 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f00b74a0 msgr2=0x7f55f0148000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/593142056 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f00b74a0 0x7f55f0148000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f55f0062200 tx=0x7f55ec006eb0).stop 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- 172.21.15.185:0/593142056 shutdown_connections 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/593142056 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f00b74a0 0x7f55f0148000 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/593142056 >> v2:172.21.15.96:3300/0 conn(0x7f55f0143fb0 0x7f55f00b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- 172.21.15.185:0/593142056 >> 172.21.15.185:0/593142056 conn(0x7f55f00682e0 msgr2=0x7f55f00ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- 172.21.15.185:0/593142056 shutdown_connections 2020-11-16T22:18:50.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- 172.21.15.185:0/593142056 wait complete. 2020-11-16T22:18:50.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 Processor -- start 2020-11-16T22:18:50.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.877+0000 7f55f7e9c700 1 -- start start 2020-11-16T22:18:50.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 0x7f55f0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 0x7f55f0149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 0x7f55f0140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00b1e90 con 0x7f55f0144440 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f0154210 con 0x7f55f0143fb0 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00b1950 con 0x7f55f00b74a0 2020-11-16T22:18:50.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5437700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 0x7f55f0149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5437700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 0x7f55f0149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32962/0 (socket says 172.21.15.185:32962) 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5437700 1 -- 172.21.15.185:0/138487950 learned_addr learned my addr 172.21.15.185:0/138487950 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 0x7f55f0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f6439700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 0x7f55f0140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 -- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 msgr2=0x7f55f0149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 0x7f55f0149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 -- 172.21.15.185:0/138487950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 msgr2=0x7f55f0140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:50.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 0x7f55f0140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 -- 172.21.15.185:0/138487950 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00b1bf0 con 0x7f55f00b74a0 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f6439700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 0x7f55f0140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f5c38700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 0x7f55f0153d20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f55f00ad160 tx=0x7f55ec015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011dd0 con 0x7f55f00b74a0 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec01e460 con 0x7f55f00b74a0 2020-11-16T22:18:50.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec01e800 con 0x7f55f00b74a0 2020-11-16T22:18:50.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00b2ac0 con 0x7f55f00b74a0 2020-11-16T22:18:50.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f00b13f0 con 0x7f55f00b74a0 2020-11-16T22:18:50.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f00b1640 con 0x7f55f00b74a0 2020-11-16T22:18:50.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.881+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f55ec01a040 con 0x7f55f00b74a0 2020-11-16T22:18:50.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.885+0000 7f55ddffb700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 0x7f55c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:50.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.885+0000 7f55f5437700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 0x7f55c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:50.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.885+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f55ec06f2a0 con 0x7f55f00b74a0 2020-11-16T22:18:50.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.885+0000 7f55f5437700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 0x7f55c8057520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f55f0142f80 tx=0x7f55e0006f90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:50.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:50.889+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f55ec045480 con 0x7f55f00b74a0 2020-11-16T22:18:51.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.045+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55f005d200 con 0x7f55f00b74a0 2020-11-16T22:18:51.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.045+0000 7f55ddffb700 1 -- 172.21.15.185:0/138487950 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16842 (secure 0 0 0) 0x7f55ec0301e0 con 0x7f55f00b74a0 2020-11-16T22:18:51.031 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:51.032 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":64,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:50.731590+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:51.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 msgr2=0x7f55c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 0x7f55c8057520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f55f0142f80 tx=0x7f55e0006f90).stop 2020-11-16T22:18:51.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 msgr2=0x7f55f0153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 0x7f55f0153d20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f55f00ad160 tx=0x7f55ec015000).stop 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 shutdown_connections 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f55c8055060 0x7f55c8057520 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f0144440 0x7f55f0140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f55f00b74a0 0x7f55f0153d20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 --2- 172.21.15.185:0/138487950 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f55f0143fb0 0x7f55f0149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 >> 172.21.15.185:0/138487950 conn(0x7f55f00682e0 msgr2=0x7f55f00b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 shutdown_connections 2020-11-16T22:18:51.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.049+0000 7f55f7e9c700 1 -- 172.21.15.185:0/138487950 wait complete. 2020-11-16T22:18:51.047 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:51.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.181+0000 7f751bfff700 1 Processor -- start 2020-11-16T22:18:51.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.181+0000 7f751bfff700 1 -- start start 2020-11-16T22:18:51.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f751c061e60 0x7f751c14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751bfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0abf90 con 0x7f751c0669a0 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751bfff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0ac230 con 0x7f751c0acc10 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751bfff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0ac4d0 con 0x7f751c061e60 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f751c0acc10 0x7f751c1432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54640/0 (socket says 172.21.15.185:54640) 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 -- 172.21.15.185:0/1761551393 learned_addr learned my addr 172.21.15.185:0/1761551393 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751b7fe700 1 --2- 172.21.15.185:0/1761551393 >> v2:172.21.15.185:3300/0 conn(0x7f751c061e60 0x7f751c14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425904678 0 0) 0x7f751c0ac230 con 0x7f751c0acc10 2020-11-16T22:18:51.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7500003240 con 0x7f751c0acc10 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4001655965 0 0) 0x7f7500003240 con 0x7f751c0acc10 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 >> v2:172.21.15.185:3300/0 conn(0x7f751c061e60 msgr2=0x7f751c14e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 --2- 172.21.15.185:0/1761551393 >> v2:172.21.15.185:3300/0 conn(0x7f751c061e60 0x7f751c14e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 >> v1:172.21.15.96:6789/0 conn(0x7f751c0669a0 legacy=0x7f751c066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0ac770 con 0x7f751c0acc10 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f75040014e0 con 0x7f751c0acc10 2020-11-16T22:18:51.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 >> v1:172.21.15.185:6789/0 conn(0x7f751c0acc10 legacy=0x7f751c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7500003730 0x7f7500003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f7500004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500004e50 0x7f750000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0ac4d0 con 0x7f7500004e50 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500004260 con 0x7f75000044e0 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750000bc20 con 0x7f7500003730 2020-11-16T22:18:51.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751affd700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7500003730 0x7f7500003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f7500004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751b7fe700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500004e50 0x7f750000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 -- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7500003730 msgr2=0x7f7500003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7500003730 0x7f7500003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 -- 172.21.15.185:0/1761551393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500004e50 msgr2=0x7f750000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500004e50 0x7f750000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 -- 172.21.15.185:0/1761551393 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500003240 con 0x7f75000044e0 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751affd700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7500003730 0x7f7500003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:51.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751b7fe700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7500004e50 0x7f750000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751b7fe700 1 -- 172.21.15.185:0/1761551393 reap_dead start 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f75127fc700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f7500004960 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f750400c440 tx=0x7f751c0b21b0).ready entity=mon.1 client_cookie=47d968af056b1fe2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040070d0 con 0x7f75000044e0 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504007230 con 0x7f75000044e0 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.185+0000 7f751a7fc700 1 -- 172.21.15.185:0/1761551393 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040055f0 con 0x7f75000044e0 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 msgr2=0x7f7500004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f7500004960 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f750400c440 tx=0x7f751c0b21b0).stop 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- 172.21.15.185:0/1761551393 shutdown_connections 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 --2- 172.21.15.185:0/1761551393 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f7500004960 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- 172.21.15.185:0/1761551393 >> 172.21.15.185:0/1761551393 conn(0x7f751c060130 msgr2=0x7f751c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- 172.21.15.185:0/1761551393 shutdown_connections 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- 172.21.15.185:0/1761551393 wait complete. 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 Processor -- start 2020-11-16T22:18:51.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- start start 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 0x7f751c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 0x7f751c13cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c156f50 con 0x7f751c141650 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c148300 con 0x7f75000044e0 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c1571b0 con 0x7f751c147680 2020-11-16T22:18:51.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f75127fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 0x7f751c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32982/0 (socket says 172.21.15.185:32982) 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 -- 172.21.15.185:0/3396540341 learned_addr learned my addr 172.21.15.185:0/3396540341 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751b7fe700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 0x7f751c13cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 -- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 msgr2=0x7f751c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 0x7f751c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 -- 172.21.15.185:0/3396540341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 msgr2=0x7f751c13cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 0x7f751c13cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 -- 172.21.15.185:0/3396540341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c1485a0 con 0x7f75000044e0 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f75127fc700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 0x7f751c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751b7fe700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 0x7f751c13cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.189+0000 7f751affd700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f750c007570 tx=0x7f750c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c007ea0 con 0x7f75000044e0 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c00d8f0 con 0x7f75000044e0 2020-11-16T22:18:51.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c147f80 con 0x7f75000044e0 2020-11-16T22:18:51.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00dc70 con 0x7f75000044e0 2020-11-16T22:18:51.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c13ee40 con 0x7f75000044e0 2020-11-16T22:18:51.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c148110 con 0x7f75000044e0 2020-11-16T22:18:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f750c01a030 con 0x7f75000044e0 2020-11-16T22:18:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 0x7f74ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f750c06d750 con 0x7f75000044e0 2020-11-16T22:18:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75127fc700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 0x7f74ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.193+0000 7f75127fc700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 0x7f74ec057520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f751c0b0fe0 tx=0x7f7504013000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.201+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f750c043950 con 0x7f75000044e0 2020-11-16T22:18:51.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f751c05d200 con 0x7f75000044e0 2020-11-16T22:18:51.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f75137fe700 1 -- 172.21.15.185:0/3396540341 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16842 (secure 0 0 0) 0x7f750c0261e0 con 0x7f75000044e0 2020-11-16T22:18:51.339 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:51.339 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":64,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:50.731590+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:51.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 msgr2=0x7f74ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 0x7f74ec057520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f751c0b0fe0 tx=0x7f7504013000).stop 2020-11-16T22:18:51.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 msgr2=0x7f751c148ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f750c007570 tx=0x7f750c009040).stop 2020-11-16T22:18:51.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.353+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 shutdown_connections 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f74ec055060 0x7f74ec057520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f75000044e0 0x7f751c148ac0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751c141650 0x7f751c13cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 --2- 172.21.15.185:0/3396540341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f751c147680 0x7f751c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 >> 172.21.15.185:0/3396540341 conn(0x7f751c060130 msgr2=0x7f751c142590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 shutdown_connections 2020-11-16T22:18:51.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.357+0000 7f751bfff700 1 -- 172.21.15.185:0/3396540341 wait complete. 2020-11-16T22:18:51.355 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:18:51.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 Processor -- start 2020-11-16T22:18:51.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- start start 2020-11-16T22:18:51.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f400c0b3700 0x7f400c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 0x7f400c0b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c060550 con 0x7f400c060a10 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0602b0 con 0x7f400c0b3700 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0b1710 con 0x7f400c061560 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f400c0b3700 0x7f400c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f400c0b3700 0x7f400c0b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:32992/0 (socket says 172.21.15.185:32992) 2020-11-16T22:18:51.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 -- 172.21.15.185:0/3713158784 learned_addr learned my addr 172.21.15.185:0/3713158784 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f400bfff700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f40117d7700 1 --2- 172.21.15.185:0/3713158784 >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 0x7f400c0b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 -- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 msgr2=0x7f400c143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 -- 172.21.15.185:0/3713158784 >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 msgr2=0x7f400c0b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 --2- 172.21.15.185:0/3713158784 >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 0x7f400c0b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 -- 172.21.15.185:0/3713158784 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0b19b0 con 0x7f400c0b3700 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f40117d7700 1 --2- 172.21.15.185:0/3713158784 >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 0x7f400c0b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4010fd6700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c0b3700 0x7f400c0b3ae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f400c061d80 tx=0x7f400400a430).ready entity=mon.1 client_cookie=f4d403bc43365742 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f400b7fe700 1 -- 172.21.15.185:0/3713158784 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400ce60 con 0x7f400c0b3700 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f400b7fe700 1 -- 172.21.15.185:0/3713158784 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4004010410 con 0x7f400c0b3700 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f400b7fe700 1 -- 172.21.15.185:0/3713158784 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004010790 con 0x7f400c0b3700 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c0b3700 msgr2=0x7f400c0b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c0b3700 0x7f400c0b3ae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f400c061d80 tx=0x7f400400a430).stop 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- 172.21.15.185:0/3713158784 shutdown_connections 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- 172.21.15.185:0/3713158784 >> v2:172.21.15.96:3300/0 conn(0x7f400c060a10 0x7f400c0b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c0b3700 0x7f400c0b3ae0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 --2- 172.21.15.185:0/3713158784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- 172.21.15.185:0/3713158784 >> 172.21.15.185:0/3713158784 conn(0x7f400c063810 msgr2=0x7f400c063df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- 172.21.15.185:0/3713158784 shutdown_connections 2020-11-16T22:18:51.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.493+0000 7f4011fd8700 1 -- 172.21.15.185:0/3713158784 wait complete. 2020-11-16T22:18:51.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 Processor -- start 2020-11-16T22:18:51.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- start start 2020-11-16T22:18:51.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 0x7f400c1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c1467d0 0x7f400c148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c13c570 con 0x7f400c0b3700 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1557e0 con 0x7f400c1467d0 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c146c50 con 0x7f400c061560 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54672/0 (socket says 172.21.15.185:54672) 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 -- 172.21.15.185:0/72969873 learned_addr learned my addr 172.21.15.185:0/72969873 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:51.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f40117d7700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c1467d0 0x7f400c148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f400bfff700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 0x7f400c1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 -- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c1467d0 msgr2=0x7f400c148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c1467d0 0x7f400c148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 -- 172.21.15.185:0/72969873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 msgr2=0x7f400c1462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 0x7f400c1462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 -- 172.21.15.185:0/72969873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c1552a0 con 0x7f400c061560 2020-11-16T22:18:51.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f400bfff700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 0x7f400c1462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4010fd6700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f400c0603f0 tx=0x7f400c0abef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400401a750 con 0x7f400c061560 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400401a8b0 con 0x7f400c061560 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c155540 con 0x7f400c061560 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400401ac30 con 0x7f400c061560 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c0b0580 con 0x7f400c061560 2020-11-16T22:18:51.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.497+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f400400e040 con 0x7f400c061560 2020-11-16T22:18:51.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.501+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c05d200 con 0x7f400c061560 2020-11-16T22:18:51.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.501+0000 7f4008ff9700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 0x7f3fe8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.501+0000 7f400bfff700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 0x7f3fe8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.501+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f400406faa0 con 0x7f400c061560 2020-11-16T22:18:51.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.501+0000 7f400bfff700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 0x7f3fe8057520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3ff4006bd0 tx=0x7f3ff4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.505+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4004045bf0 con 0x7f400c061560 2020-11-16T22:18:51.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.709+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f400c144aa0 con 0x7f400c061560 2020-11-16T22:18:51.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4008ff9700 1 -- 172.21.15.185:0/72969873 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4004049210 con 0x7f400c061560 2020-11-16T22:18:51.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 msgr2=0x7f3fe8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 0x7f3fe8057520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f3ff4006bd0 tx=0x7f3ff4008040).stop 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 msgr2=0x7f400c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f400c0603f0 tx=0x7f400c0abef0).stop 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 shutdown_connections 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3fe8055060 0x7f3fe8057520 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f400c1467d0 0x7f400c148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f400c061560 0x7f400c144db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 --2- 172.21.15.185:0/72969873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c0b3700 0x7f400c1462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 >> 172.21.15.185:0/72969873 conn(0x7f400c060a10 msgr2=0x7f400c143550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 shutdown_connections 2020-11-16T22:18:51.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.733+0000 7f4011fd8700 1 -- 172.21.15.185:0/72969873 wait complete. 2020-11-16T22:18:51.735 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:51.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 Processor -- start 2020-11-16T22:18:51.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- start start 2020-11-16T22:18:51.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 0x7fdf5c144b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0b82e0 con 0x7fdf5c0b2cf0 2020-11-16T22:18:51.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c145080 con 0x7fdf5c0b4720 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0b7650 con 0x7fdf5c0b2380 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54680/0 (socket says 172.21.15.185:54680) 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 -- 172.21.15.185:0/2126477557 learned_addr learned my addr 172.21.15.185:0/2126477557 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 -- 172.21.15.185:0/2126477557 >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 msgr2=0x7fdf5c144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5affd700 1 --2- 172.21.15.185:0/2126477557 >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 0x7fdf5c144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 --2- 172.21.15.185:0/2126477557 >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 0x7fdf5c144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 -- 172.21.15.185:0/2126477557 >> v1:172.21.15.96:6789/0 conn(0x7fdf5c0b2cf0 legacy=0x7fdf5c1476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 -- 172.21.15.185:0/2126477557 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0b78f0 con 0x7fdf5c0b2380 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5affd700 1 --2- 172.21.15.185:0/2126477557 >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 0x7fdf5c144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf527fc700 1 --2- 172.21.15.185:0/2126477557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdf44006fd0 tx=0x7fdf44006eb0).ready entity=mon.2 client_cookie=786d5565c24fa542 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5a7fc700 1 -- 172.21.15.185:0/2126477557 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44009890 con 0x7fdf5c0b2380 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5a7fc700 1 -- 172.21.15.185:0/2126477557 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf440099f0 con 0x7fdf5c0b2380 2020-11-16T22:18:51.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5a7fc700 1 -- 172.21.15.185:0/2126477557 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44009d70 con 0x7fdf5c0b2380 2020-11-16T22:18:51.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- 172.21.15.185:0/2126477557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 msgr2=0x7fdf5c0b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/2126477557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdf44006fd0 tx=0x7fdf44006eb0).stop 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- 172.21.15.185:0/2126477557 shutdown_connections 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/2126477557 >> v2:172.21.15.96:3300/0 conn(0x7fdf5c0b4720 0x7fdf5c144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/2126477557 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b2380 0x7fdf5c0b2800 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.865+0000 7fdf5bfff700 1 -- 172.21.15.185:0/2126477557 >> 172.21.15.185:0/2126477557 conn(0x7fdf5c067860 msgr2=0x7fdf5c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.869+0000 7fdf5bfff700 1 -- 172.21.15.185:0/2126477557 shutdown_connections 2020-11-16T22:18:51.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.869+0000 7fdf5bfff700 1 -- 172.21.15.185:0/2126477557 wait complete. 2020-11-16T22:18:51.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.869+0000 7fdf5bfff700 1 Processor -- start 2020-11-16T22:18:51.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.869+0000 7fdf5bfff700 1 -- start start 2020-11-16T22:18:51.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 0x7fdf5c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 0x7fdf5c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0aece0 con 0x7fdf5c0b2cf0 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c153e50 con 0x7fdf5c0b2380 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c1540d0 con 0x7fdf5c0b4720 2020-11-16T22:18:51.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 0x7fdf5c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54688/0 (socket says 172.21.15.185:54688) 2020-11-16T22:18:51.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5affd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 0x7fdf5c0af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33010/0 (socket says 172.21.15.185:33010) 2020-11-16T22:18:51.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 -- 172.21.15.185:0/3492964161 learned_addr learned my addr 172.21.15.185:0/3492964161 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:51.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf527fc700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 0x7fdf5c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 -- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 msgr2=0x7fdf5c0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 0x7fdf5c0af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 -- 172.21.15.185:0/3492964161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 msgr2=0x7fdf5c0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 0x7fdf5c0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 -- 172.21.15.185:0/3492964161 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c154370 con 0x7fdf5c0b4720 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf527fc700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 0x7fdf5c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5b7fe700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fdf5400c890 tx=0x7fdf54007f20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400a5c0 con 0x7fdf5c0b4720 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400a740 con 0x7fdf5c0b4720 2020-11-16T22:18:51.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0ad830 con 0x7fdf5c0b4720 2020-11-16T22:18:51.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400ab00 con 0x7fdf5c0b4720 2020-11-16T22:18:51.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.873+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c12b1d0 con 0x7fdf5c0b4720 2020-11-16T22:18:51.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdf54010070 con 0x7fdf5c0b4720 2020-11-16T22:18:51.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c05d200 con 0x7fdf5c0b4720 2020-11-16T22:18:51.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf537fe700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 0x7fdf38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:51.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf5406db40 con 0x7fdf5c0b4720 2020-11-16T22:18:51.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf5affd700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 0x7fdf38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:51.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.877+0000 7fdf5affd700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 0x7fdf38057520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fdf4c00c600 tx=0x7fdf4c008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:51.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:51.881+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdf54043da0 con 0x7fdf5c0b4720 2020-11-16T22:18:52.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.085+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf5c0aee20 con 0x7fdf5c0b4720 2020-11-16T22:18:52.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.085+0000 7fdf537fe700 1 -- 172.21.15.185:0/3492964161 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+2489 (secure 0 0 0) 0x7fdf540473c0 con 0x7fdf5c0b4720 2020-11-16T22:18:52.072 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:52.072 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64},{"gid":6188,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:51.735017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6642},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:18:52.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 msgr2=0x7fdf38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 0x7fdf38057520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fdf4c00c600 tx=0x7fdf4c008040).stop 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 msgr2=0x7fdf5c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fdf5400c890 tx=0x7fdf54007f20).stop 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 shutdown_connections 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf38055060 0x7fdf38057520 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf5c0b2380 0x7fdf5c0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf5c0b4720 0x7fdf5c156a70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 --2- 172.21.15.185:0/3492964161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf5c0b2cf0 0x7fdf5c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 >> 172.21.15.185:0/3492964161 conn(0x7fdf5c067860 msgr2=0x7fdf5c0bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 shutdown_connections 2020-11-16T22:18:52.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.089+0000 7fdf5bfff700 1 -- 172.21.15.185:0/3492964161 wait complete. 2020-11-16T22:18:52.076 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 67 2020-11-16T22:18:52.087 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:52.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 Processor -- start 2020-11-16T22:18:52.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- start start 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f15e80b7510 0x7f15e80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e8060130 0x7f15e8060ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 0x7f15e813bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e8060fa0 con 0x7f15e813ba80 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e8141ec0 con 0x7f15e80b7510 2020-11-16T22:18:52.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81420f0 con 0x7f15e8060130 2020-11-16T22:18:52.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f15e80b7510 0x7f15e80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ecb0c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e8060130 0x7f15e8060ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f15e80b7510 0x7f15e80b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33018/0 (socket says 172.21.15.185:33018) 2020-11-16T22:18:52.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 -- 172.21.15.185:0/16196337 learned_addr learned my addr 172.21.15.185:0/16196337 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15edb0e700 1 --2- 172.21.15.185:0/16196337 >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 0x7f15e813bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 -- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e8060130 msgr2=0x7f15e8060ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:52.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 --2- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e8060130 0x7f15e8060ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 -- 172.21.15.185:0/16196337 >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 msgr2=0x7f15e813bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:52.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 --2- 172.21.15.185:0/16196337 >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 0x7f15e813bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 -- 172.21.15.185:0/16196337 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8142390 con 0x7f15e80b7510 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15edb0e700 1 --2- 172.21.15.185:0/16196337 >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 0x7f15e813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ed30d700 1 --2- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e80b7510 0x7f15e80b78f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f15e8062250 tx=0x7f15d8009290).ready entity=mon.1 client_cookie=5bf47b512fc7befb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15e7fff700 1 -- 172.21.15.185:0/16196337 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800bbc0 con 0x7f15e80b7510 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15e7fff700 1 -- 172.21.15.185:0/16196337 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d800bd20 con 0x7f15e80b7510 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15e7fff700 1 -- 172.21.15.185:0/16196337 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8011610 con 0x7f15e80b7510 2020-11-16T22:18:52.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e80b7510 msgr2=0x7f15e80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e80b7510 0x7f15e80b78f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f15e8062250 tx=0x7f15d8009290).stop 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- 172.21.15.185:0/16196337 shutdown_connections 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- 172.21.15.185:0/16196337 >> v2:172.21.15.96:3300/0 conn(0x7f15e813ba80 0x7f15e813bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e80b7510 0x7f15e80b78f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 --2- 172.21.15.185:0/16196337 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e8060130 0x7f15e8060ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- 172.21.15.185:0/16196337 >> 172.21.15.185:0/16196337 conn(0x7f15e805ba80 msgr2=0x7f15e805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- 172.21.15.185:0/16196337 shutdown_connections 2020-11-16T22:18:52.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.221+0000 7f15ee30f700 1 -- 172.21.15.185:0/16196337 wait complete. 2020-11-16T22:18:52.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 Processor -- start 2020-11-16T22:18:52.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- start start 2020-11-16T22:18:52.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e813ba80 0x7f15e8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e80afbd0 0x7f15e8146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8144520 con 0x7f15e80afbd0 2020-11-16T22:18:52.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e813f3b0 con 0x7f15e8060130 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81548b0 con 0x7f15e813ba80 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33024/0 (socket says 172.21.15.185:33024) 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 -- 172.21.15.185:0/1463851598 learned_addr learned my addr 172.21.15.185:0/1463851598 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ecb0c700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e813ba80 0x7f15e8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15edb0e700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e80afbd0 0x7f15e8146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 -- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e813ba80 msgr2=0x7f15e8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e813ba80 0x7f15e8155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 -- 172.21.15.185:0/1463851598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e80afbd0 msgr2=0x7f15e8146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e80afbd0 0x7f15e8146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 -- 172.21.15.185:0/1463851598 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8154b50 con 0x7f15e8060130 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ed30d700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f15e8141db0 tx=0x7f15d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80032a0 con 0x7f15e8060130 2020-11-16T22:18:52.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d801b5e0 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d801b960 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e8154df0 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.225+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8061990 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e805d200 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f15d8017070 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15e57fa700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15d806fe50 con 0x7f15e8060130 2020-11-16T22:18:52.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15ecb0c700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.229+0000 7f15ecb0c700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 0x7f15c0057520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f15e813ed30 tx=0x7f15e0006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.237+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f15d8045fa0 con 0x7f15e8060130 2020-11-16T22:18:52.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.389+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15e805c720 con 0x7f15e8060130 2020-11-16T22:18:52.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15e57fa700 1 -- 172.21.15.185:0/1463851598 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f15d8071070 con 0x7f15e8060130 2020-11-16T22:18:52.378 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:52.379 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":65,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:51.735891+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:52.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 msgr2=0x7f15c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 0x7f15c0057520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f15e813ed30 tx=0x7f15e0006df0).stop 2020-11-16T22:18:52.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 msgr2=0x7f15e813ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f15e8141db0 tx=0x7f15d8016000).stop 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 shutdown_connections 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f15c0055060 0x7f15c0057520 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15e8060130 0x7f15e813ed80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e80afbd0 0x7f15e8146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 --2- 172.21.15.185:0/1463851598 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15e813ba80 0x7f15e8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 >> 172.21.15.185:0/1463851598 conn(0x7f15e805ba80 msgr2=0x7f15e813aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 shutdown_connections 2020-11-16T22:18:52.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.393+0000 7f15ee30f700 1 -- 172.21.15.185:0/1463851598 wait complete. 2020-11-16T22:18:52.393 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:18:52.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.537+0000 7f839da61700 1 Processor -- start 2020-11-16T22:18:52.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.537+0000 7f839da61700 1 -- start start 2020-11-16T22:18:52.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83980b0720 0x7f83981455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b2740 0x7f8398145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8398067cd0 con 0x7f839805ba80 2020-11-16T22:18:52.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980b2000 con 0x7f83980b0720 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980b22a0 con 0x7f83980b2740 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83980b0720 0x7f83981455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83977fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b2740 0x7f8398145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f83980b0720 0x7f83981455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33032/0 (socket says 172.21.15.185:33032) 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 -- 172.21.15.185:0/1457803323 learned_addr learned my addr 172.21.15.185:0/1457803323 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83967fc700 1 --1- 172.21.15.185:0/1457803323 >> v1:172.21.15.185:6789/0 conn(0x7f839805ba80 0x7f8398141430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54704/0 (socket says 172.21.15.185:54704) 2020-11-16T22:18:52.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 -- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b2740 msgr2=0x7f8398145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:52.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 --2- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b2740 0x7f8398145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 -- 172.21.15.185:0/1457803323 >> v1:172.21.15.185:6789/0 conn(0x7f839805ba80 legacy=0x7f8398141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 -- 172.21.15.185:0/1457803323 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980b11f0 con 0x7f83980b0720 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8396ffd700 1 --2- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b0720 0x7f83981455e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f8388006fd0 tx=0x7f8388006eb0).ready entity=mon.1 client_cookie=4af2a4dbd4818f12 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8395ffb700 1 -- 172.21.15.185:0/1457803323 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388009810 con 0x7f83980b0720 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8395ffb700 1 -- 172.21.15.185:0/1457803323 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388009970 con 0x7f83980b0720 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f8395ffb700 1 -- 172.21.15.185:0/1457803323 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388009cf0 con 0x7f83980b0720 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b0720 msgr2=0x7f83981455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b0720 0x7f83981455e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f8388006fd0 tx=0x7f8388006eb0).stop 2020-11-16T22:18:52.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- 172.21.15.185:0/1457803323 shutdown_connections 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b2740 0x7f8398145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- 172.21.15.185:0/1457803323 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b0720 0x7f83981455e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- 172.21.15.185:0/1457803323 >> 172.21.15.185:0/1457803323 conn(0x7f8398063ff0 msgr2=0x7f83980643d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- 172.21.15.185:0/1457803323 shutdown_connections 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- 172.21.15.185:0/1457803323 wait complete. 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 Processor -- start 2020-11-16T22:18:52.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- start start 2020-11-16T22:18:52.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 0x7f8398155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 0x7f8398141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398146420 con 0x7f839805ba80 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398111d60 con 0x7f83980b2740 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f839da61700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8398154e40 con 0x7f83980b0720 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83967fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 0x7f8398141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83977fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83977fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33040/0 (socket says 172.21.15.185:33040) 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.541+0000 7f83967fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 0x7f8398141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54718/0 (socket says 172.21.15.185:54718) 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 -- 172.21.15.185:0/2278897730 learned_addr learned my addr 172.21.15.185:0/2278897730 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f8396ffd700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 0x7f8398155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 -- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 msgr2=0x7f8398141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 0x7f8398141450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 -- 172.21.15.185:0/2278897730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 msgr2=0x7f8398155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 0x7f8398155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 -- 172.21.15.185:0/2278897730 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83981550e0 con 0x7f83980b2740 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f8396ffd700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 0x7f8398155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f83977fe700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f839000ef20 tx=0x7f83900091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390009690 con 0x7f83980b2740 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8398155380 con 0x7f83980b2740 2020-11-16T22:18:52.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000cbc0 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900123f0 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83981460a0 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839805bee0 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8390022070 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 0x7f8374057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f839006e5f0 con 0x7f83980b2740 2020-11-16T22:18:52.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.545+0000 7f8396ffd700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 0x7f8374057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.549+0000 7f8396ffd700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 0x7f8374057520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f8398145ff0 tx=0x7f8388015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.553+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8390044850 con 0x7f83980b2740 2020-11-16T22:18:52.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.761+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f839805d4e0 con 0x7f83980b2740 2020-11-16T22:18:52.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.769+0000 7f838effd700 1 -- 172.21.15.185:0/2278897730 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8390016070 con 0x7f83980b2740 2020-11-16T22:18:52.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 msgr2=0x7f8374057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 0x7f8374057520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f8398145ff0 tx=0x7f8388015000).stop 2020-11-16T22:18:52.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 msgr2=0x7f83981327f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f839000ef20 tx=0x7f83900091d0).stop 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 shutdown_connections 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8374055060 0x7f8374057520 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f83980b2740 0x7f83981327f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f839805ba80 0x7f8398155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 --2- 172.21.15.185:0/2278897730 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f83980b0720 0x7f8398141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 >> 172.21.15.185:0/2278897730 conn(0x7f8398063ff0 msgr2=0x7f83980ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 shutdown_connections 2020-11-16T22:18:52.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.773+0000 7f839da61700 1 -- 172.21.15.185:0/2278897730 wait complete. 2020-11-16T22:18:52.762 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:18:52.776 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 Processor -- start 2020-11-16T22:18:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 -- start start 2020-11-16T22:18:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d100b8820 0x7f0d100af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100af130 con 0x7f0d10141910 2020-11-16T22:18:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d10141d80 con 0x7f0d100b8820 2020-11-16T22:18:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d14b02700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100af360 con 0x7f0d100b30e0 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54722/0 (socket says 172.21.15.185:54722) 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 -- 172.21.15.185:0/411344017 learned_addr learned my addr 172.21.15.185:0/411344017 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0f7fe700 1 --2- 172.21.15.185:0/411344017 >> v2:172.21.15.96:3300/0 conn(0x7f0d100b8820 0x7f0d100af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 -- 172.21.15.185:0/411344017 >> v2:172.21.15.96:3300/0 conn(0x7f0d100b8820 msgr2=0x7f0d100af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 --2- 172.21.15.185:0/411344017 >> v2:172.21.15.96:3300/0 conn(0x7f0d100b8820 0x7f0d100af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 -- 172.21.15.185:0/411344017 >> v1:172.21.15.96:6789/0 conn(0x7f0d10141910 legacy=0x7f0d101454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 -- 172.21.15.185:0/411344017 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100af600 con 0x7f0d100b30e0 2020-11-16T22:18:52.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0effd700 1 --2- 172.21.15.185:0/411344017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f0d00006fd0 tx=0x7f0d00006eb0).ready entity=mon.2 client_cookie=b5fd9eee63a6c44f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/411344017 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000098d0 con 0x7f0d100b30e0 2020-11-16T22:18:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/411344017 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00009a30 con 0x7f0d100b30e0 2020-11-16T22:18:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.909+0000 7f0d0e7fc700 1 -- 172.21.15.185:0/411344017 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00009db0 con 0x7f0d100b30e0 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- 172.21.15.185:0/411344017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 msgr2=0x7f0d10141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- 172.21.15.185:0/411344017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f0d00006fd0 tx=0x7f0d00006eb0).stop 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- 172.21.15.185:0/411344017 shutdown_connections 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- 172.21.15.185:0/411344017 >> v2:172.21.15.96:3300/0 conn(0x7f0d100b8820 0x7f0d100af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- 172.21.15.185:0/411344017 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b30e0 0x7f0d10141420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- 172.21.15.185:0/411344017 >> 172.21.15.185:0/411344017 conn(0x7f0d100645d0 msgr2=0x7f0d100649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- 172.21.15.185:0/411344017 shutdown_connections 2020-11-16T22:18:52.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- 172.21.15.185:0/411344017 wait complete. 2020-11-16T22:18:52.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 Processor -- start 2020-11-16T22:18:52.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- start start 2020-11-16T22:18:52.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 0x7f0d100ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 0x7f0d100b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d10141910 0x7f0d10143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10144980 con 0x7f0d10141910 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1013e9f0 con 0x7f0d100b30e0 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d14b02700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1013ec90 con 0x7f0d100b8820 2020-11-16T22:18:52.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 0x7f0d100ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 0x7f0d100b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 0x7f0d100ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33052/0 (socket says 172.21.15.185:33052) 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0f7fe700 1 -- 172.21.15.185:0/224139421 learned_addr learned my addr 172.21.15.185:0/224139421 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0ffff700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d10141910 0x7f0d10143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 -- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 msgr2=0x7f0d100ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 0x7f0d100ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 -- 172.21.15.185:0/224139421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d10141910 msgr2=0x7f0d10143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d10141910 0x7f0d10143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.913+0000 7f0d0effd700 1 -- 172.21.15.185:0/224139421 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10143790 con 0x7f0d100b8820 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d0effd700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 0x7f0d100b7890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0d00006fd0 tx=0x7f0d00015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00011c80 con 0x7f0d100b8820 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00011de0 con 0x7f0d100b8820 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100b7d80 con 0x7f0d100b8820 2020-11-16T22:18:52.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0001e670 con 0x7f0d100b8820 2020-11-16T22:18:52.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1013ef30 con 0x7f0d100b8820 2020-11-16T22:18:52.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10143970 con 0x7f0d100b8820 2020-11-16T22:18:52.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0d00025030 con 0x7f0d100b8820 2020-11-16T22:18:52.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.917+0000 7f0d077fe700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 0x7f0cec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:52.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.921+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d0006f4a0 con 0x7f0d100b8820 2020-11-16T22:18:52.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.921+0000 7f0d0f7fe700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 0x7f0cec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:52.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.921+0000 7f0d0f7fe700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 0x7f0cec057520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0cf800b620 tx=0x7f0cf8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:52.925+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0d10143970 con 0x7f0d100b8820 2020-11-16T22:18:53.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.125+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d10143970 con 0x7f0d100b8820 2020-11-16T22:18:53.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.125+0000 7f0d077fe700 1 -- 172.21.15.185:0/224139421 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f0d10143970 con 0x7f0d100b8820 2020-11-16T22:18:53.113 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:53.113 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64},{"gid":6188,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:51.735017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6642},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:18:53.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 msgr2=0x7f0cec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:53.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 0x7f0cec057520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0cf800b620 tx=0x7f0cf8008040).stop 2020-11-16T22:18:53.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 msgr2=0x7f0d100b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 0x7f0d100b7890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0d00006fd0 tx=0x7f0d00015000).stop 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 shutdown_connections 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0cec055060 0x7f0cec057520 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0d100b30e0 0x7f0d100ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d10141910 0x7f0d10143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 --2- 172.21.15.185:0/224139421 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0d100b8820 0x7f0d100b7890 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:53.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 >> 172.21.15.185:0/224139421 conn(0x7f0d100645d0 msgr2=0x7f0d1013df90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:53.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 shutdown_connections 2020-11-16T22:18:53.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:53.129+0000 7f0d14b02700 1 -- 172.21.15.185:0/224139421 wait complete. 2020-11-16T22:18:53.117 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 67 2020-11-16T22:18:53.430 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:18:50.731986+0000', 'modified': '2020-11-16T22:18:51.735017+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6642}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6642': {'gid': 6642, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6833/974001631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 974001631}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 974001631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:18:53.431 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:18:53.432 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:18:53.573 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6642,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.8524767560000002,"mdsmap_epoch":67,"osdmap_epoch":65,"osdmap_epoch_barrier":64,"uptime":8.1074411099999999} 2020-11-16T22:18:53.574 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6642, "mdsmap_epoch": 67, "osdmap_epoch": 65, "osdmap_epoch_barrier": 64, "rank_uptime": 2.852476756, "state": "up:active", "uptime": 8.10744111, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:18:53.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:18:53.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:18:53.575 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:18:53.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:18:53.576 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:18:53.576 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:18:53.576 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:18:53.576 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:18:53.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.577 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:18:53.594 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:18:53.594 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:18:53.594 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:18:53.594 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:18:53.595 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:18:53.596 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.597 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:18:53.598 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:18:53.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.600 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:18:53.680 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:18:53.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.684 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:18:53.698 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:18:53.698 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:18:53.698 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:18:53.699 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:18:53.699 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:18:53.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.700 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:18:53.700 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:18:53.701 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:18:53.701 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:18:53.701 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:18:53.702 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:18:53.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.815 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:18:53.832 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:18:53.832 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:18:53.833 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:18:53.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:53.834 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:18:53.865 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:18:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.013+0000 7fbb82d1b700 1 Processor -- start 2020-11-16T22:18:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.013+0000 7fbb82d1b700 1 -- start start 2020-11-16T22:18:54.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 0x7fbb7c143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c067510 con 0x7fbb7c067070 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0677b0 con 0x7fbb7c0b7470 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b17a0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb81d19700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb81d19700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33060/0 (socket says 172.21.15.185:33060) 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 0x7fbb7c143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb81d19700 1 -- 172.21.15.185:0/91532423 learned_addr learned my addr 172.21.15.185:0/91532423 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 -- 172.21.15.185:0/91532423 >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 msgr2=0x7fbb7c0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 --2- 172.21.15.185:0/91532423 >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 -- 172.21.15.185:0/91532423 >> v1:172.21.15.185:6789/0 conn(0x7fbb7c067070 legacy=0x7fbb7c0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 -- 172.21.15.185:0/91532423 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0b1a40 con 0x7fbb7c0682b0 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb80d17700 1 -- 172.21.15.185:0/91532423 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831706980 0 0) 0x7fbb7c067510 con 0x7fbb7c067070 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb81d19700 1 --2- 172.21.15.185:0/91532423 >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb8251a700 1 --2- 172.21.15.185:0/91532423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 0x7fbb7c143a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fbb7400cb30 tx=0x7fbb74006e80).ready entity=mon.2 client_cookie=327549a89a9c90c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb80d17700 1 -- 172.21.15.185:0/91532423 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74014070 con 0x7fbb7c0682b0 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb80d17700 1 -- 172.21.15.185:0/91532423 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7400a840 con 0x7fbb7c0682b0 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb80d17700 1 -- 172.21.15.185:0/91532423 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400abc0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- 172.21.15.185:0/91532423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 msgr2=0x7fbb7c143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/91532423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 0x7fbb7c143a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fbb7400cb30 tx=0x7fbb74006e80).stop 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- 172.21.15.185:0/91532423 shutdown_connections 2020-11-16T22:18:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/91532423 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0682b0 0x7fbb7c143a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/91532423 >> v2:172.21.15.185:3300/0 conn(0x7fbb7c0b7470 0x7fbb7c0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- 172.21.15.185:0/91532423 >> 172.21.15.185:0/91532423 conn(0x7fbb7c0637b0 msgr2=0x7fbb7c0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.017+0000 7fbb82d1b700 1 -- 172.21.15.185:0/91532423 shutdown_connections 2020-11-16T22:18:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- 172.21.15.185:0/91532423 wait complete. 2020-11-16T22:18:54.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 Processor -- start 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- start start 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb7c067070 0x7fbb7c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0ad390 con 0x7fbb7c067070 2020-11-16T22:18:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c05d4e0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c13d840 con 0x7fbb7c0b7470 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb8251a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33068/0 (socket says 172.21.15.185:33068) 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb8251a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54746/0 (socket says 172.21.15.185:54746) 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 -- 172.21.15.185:0/2612632366 learned_addr learned my addr 172.21.15.185:0/2612632366 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81d19700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb7c067070 0x7fbb7c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 -- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 msgr2=0x7fbb7c0ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 -- 172.21.15.185:0/2612632366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb7c067070 msgr2=0x7fbb7c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb7c067070 0x7fbb7c13f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 -- 172.21.15.185:0/2612632366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c1417d0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb8251a700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb81518700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fbb6c00c760 tx=0x7fbb6c007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00e490 con 0x7fbb7c0682b0 2020-11-16T22:18:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c00e5f0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00e990 con 0x7fbb7c0682b0 2020-11-16T22:18:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c141a70 con 0x7fbb7c0682b0 2020-11-16T22:18:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.021+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c1424f0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbb6c019030 con 0x7fbb7c0682b0 2020-11-16T22:18:54.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c141c00 con 0x7fbb7c0682b0 2020-11-16T22:18:54.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb69ffb700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 0x7fbb54057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb81d19700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 0x7fbb54057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb6c06d430 con 0x7fbb7c0682b0 2020-11-16T22:18:54.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.025+0000 7fbb81d19700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 0x7fbb54057520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fbb7c0b7c70 tx=0x7fbb78009290).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.033+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbb6c043580 con 0x7fbb7c0682b0 2020-11-16T22:18:54.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.197+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb7c05d200 con 0x7fbb7c0682b0 2020-11-16T22:18:54.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.197+0000 7fbb69ffb700 1 -- 172.21.15.185:0/2612632366 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16829 (secure 0 0 0) 0x7fbb6c046ba0 con 0x7fbb7c0682b0 2020-11-16T22:18:54.185 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:54.185 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":65,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:51.735891+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:18:54.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 msgr2=0x7fbb54057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 0x7fbb54057520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fbb7c0b7c70 tx=0x7fbb78009290).stop 2020-11-16T22:18:54.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 msgr2=0x7fbb7c0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fbb6c00c760 tx=0x7fbb6c007a20).stop 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 shutdown_connections 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbb54055060 0x7fbb54057520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbb7c0b7470 0x7fbb7c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb7c067070 0x7fbb7c13f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 --2- 172.21.15.185:0/2612632366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbb7c0682b0 0x7fbb7c0ae1d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 >> 172.21.15.185:0/2612632366 conn(0x7fbb7c0637b0 msgr2=0x7fbb7c0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 shutdown_connections 2020-11-16T22:18:54.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.201+0000 7fbb82d1b700 1 -- 172.21.15.185:0/2612632366 wait complete. 2020-11-16T22:18:54.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:18:48.576973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:18:49.752256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:18:54.202 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:18:54.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 Processor -- start 2020-11-16T22:18:54.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 -- start start 2020-11-16T22:18:54.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07700b76b0 0x7f0770064c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f07700b2980 0x7f07700b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700684f0 con 0x7f0770144490 2020-11-16T22:18:54.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700ad020 con 0x7f07700b76b0 2020-11-16T22:18:54.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700b2240 con 0x7f07700b2980 2020-11-16T22:18:54.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f077569d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f0770144490 0x7f0770144900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54746/0 (socket says 172.21.15.185:54746) 2020-11-16T22:18:54.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f077569d700 1 -- 172.21.15.185:0/586500572 learned_addr learned my addr 172.21.15.185:0/586500572 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:54.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 --2- 172.21.15.185:0/586500572 >> v2:172.21.15.185:3300/0 conn(0x7f07700b2980 0x7f07700b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0774e9c700 1 --2- 172.21.15.185:0/586500572 >> v2:172.21.15.96:3300/0 conn(0x7f07700b76b0 0x7f0770064c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076f7fe700 1 -- 172.21.15.185:0/586500572 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449677614 0 0) 0x7f07700684f0 con 0x7f0770144490 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076f7fe700 1 -- 172.21.15.185:0/586500572 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0754003240 con 0x7f0770144490 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 -- 172.21.15.185:0/586500572 >> v2:172.21.15.96:3300/0 conn(0x7f07700b76b0 msgr2=0x7f0770064c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 --2- 172.21.15.185:0/586500572 >> v2:172.21.15.96:3300/0 conn(0x7f07700b76b0 0x7f0770064c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 -- 172.21.15.185:0/586500572 >> v1:172.21.15.185:6789/0 conn(0x7f0770144490 legacy=0x7f0770144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 -- 172.21.15.185:0/586500572 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700b24e0 con 0x7f07700b2980 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076ffff700 1 --2- 172.21.15.185:0/586500572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f07700b7a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f07700b51a0 tx=0x7f075c006df0).ready entity=mon.1 client_cookie=6cf571f218437df6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076f7fe700 1 -- 172.21.15.185:0/586500572 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c009640 con 0x7f07700b2980 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076f7fe700 1 -- 172.21.15.185:0/586500572 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c0097a0 con 0x7f07700b2980 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f076f7fe700 1 -- 172.21.15.185:0/586500572 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c009b20 con 0x7f07700b2980 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 -- 172.21.15.185:0/586500572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 msgr2=0x7f07700b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.341+0000 7f0777100700 1 --2- 172.21.15.185:0/586500572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f07700b7a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f07700b51a0 tx=0x7f075c006df0).stop 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- 172.21.15.185:0/586500572 shutdown_connections 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 --2- 172.21.15.185:0/586500572 >> v2:172.21.15.96:3300/0 conn(0x7f07700b76b0 0x7f0770064c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 --2- 172.21.15.185:0/586500572 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f07700b7a90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- 172.21.15.185:0/586500572 >> 172.21.15.185:0/586500572 conn(0x7f0770062ea0 msgr2=0x7f0770063280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- 172.21.15.185:0/586500572 shutdown_connections 2020-11-16T22:18:54.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- 172.21.15.185:0/586500572 wait complete. 2020-11-16T22:18:54.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 Processor -- start 2020-11-16T22:18:54.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- start start 2020-11-16T22:18:54.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 0x7f07700990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0770144490 0x7f07700995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770099aa0 con 0x7f07700b76b0 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077009d740 con 0x7f07700b2980 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0777100700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077009c9d0 con 0x7f0770144490 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33080/0 (socket says 172.21.15.185:33080) 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 -- 172.21.15.185:0/338066918 learned_addr learned my addr 172.21.15.185:0/338066918 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f077569d700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0770144490 0x7f07700995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f076ffff700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 0x7f07700990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 -- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0770144490 msgr2=0x7f07700995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0770144490 0x7f07700995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 -- 172.21.15.185:0/338066918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 msgr2=0x7f07700990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 0x7f07700990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.345+0000 7f0774e9c700 1 -- 172.21.15.185:0/338066918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077009cd60 con 0x7f07700b2980 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f0774e9c700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f076400ebd0 tx=0x7f0764006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400b3a0 con 0x7f07700b2980 2020-11-16T22:18:54.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076400c9b0 con 0x7f07700b2980 2020-11-16T22:18:54.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400cd70 con 0x7f07700b2980 2020-11-16T22:18:54.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076ffff700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 0x7f07700990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:54.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077009d000 con 0x7f07700b2980 2020-11-16T22:18:54.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077013f1f0 con 0x7f07700b2980 2020-11-16T22:18:54.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077005d4e0 con 0x7f07700b2980 2020-11-16T22:18:54.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0764012600 con 0x7f07700b2980 2020-11-16T22:18:54.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 0x7f0748057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f076406e130 con 0x7f07700b2980 2020-11-16T22:18:54.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076ffff700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 0x7f0748057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.349+0000 7f076ffff700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 0x7f0748057520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f075c006e30 tx=0x7f075c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.357+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0764044330 con 0x7f07700b2980 2020-11-16T22:18:54.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.509+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0770068170 con 0x7f07700b2980 2020-11-16T22:18:54.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.509+0000 7f076cff9700 1 -- 172.21.15.185:0/338066918 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v65) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0764047950 con 0x7f07700b2980 2020-11-16T22:18:54.497 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:18:54.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 msgr2=0x7f0748057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 0x7f0748057520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f075c006e30 tx=0x7f075c015000).stop 2020-11-16T22:18:54.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 msgr2=0x7f077013ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f076400ebd0 tx=0x7f0764006df0).stop 2020-11-16T22:18:54.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 shutdown_connections 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0748055060 0x7f0748057520 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f07700b2980 0x7f077013ed00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0770144490 0x7f07700995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 --2- 172.21.15.185:0/338066918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07700b76b0 0x7f07700990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 >> 172.21.15.185:0/338066918 conn(0x7f0770062ea0 msgr2=0x7f0770099c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 shutdown_connections 2020-11-16T22:18:54.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.513+0000 7f0777100700 1 -- 172.21.15.185:0/338066918 wait complete. 2020-11-16T22:18:54.512 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:18:54.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 Processor -- start 2020-11-16T22:18:54.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 -- start start 2020-11-16T22:18:54.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b180b2e80 0x7f4b180b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b180b1310 con 0x7f4b180b3920 2020-11-16T22:18:54.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1813e400 con 0x7f4b180b2e80 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180ac070 con 0x7f4b18143ef0 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54768/0 (socket says 172.21.15.185:54768) 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 -- 172.21.15.185:0/3406535531 learned_addr learned my addr 172.21.15.185:0/3406535531 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1659c700 1 --2- 172.21.15.185:0/3406535531 >> v2:172.21.15.96:3300/0 conn(0x7f4b180b2e80 0x7f4b180b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 -- 172.21.15.185:0/3406535531 >> v2:172.21.15.96:3300/0 conn(0x7f4b180b2e80 msgr2=0x7f4b180b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:54.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 --2- 172.21.15.185:0/3406535531 >> v2:172.21.15.96:3300/0 conn(0x7f4b180b2e80 0x7f4b180b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 -- 172.21.15.185:0/3406535531 >> v1:172.21.15.96:6789/0 conn(0x7f4b180b3920 legacy=0x7f4b1813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:54.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 -- 172.21.15.185:0/3406535531 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b180ac310 con 0x7f4b18143ef0 2020-11-16T22:18:54.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b16d9d700 1 --2- 172.21.15.185:0/3406535531 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f4b10009d50 tx=0x7f4b10009290).ready entity=mon.2 client_cookie=2831fc6d32a071b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b15d9b700 1 -- 172.21.15.185:0/3406535531 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000bc40 con 0x7f4b18143ef0 2020-11-16T22:18:54.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b15d9b700 1 -- 172.21.15.185:0/3406535531 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1000bda0 con 0x7f4b18143ef0 2020-11-16T22:18:54.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b15d9b700 1 -- 172.21.15.185:0/3406535531 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10011610 con 0x7f4b18143ef0 2020-11-16T22:18:54.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 -- 172.21.15.185:0/3406535531 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 msgr2=0x7f4b180b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.653+0000 7f4b1759e700 1 --2- 172.21.15.185:0/3406535531 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f4b10009d50 tx=0x7f4b10009290).stop 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 -- 172.21.15.185:0/3406535531 shutdown_connections 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 --2- 172.21.15.185:0/3406535531 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b180b16b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 --2- 172.21.15.185:0/3406535531 >> v2:172.21.15.96:3300/0 conn(0x7f4b180b2e80 0x7f4b180b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 -- 172.21.15.185:0/3406535531 >> 172.21.15.185:0/3406535531 conn(0x7f4b18060130 msgr2=0x7f4b18060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 -- 172.21.15.185:0/3406535531 shutdown_connections 2020-11-16T22:18:54.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 -- 172.21.15.185:0/3406535531 wait complete. 2020-11-16T22:18:54.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 Processor -- start 2020-11-16T22:18:54.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.657+0000 7f4b1759e700 1 -- start start 2020-11-16T22:18:54.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b180b2e80 0x7f4b18146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b180b3920 0x7f4b18142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18145710 con 0x7f4b180b2e80 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181459b0 con 0x7f4b180b3920 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18153110 con 0x7f4b18143ef0 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b0ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b180b3920 0x7f4b18142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54774/0 (socket says 172.21.15.185:54774) 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 -- 172.21.15.185:0/340215227 learned_addr learned my addr 172.21.15.185:0/340215227 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1659c700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b180b2e80 0x7f4b18146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 -- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b180b3920 msgr2=0x7f4b18142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b180b3920 0x7f4b18142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 -- 172.21.15.185:0/340215227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b180b2e80 msgr2=0x7f4b18146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b180b2e80 0x7f4b18146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 -- 172.21.15.185:0/340215227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b181533b0 con 0x7f4b18143ef0 2020-11-16T22:18:54.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b16d9d700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f4b10007fd0 tx=0x7f4b10016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000bc40 con 0x7f4b18143ef0 2020-11-16T22:18:54.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b10009e90 con 0x7f4b18143ef0 2020-11-16T22:18:54.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1814b4d0 con 0x7f4b18143ef0 2020-11-16T22:18:54.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10010cd0 con 0x7f4b18143ef0 2020-11-16T22:18:54.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18153d40 con 0x7f4b18143ef0 2020-11-16T22:18:54.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.661+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18153540 con 0x7f4b18143ef0 2020-11-16T22:18:54.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.665+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4b10017070 con 0x7f4b18143ef0 2020-11-16T22:18:54.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.665+0000 7f4b0e7fc700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 0x7f4af4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.665+0000 7f4b1659c700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 0x7f4af4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.665+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b1006fa90 con 0x7f4b18143ef0 2020-11-16T22:18:54.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.665+0000 7f4b1659c700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 0x7f4af4057520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f4b0800d5f0 tx=0x7f4b08008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.669+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4b10045cf0 con 0x7f4b18143ef0 2020-11-16T22:18:54.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.821+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4b1805d200 con 0x7f4b18143ef0 2020-11-16T22:18:54.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.821+0000 7f4b0e7fc700 1 -- 172.21.15.185:0/340215227 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4b10015030 con 0x7f4b18143ef0 2020-11-16T22:18:54.809 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 msgr2=0x7f4af4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 0x7f4af4057520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f4b0800d5f0 tx=0x7f4b08008040).stop 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 msgr2=0x7f4b18153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f4b10007fd0 tx=0x7f4b10016000).stop 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 shutdown_connections 2020-11-16T22:18:54.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4af4055060 0x7f4af4057520 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4b18143ef0 0x7f4b18153850 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b180b2e80 0x7f4b18146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 --2- 172.21.15.185:0/340215227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4b180b3920 0x7f4b18142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 >> 172.21.15.185:0/340215227 conn(0x7f4b18060130 msgr2=0x7f4b18066720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 shutdown_connections 2020-11-16T22:18:54.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.824+0000 7f4b1759e700 1 -- 172.21.15.185:0/340215227 wait complete. 2020-11-16T22:18:54.823 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:18:54.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.964+0000 7f20a79d9700 1 Processor -- start 2020-11-16T22:18:54.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.964+0000 7f20a79d9700 1 -- start start 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f20a00afc40 0x7f20a014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b0090 con 0x7f20a00b74b0 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b02e0 con 0x7f20a0144c00 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a0143fa0 con 0x7f20a00afc40 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a4f74700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f20a0144c00 0x7f20a00ade00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54774/0 (socket says 172.21.15.185:54774) 2020-11-16T22:18:54.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a4f74700 1 -- 172.21.15.185:0/4147488228 learned_addr learned my addr 172.21.15.185:0/4147488228 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:54.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a5f76700 1 --2- 172.21.15.185:0/4147488228 >> v2:172.21.15.96:3300/0 conn(0x7f20a00afc40 0x7f20a014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195401358 0 0) 0x7f20a00b02e0 con 0x7f20a0144c00 2020-11-16T22:18:54.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003240 con 0x7f20a0144c00 2020-11-16T22:18:54.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a5f76700 1 -- 172.21.15.185:0/4147488228 >> v1:172.21.15.185:6789/0 conn(0x7f20a0144c00 legacy=0x7f20a00ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a5f76700 1 -- 172.21.15.185:0/4147488228 >> v1:172.21.15.96:6789/0 conn(0x7f20a00b74b0 legacy=0x7f20a00ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268886794 0 0) 0x7f20a00b0090 con 0x7f20a00b74b0 2020-11-16T22:18:54.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a5f76700 1 -- 172.21.15.185:0/4147488228 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00b0580 con 0x7f20a00afc40 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a5f76700 1 --2- 172.21.15.185:0/4147488228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a014ec20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2098009c90 tx=0x7f20980091d0).ready entity=mon.0 client_cookie=98f9963af0080683 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800ba70 con 0x7f20a00afc40 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800bbd0 con 0x7f20a00afc40 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f2097fff700 1 -- 172.21.15.185:0/4147488228 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098011400 con 0x7f20a00afc40 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- 172.21.15.185:0/4147488228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 msgr2=0x7f20a014ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 --2- 172.21.15.185:0/4147488228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a014ec20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f2098009c90 tx=0x7f20980091d0).stop 2020-11-16T22:18:54.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- 172.21.15.185:0/4147488228 shutdown_connections 2020-11-16T22:18:54.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 --2- 172.21.15.185:0/4147488228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a014ec20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- 172.21.15.185:0/4147488228 >> 172.21.15.185:0/4147488228 conn(0x7f20a0067860 msgr2=0x7f20a0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:54.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- 172.21.15.185:0/4147488228 shutdown_connections 2020-11-16T22:18:54.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.968+0000 7f20a79d9700 1 -- 172.21.15.185:0/4147488228 wait complete. 2020-11-16T22:18:54.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 Processor -- start 2020-11-16T22:18:54.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 -- start start 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a00b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 0x7f20a0140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b47c0 con 0x7f20a00afc40 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b1780 con 0x7f20a00b74b0 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a79d9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b3e20 con 0x7f20a0144c00 2020-11-16T22:18:54.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a4f74700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 0x7f20a0140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54786/0 (socket says 172.21.15.185:54786) 2020-11-16T22:18:54.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 -- 172.21.15.185:0/3283094861 learned_addr learned my addr 172.21.15.185:0/3283094861 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:54.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5775700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a00b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 -- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 msgr2=0x7f20a0140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 0x7f20a0140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 -- 172.21.15.185:0/3283094861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 msgr2=0x7f20a00b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a00b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 -- 172.21.15.185:0/3283094861 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00b19b0 con 0x7f20a0144c00 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a4f74700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 0x7f20a0140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.972+0000 7f20a5f76700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2098011000 tx=0x7f2098016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209801c5f0 con 0x7f20a0144c00 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209801c750 con 0x7f20a0144c00 2020-11-16T22:18:54.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00b4080 con 0x7f20a0144c00 2020-11-16T22:18:54.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209801cad0 con 0x7f20a0144c00 2020-11-16T22:18:54.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a00b0fd0 con 0x7f20a0144c00 2020-11-16T22:18:54.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a0147cf0 con 0x7f20a0144c00 2020-11-16T22:18:54.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2098024030 con 0x7f20a0144c00 2020-11-16T22:18:54.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 0x7f2078057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:54.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20a5775700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 0x7f2078057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:54.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f209806f910 con 0x7f20a0144c00 2020-11-16T22:18:54.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.976+0000 7f20a5775700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 0x7f2078057520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f20900011a0 tx=0x7f2090007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:54.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:54.984+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2098045ca0 con 0x7f20a0144c00 2020-11-16T22:18:55.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.132+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20a005d4e0 con 0x7f20a0144c00 2020-11-16T22:18:55.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20957fa700 1 -- 172.21.15.185:0/3283094861 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7f20980220a0 con 0x7f20a0144c00 2020-11-16T22:18:55.122 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 msgr2=0x7f2078057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 0x7f2078057520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f20900011a0 tx=0x7f2090007a00).stop 2020-11-16T22:18:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 msgr2=0x7f20a0140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2098011000 tx=0x7f2098016000).stop 2020-11-16T22:18:55.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 shutdown_connections 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2078055060 0x7f2078057520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f20a0144c00 0x7f20a0140a80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20a00afc40 0x7f20a00b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 --2- 172.21.15.185:0/3283094861 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f20a00b74b0 0x7f20a0140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 >> 172.21.15.185:0/3283094861 conn(0x7f20a0067860 msgr2=0x7f20a00b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 shutdown_connections 2020-11-16T22:18:55.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.136+0000 7f20a79d9700 1 -- 172.21.15.185:0/3283094861 wait complete. 2020-11-16T22:18:55.136 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:18:55.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 Processor -- start 2020-11-16T22:18:55.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 -- start start 2020-11-16T22:18:55.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fab34141150 0x7fab341415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab340acdf0 con 0x7fab340b3020 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab341462b0 con 0x7fab34141150 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab38f4d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340ad2c0 con 0x7fab340abef0 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54792/0 (socket says 172.21.15.185:54792) 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 -- 172.21.15.185:0/2167475379 learned_addr learned my addr 172.21.15.185:0/2167475379 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab33fff700 1 --2- 172.21.15.185:0/2167475379 >> v2:172.21.15.185:3300/0 conn(0x7fab34141150 0x7fab341415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 -- 172.21.15.185:0/2167475379 >> v2:172.21.15.185:3300/0 conn(0x7fab34141150 msgr2=0x7fab341415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 --2- 172.21.15.185:0/2167475379 >> v2:172.21.15.185:3300/0 conn(0x7fab34141150 0x7fab341415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 -- 172.21.15.185:0/2167475379 >> v1:172.21.15.96:6789/0 conn(0x7fab340b3020 legacy=0x7fab34145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 -- 172.21.15.185:0/2167475379 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340ad020 con 0x7fab340abef0 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab32ffd700 1 --2- 172.21.15.185:0/2167475379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fab1c006fd0 tx=0x7fab1c006eb0).ready entity=mon.2 client_cookie=c4b51b648961b2d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.276+0000 7fab327fc700 1 -- 172.21.15.185:0/2167475379 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c009890 con 0x7fab340abef0 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab327fc700 1 -- 172.21.15.185:0/2167475379 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0099f0 con 0x7fab340abef0 2020-11-16T22:18:55.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab327fc700 1 -- 172.21.15.185:0/2167475379 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c009d70 con 0x7fab340abef0 2020-11-16T22:18:55.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- 172.21.15.185:0/2167475379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 msgr2=0x7fab340b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- 172.21.15.185:0/2167475379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fab1c006fd0 tx=0x7fab1c006eb0).stop 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- 172.21.15.185:0/2167475379 shutdown_connections 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- 172.21.15.185:0/2167475379 >> v2:172.21.15.185:3300/0 conn(0x7fab34141150 0x7fab341415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- 172.21.15.185:0/2167475379 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340abef0 0x7fab340b17c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- 172.21.15.185:0/2167475379 >> 172.21.15.185:0/2167475379 conn(0x7fab34060130 msgr2=0x7fab34060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:55.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- 172.21.15.185:0/2167475379 shutdown_connections 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- 172.21.15.185:0/2167475379 wait complete. 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 Processor -- start 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- start start 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340b3020 0x7fab340ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab34141150 0x7fab340aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34068470 con 0x7fab34141150 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340ad630 con 0x7fab340abef0 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab38f4d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340adf40 con 0x7fab340b3020 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33122/0 (socket says 172.21.15.185:33122) 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 -- 172.21.15.185:0/1642861342 learned_addr learned my addr 172.21.15.185:0/1642861342 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab32ffd700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340b3020 0x7fab340ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab33fff700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab34141150 0x7fab340aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 -- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340b3020 msgr2=0x7fab340ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340b3020 0x7fab340ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 -- 172.21.15.185:0/1642861342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab34141150 msgr2=0x7fab340aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab34141150 0x7fab340aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.280+0000 7fab337fe700 1 -- 172.21.15.185:0/1642861342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340ae1e0 con 0x7fab340abef0 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab337fe700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fab2400c730 tx=0x7fab24007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e560 con 0x7fab340abef0 2020-11-16T22:18:55.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2400e6e0 con 0x7fab340abef0 2020-11-16T22:18:55.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340679f0 con 0x7fab340abef0 2020-11-16T22:18:55.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400eaa0 con 0x7fab340abef0 2020-11-16T22:18:55.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3413a850 con 0x7fab340abef0 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fab24010030 con 0x7fab340abef0 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3405d200 con 0x7fab340abef0 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 0x7fab10057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab2406dc60 con 0x7fab340abef0 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab32ffd700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 0x7fab10057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.284+0000 7fab32ffd700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 0x7fab10057550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fab1c006ef0 tx=0x7fab1c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.292+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fab24044290 con 0x7fab340abef0 2020-11-16T22:18:55.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.440+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab340008d0 con 0x7fab340abef0 2020-11-16T22:18:55.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab2b7fe700 1 -- 172.21.15.185:0/1642861342 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7fab24007d50 con 0x7fab340abef0 2020-11-16T22:18:55.430 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:18:55.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 msgr2=0x7fab10057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 0x7fab10057550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fab1c006ef0 tx=0x7fab1c015000).stop 2020-11-16T22:18:55.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 msgr2=0x7fab340b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fab2400c730 tx=0x7fab24007c30).stop 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 shutdown_connections 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fab10055090 0x7fab10057550 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fab340abef0 0x7fab340b56c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab34141150 0x7fab340aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 --2- 172.21.15.185:0/1642861342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fab340b3020 0x7fab340ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 >> 172.21.15.185:0/1642861342 conn(0x7fab34060130 msgr2=0x7fab34135750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:55.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 shutdown_connections 2020-11-16T22:18:55.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.444+0000 7fab38f4d700 1 -- 172.21.15.185:0/1642861342 wait complete. 2020-11-16T22:18:55.444 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:18:55.444 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:55.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 Processor -- start 2020-11-16T22:18:55.567 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 -- start start 2020-11-16T22:18:55.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f05000ad110 0x7f0500147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.568 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0500061fe0 con 0x7f05000b3090 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05000b0fe0 con 0x7f05000b1d30 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05000b2b60 con 0x7f05000ad110 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fe7fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f05000b1d30 0x7f05001477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54800/0 (socket says 172.21.15.185:54800) 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fe7fc700 1 -- 172.21.15.185:0/3544818042 learned_addr learned my addr 172.21.15.185:0/3544818042 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04ff7fe700 1 --2- 172.21.15.185:0/3544818042 >> v2:172.21.15.96:3300/0 conn(0x7f05000ad110 0x7f0500147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749192508 0 0) 0x7f05000b0fe0 con 0x7f05000b1d30 2020-11-16T22:18:55.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e0003240 con 0x7f05000b1d30 2020-11-16T22:18:55.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2591067323 0 0) 0x7f04e0003240 con 0x7f05000b1d30 2020-11-16T22:18:55.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 >> v2:172.21.15.96:3300/0 conn(0x7f05000ad110 msgr2=0x7f0500147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 --2- 172.21.15.185:0/3544818042 >> v2:172.21.15.96:3300/0 conn(0x7f05000ad110 0x7f0500147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 >> v1:172.21.15.96:6789/0 conn(0x7f05000b3090 legacy=0x7f050014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000b1280 con 0x7f05000b1d30 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f04e80014e0 con 0x7f05000b1d30 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 >> v1:172.21.15.185:6789/0 conn(0x7f05000b1d30 legacy=0x7f05001477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 0x7f04e0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.572 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e00044e0 0x7f04e0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f04e0004e50 0x7f04e000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000b2b60 con 0x7f04e0003730 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0004260 con 0x7f04e00044e0 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.580+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e000bc20 con 0x7f04e0004e50 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fe7fc700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e00044e0 0x7f04e0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.573 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 0x7f04e0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04ff7fe700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f04e0004e50 0x7f04e000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 -- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f04e0004e50 msgr2=0x7f04e000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f04e0004e50 0x7f04e000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 -- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e00044e0 msgr2=0x7f04e0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e00044e0 0x7f04e0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 -- 172.21.15.185:0/3544818042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0003240 con 0x7f04e0003730 2020-11-16T22:18:55.574 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04ff7fe700 1 -- 172.21.15.185:0/3544818042 reap_dead start 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04feffd700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 0x7f04e0003b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f04f0001910 tx=0x7f04f0007b90).ready entity=mon.0 client_cookie=f128a0e28016327a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0007020 con 0x7f04e0003730 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0007180 con 0x7f04e0003730 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fdffb700 1 -- 172.21.15.185:0/3544818042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000c830 con 0x7f04e0003730 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 msgr2=0x7f04e0003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 0x7f04e0003b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f04f0001910 tx=0x7f04f0007b90).stop 2020-11-16T22:18:55.575 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- 172.21.15.185:0/3544818042 shutdown_connections 2020-11-16T22:18:55.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 --2- 172.21.15.185:0/3544818042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04e0003730 0x7f04e0003b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- 172.21.15.185:0/3544818042 >> 172.21.15.185:0/3544818042 conn(0x7f0500060130 msgr2=0x7f0500060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:55.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- 172.21.15.185:0/3544818042 shutdown_connections 2020-11-16T22:18:55.576 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- 172.21.15.185:0/3544818042 wait complete. 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 Processor -- start 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- start start 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e0003730 0x7f05000b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f050010f8c0 0x7f0500141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000ae6c0 con 0x7f050010f8c0 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500157d50 con 0x7f04e0003730 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.584+0000 7f04fffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05001448c0 con 0x7f05001133e0 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54820/0 (socket says 172.21.15.185:54820) 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 -- 172.21.15.185:0/1414100098 learned_addr learned my addr 172.21.15.185:0/1414100098 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:55.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04feffd700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e0003730 0x7f05000b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04ff7fe700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f050010f8c0 0x7f0500141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 -- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e0003730 msgr2=0x7f05000b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e0003730 0x7f05000b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 -- 172.21.15.185:0/1414100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f050010f8c0 msgr2=0x7f0500141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f050010f8c0 0x7f0500141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 -- 172.21.15.185:0/1414100098 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500144b60 con 0x7f05001133e0 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fe7fc700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f04e8004b30 tx=0x7f05000b1410).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800f840 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e800f9c0 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0500144e00 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800fd80 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0500145770 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f04e801b030 con 0x7f05001133e0 2020-11-16T22:18:55.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050005d200 con 0x7f05001133e0 2020-11-16T22:18:55.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 0x7f04dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:55.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f04e806d1d0 con 0x7f05001133e0 2020-11-16T22:18:55.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04feffd700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 0x7f04dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:55.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.588+0000 7f04feffd700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 0x7f04dc057520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f04f0000fd0 tx=0x7f04f0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:55.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.600+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f04e8043430 con 0x7f05001133e0 2020-11-16T22:18:55.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0500064ca0 con 0x7f05001133e0 2020-11-16T22:18:55.783 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04f6ffd700 1 -- 172.21.15.185:0/1414100098 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f04e8019030 con 0x7f05001133e0 2020-11-16T22:18:55.783 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:55.783 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":67,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64},{"gid":6188,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:51.735017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6642},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 msgr2=0x7f04dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 0x7f04dc057520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f04f0000fd0 tx=0x7f04f0015000).stop 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 msgr2=0x7f05001417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f04e8004b30 tx=0x7f05000b1410).stop 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 shutdown_connections 2020-11-16T22:18:55.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f04dc055060 0x7f04dc057520 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f04e0003730 0x7f05000b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f050010f8c0 0x7f0500141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 --2- 172.21.15.185:0/1414100098 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05001133e0 0x7f05001417d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 >> 172.21.15.185:0/1414100098 conn(0x7f0500060130 msgr2=0x7f0500067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 shutdown_connections 2020-11-16T22:18:55.786 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:55.796+0000 7f04fffff700 1 -- 172.21.15.185:0/1414100098 wait complete. 2020-11-16T22:18:55.787 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 67 2020-11-16T22:18:56.098 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:18:50.731986+0000', 'modified': '2020-11-16T22:18:51.735017+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6642}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6642': {'gid': 6642, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6833/974001631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 974001631}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 974001631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:18:56.098 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:18:56.099 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:18:56.235 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6642,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.514385646,"mdsmap_epoch":67,"osdmap_epoch":65,"osdmap_epoch_barrier":64,"uptime":10.76934365} 2020-11-16T22:18:56.236 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6642, "mdsmap_epoch": 67, "osdmap_epoch": 65, "osdmap_epoch_barrier": 64, "rank_uptime": 5.514385646, "state": "up:active", "uptime": 10.76934365, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:18:56.236 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:18:56.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:56.237 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:18:56.254 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:18:56.254 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:18:56.349 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:18:56.374+0000 7f5b056f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:18:56.372 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:18:56.373 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:18:56.401 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:18:56.401 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:18:56.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:18:56.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:18:56.409 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:18:56.439 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:18:56.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 Processor -- start 2020-11-16T22:18:56.578 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- start start 2020-11-16T22:18:56.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb5040b6bf0 0x7fb5040b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.579 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50414bc30 con 0x7fb504063ca0 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50414cb10 con 0x7fb5040b6bf0 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414be90 con 0x7fb504149180 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54830/0 (socket says 172.21.15.185:54830) 2020-11-16T22:18:56.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 -- 172.21.15.185:0/1010055489 learned_addr learned my addr 172.21.15.185:0/1010055489 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:56.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb502ffd700 1 --2- 172.21.15.185:0/1010055489 >> v2:172.21.15.96:3300/0 conn(0x7fb5040b6bf0 0x7fb5040b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 -- 172.21.15.185:0/1010055489 >> v2:172.21.15.96:3300/0 conn(0x7fb5040b6bf0 msgr2=0x7fb5040b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 --2- 172.21.15.185:0/1010055489 >> v2:172.21.15.96:3300/0 conn(0x7fb5040b6bf0 0x7fb5040b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 -- 172.21.15.185:0/1010055489 >> v1:172.21.15.96:6789/0 conn(0x7fb504063ca0 legacy=0x7fb504147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 -- 172.21.15.185:0/1010055489 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50414c130 con 0x7fb504149180 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb5037fe700 1 --2- 172.21.15.185:0/1010055489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb4fc006fd0 tx=0x7fb4fc006eb0).ready entity=mon.2 client_cookie=b1cf1c9d6185a19b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:56.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb501ffb700 1 -- 172.21.15.185:0/1010055489 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0098d0 con 0x7fb504149180 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb501ffb700 1 -- 172.21.15.185:0/1010055489 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc009a30 con 0x7fb504149180 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb501ffb700 1 -- 172.21.15.185:0/1010055489 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc009db0 con 0x7fb504149180 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- 172.21.15.185:0/1010055489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 msgr2=0x7fb5040ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 --2- 172.21.15.185:0/1010055489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb4fc006fd0 tx=0x7fb4fc006eb0).stop 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- 172.21.15.185:0/1010055489 shutdown_connections 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 --2- 172.21.15.185:0/1010055489 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb5040ac4c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:56.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 --2- 172.21.15.185:0/1010055489 >> v2:172.21.15.96:3300/0 conn(0x7fb5040b6bf0 0x7fb5040b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:56.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- 172.21.15.185:0/1010055489 >> 172.21.15.185:0/1010055489 conn(0x7fb5040657b0 msgr2=0x7fb504065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:56.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- 172.21.15.185:0/1010055489 shutdown_connections 2020-11-16T22:18:56.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.592+0000 7fb50988c700 1 -- 172.21.15.185:0/1010055489 wait complete. 2020-11-16T22:18:56.585 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 Processor -- start 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- start start 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 0x7fb5040ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 0x7fb50414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414fa70 con 0x7fb5040b6bf0 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414baa0 con 0x7fb504063ca0 2020-11-16T22:18:56.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50414aeb0 con 0x7fb504149180 2020-11-16T22:18:56.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 0x7fb5040ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54834/0 (socket says 172.21.15.185:54834) 2020-11-16T22:18:56.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5037fe700 1 -- 172.21.15.185:0/3580016103 learned_addr learned my addr 172.21.15.185:0/3580016103 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:56.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5027fc700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 0x7fb50414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 -- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 msgr2=0x7fb50414f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 -- 172.21.15.185:0/3580016103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 msgr2=0x7fb50414f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 0x7fb50414f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 -- 172.21.15.185:0/3580016103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504144670 con 0x7fb504063ca0 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5027fc700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 0x7fb50414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:18:56.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb5037fe700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb502ffd700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 0x7fb5040ac5d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fb4f400ec50 tx=0x7fb4f4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009790 con 0x7fb504063ca0 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f40098f0 con 0x7fb504063ca0 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5041448c0 con 0x7fb504063ca0 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009c70 con 0x7fb504063ca0 2020-11-16T22:18:56.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.596+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5040acac0 con 0x7fb504063ca0 2020-11-16T22:18:56.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50405d4e0 con 0x7fb504063ca0 2020-11-16T22:18:56.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb4f401d460 con 0x7fb504063ca0 2020-11-16T22:18:56.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb4eaffd700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:56.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb4f406dcf0 con 0x7fb504063ca0 2020-11-16T22:18:56.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb5027fc700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:56.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.600+0000 7fb5027fc700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 0x7fb4e0057520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb5040b6270 tx=0x7fb4f8009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:56.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.604+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb4f4043ef0 con 0x7fb504063ca0 2020-11-16T22:18:56.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:56.808+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fb504147e10 con 0x7fb504063ca0 2020-11-16T22:18:56.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.004+0000 7fb4eaffd700 1 -- 172.21.15.185:0/3580016103 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v68) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb4f4047510 con 0x7fb504063ca0 2020-11-16T22:18:57.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 msgr2=0x7fb4e0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 0x7fb4e0057520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb5040b6270 tx=0x7fb4f8009040).stop 2020-11-16T22:18:57.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 msgr2=0x7fb5040ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 0x7fb5040ac5d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fb4f400ec50 tx=0x7fb4f4008040).stop 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 shutdown_connections 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb4e0055060 0x7fb4e0057520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb504063ca0 0x7fb5040ac5d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb504149180 0x7fb50414f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 --2- 172.21.15.185:0/3580016103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5040b6bf0 0x7fb50414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 >> 172.21.15.185:0/3580016103 conn(0x7fb5040657b0 msgr2=0x7fb504066490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:57.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 shutdown_connections 2020-11-16T22:18:57.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.024+0000 7fb50988c700 1 -- 172.21.15.185:0/3580016103 wait complete. 2020-11-16T22:18:57.022 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:57.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 Processor -- start 2020-11-16T22:18:57.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 -- start start 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2cd40acc10 0x7f2cd41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40abf90 con 0x7f2cd40669a0 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40ac230 con 0x7f2cd40acc10 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40ac4d0 con 0x7f2cd4061e60 2020-11-16T22:18:57.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54854/0 (socket says 172.21.15.185:54854) 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 -- 172.21.15.185:0/3855321928 learned_addr learned my addr 172.21.15.185:0/3855321928 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd1d9b700 1 --2- 172.21.15.185:0/3855321928 >> v2:172.21.15.96:3300/0 conn(0x7f2cd40acc10 0x7f2cd41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 -- 172.21.15.185:0/3855321928 >> v2:172.21.15.96:3300/0 conn(0x7f2cd40acc10 msgr2=0x7f2cd41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 --2- 172.21.15.185:0/3855321928 >> v2:172.21.15.96:3300/0 conn(0x7f2cd40acc10 0x7f2cd41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.152+0000 7f2cd2d9d700 1 -- 172.21.15.185:0/3855321928 >> v1:172.21.15.96:6789/0 conn(0x7f2cd40669a0 legacy=0x7f2cd4066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:57.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd2d9d700 1 -- 172.21.15.185:0/3855321928 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40ac770 con 0x7f2cd4061e60 2020-11-16T22:18:57.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd2d9d700 1 --2- 172.21.15.185:0/3855321928 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f2ccc009d50 tx=0x7f2ccc009290).ready entity=mon.2 client_cookie=43f8527e37ff9ebd server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:57.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd159a700 1 -- 172.21.15.185:0/3855321928 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00bc80 con 0x7f2cd4061e60 2020-11-16T22:18:57.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd159a700 1 -- 172.21.15.185:0/3855321928 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc00bde0 con 0x7f2cd4061e60 2020-11-16T22:18:57.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd159a700 1 -- 172.21.15.185:0/3855321928 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc011610 con 0x7f2cd4061e60 2020-11-16T22:18:57.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- 172.21.15.185:0/3855321928 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 msgr2=0x7f2cd41437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- 172.21.15.185:0/3855321928 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f2ccc009d50 tx=0x7f2ccc009290).stop 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- 172.21.15.185:0/3855321928 shutdown_connections 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- 172.21.15.185:0/3855321928 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd4061e60 0x7f2cd41437d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- 172.21.15.185:0/3855321928 >> v2:172.21.15.96:3300/0 conn(0x7f2cd40acc10 0x7f2cd41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- 172.21.15.185:0/3855321928 >> 172.21.15.185:0/3855321928 conn(0x7f2cd4060130 msgr2=0x7f2cd4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- 172.21.15.185:0/3855321928 shutdown_connections 2020-11-16T22:18:57.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- 172.21.15.185:0/3855321928 wait complete. 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 Processor -- start 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- start start 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cd4061e60 0x7f2cd40b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4132190 0x7f2cd4153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4132610 con 0x7f2cd4132190 2020-11-16T22:18:57.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40af370 con 0x7f2cd4061e60 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4140a90 con 0x7f2cd40669a0 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54858/0 (socket says 172.21.15.185:54858) 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 -- 172.21.15.185:0/1083948452 learned_addr learned my addr 172.21.15.185:0/1083948452 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd259c700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cd4061e60 0x7f2cd40b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd2d9d700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4132190 0x7f2cd4153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 -- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cd4061e60 msgr2=0x7f2cd40b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cd4061e60 0x7f2cd40b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 -- 172.21.15.185:0/1083948452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4132190 msgr2=0x7f2cd4153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4132190 0x7f2cd4153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 -- 172.21.15.185:0/1083948452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4157a80 con 0x7f2cd40669a0 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.156+0000 7f2cd1d9b700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f2cc800db90 tx=0x7f2cc8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800b130 con 0x7f2cd40669a0 2020-11-16T22:18:57.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8009790 con 0x7f2cd40669a0 2020-11-16T22:18:57.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd40b55b0 con 0x7f2cd40669a0 2020-11-16T22:18:57.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd4140610 con 0x7f2cd40669a0 2020-11-16T22:18:57.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8009b10 con 0x7f2cd40669a0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd40ad310 con 0x7f2cd40669a0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2cc801c460 con 0x7f2cd40669a0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 0x7f2ca8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cd259c700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 0x7f2ca8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2cc806dd00 con 0x7f2cd40669a0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.160+0000 7f2cd259c700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 0x7f2ca8057520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f2cc4003300 tx=0x7f2cc4007b90).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:57.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.164+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2cd40ad310 con 0x7f2cd40669a0 2020-11-16T22:18:57.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.368+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cd40ad310 con 0x7f2cd40669a0 2020-11-16T22:18:57.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.368+0000 7f2cc27fc700 1 -- 172.21.15.185:0/1083948452 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2505 (secure 0 0 0) 0x7f2cd40ad310 con 0x7f2cd40669a0 2020-11-16T22:18:57.357 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:57.357 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:57.007557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6642,"mds_1":6188},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:creating","state_seq":1,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:18:57.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 msgr2=0x7f2ca8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 0x7f2ca8057520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f2cc4003300 tx=0x7f2cc4007b90).stop 2020-11-16T22:18:57.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 msgr2=0x7f2cd4131ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:57.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f2cc800db90 tx=0x7f2cc8008040).stop 2020-11-16T22:18:57.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 shutdown_connections 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ca8055060 0x7f2ca8057520 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2cd4061e60 0x7f2cd40b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd4132190 0x7f2cd4153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 --2- 172.21.15.185:0/1083948452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2cd40669a0 0x7f2cd4131ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 >> 172.21.15.185:0/1083948452 conn(0x7f2cd4060130 msgr2=0x7f2cd4142710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 shutdown_connections 2020-11-16T22:18:57.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:57.372+0000 7f2cd359e700 1 -- 172.21.15.185:0/1083948452 wait complete. 2020-11-16T22:18:57.361 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 68 2020-11-16T22:18:57.673 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:18:50.731986+0000', 'modified': '2020-11-16T22:18:57.007557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6642, 'mds_1': 6188}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6188': {'gid': 6188, 'name': 'a', 'rank': 1, 'incarnation': 68, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.185:6835/3676791890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 3676791890}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 3676791890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6642': {'gid': 6642, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6833/974001631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 974001631}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 974001631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:18:57.674 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6188:up:creating 2020-11-16T22:18:58.675 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 Processor -- start 2020-11-16T22:18:58.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 -- start start 2020-11-16T22:18:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23880b7610 0x7f2388067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880b3020 con 0x7f23881441f0 2020-11-16T22:18:58.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880b3280 con 0x7f2388142f30 2020-11-16T22:18:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238e18e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880b3520 con 0x7f23880b7610 2020-11-16T22:18:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238d98d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2388142f30 0x7f2388143380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54864/0 (socket says 172.21.15.185:54864) 2020-11-16T22:18:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238d98d700 1 -- 172.21.15.185:0/518313644 learned_addr learned my addr 172.21.15.185:0/518313644 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:18:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238d18c700 1 --2- 172.21.15.185:0/518313644 >> v2:172.21.15.96:3300/0 conn(0x7f23880b7610 0x7f2388067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:58.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209154764 0 0) 0x7f23880b3280 con 0x7f2388142f30 2020-11-16T22:18:58.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2370003240 con 0x7f2388142f30 2020-11-16T22:18:58.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455905144 0 0) 0x7f23880b3020 con 0x7f23881441f0 2020-11-16T22:18:58.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23880b3280 con 0x7f23881441f0 2020-11-16T22:18:58.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.828+0000 7f238d18c700 1 -- 172.21.15.185:0/518313644 >> v1:172.21.15.185:6789/0 conn(0x7f2388142f30 legacy=0x7f2388143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:58.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 -- 172.21.15.185:0/518313644 >> v1:172.21.15.96:6789/0 conn(0x7f23881441f0 legacy=0x7f23881446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:58.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 -- 172.21.15.185:0/518313644 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388067f40 con 0x7f23880b7610 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 --2- 172.21.15.185:0/518313644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23880b7610 0x7f2388067910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f238400b9a0 tx=0x7f2384006df0).ready entity=mon.0 client_cookie=c21ef734757668a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384009680 con 0x7f23880b7610 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840097e0 con 0x7f23880b7610 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f2377fff700 1 -- 172.21.15.185:0/518313644 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384009b60 con 0x7f23880b7610 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- 172.21.15.185:0/518313644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23880b7610 msgr2=0x7f2388067910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:58.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 --2- 172.21.15.185:0/518313644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23880b7610 0x7f2388067910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f238400b9a0 tx=0x7f2384006df0).stop 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- 172.21.15.185:0/518313644 shutdown_connections 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 --2- 172.21.15.185:0/518313644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23880b7610 0x7f2388067910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- 172.21.15.185:0/518313644 >> 172.21.15.185:0/518313644 conn(0x7f23880637b0 msgr2=0x7f23880b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- 172.21.15.185:0/518313644 shutdown_connections 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- 172.21.15.185:0/518313644 wait complete. 2020-11-16T22:18:58.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 Processor -- start 2020-11-16T22:18:58.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- start start 2020-11-16T22:18:58.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:58.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2388142f30 0x7f2388146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:58.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23881441f0 0x7f2388146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23881423e0 con 0x7f2388142f30 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388142610 con 0x7f23880b7610 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238e18e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23881474b0 con 0x7f23881441f0 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33196/0 (socket says 172.21.15.185:33196) 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d18c700 1 -- 172.21.15.185:0/846596553 learned_addr learned my addr 172.21.15.185:0/846596553 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:58.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.832+0000 7f238d98d700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23881441f0 0x7f2388146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238c98b700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2388142f30 0x7f2388146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 -- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23881441f0 msgr2=0x7f2388146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23881441f0 0x7f2388146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 -- 172.21.15.185:0/846596553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2388142f30 msgr2=0x7f2388146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2388142f30 0x7f2388146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:58.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 -- 172.21.15.185:0/846596553 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388147750 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238d18c700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f2384006fd0 tx=0x7f2384015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384011d70 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238401d450 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23880ac580 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238401d810 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23880b71e0 con 0x7f23880b7610 2020-11-16T22:18:58.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238813f470 con 0x7f23880b7610 2020-11-16T22:18:58.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2384024030 con 0x7f23880b7610 2020-11-16T22:18:58.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.836+0000 7f23757fa700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 0x7f2360057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:58.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.840+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f238406f2a0 con 0x7f23880b7610 2020-11-16T22:18:59.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.840+0000 7f238c98b700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 0x7f2360057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.840+0000 7f238c98b700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 0x7f2360057520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f2378000bc0 tx=0x7f2378009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:59.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:58.844+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f238813f470 con 0x7f23880b7610 2020-11-16T22:18:59.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f238813f470 con 0x7f23880b7610 2020-11-16T22:18:59.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f23757fa700 1 -- 172.21.15.185:0/846596553 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2518 (secure 0 0 0) 0x7f238813f470 con 0x7f23880b7610 2020-11-16T22:18:59.031 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:18:59.031 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:59.037585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:18:59.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 msgr2=0x7f2360057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 0x7f2360057520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f2378000bc0 tx=0x7f2378009040).stop 2020-11-16T22:18:59.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 msgr2=0x7f23880ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f2384006fd0 tx=0x7f2384015000).stop 2020-11-16T22:18:59.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 shutdown_connections 2020-11-16T22:18:59.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2360055060 0x7f2360057520 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23880b7610 0x7f23880ac090 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23881441f0 0x7f2388146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 --2- 172.21.15.185:0/846596553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2388142f30 0x7f2388146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 >> 172.21.15.185:0/846596553 conn(0x7f23880637b0 msgr2=0x7f238813ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:59.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 shutdown_connections 2020-11-16T22:18:59.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.048+0000 7f238e18e700 1 -- 172.21.15.185:0/846596553 wait complete. 2020-11-16T22:18:59.039 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 71 2020-11-16T22:18:59.348 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:18:50.731986+0000', 'modified': '2020-11-16T22:18:59.037585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6642, 'mds_1': 6188, 'mds_2': 6185}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6185': {'gid': 6185, 'name': 'b', 'rank': 2, 'incarnation': 70, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6835/3171513205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3171513205}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3171513205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6188': {'gid': 6188, 'name': 'a', 'rank': 1, 'incarnation': 68, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.185:6835/3676791890', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 3676791890}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 3676791890}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_6642': {'gid': 6642, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6833/974001631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 974001631}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 974001631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:18:59.349 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:18:59.350 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:18:59.490 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":6185,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":1.4667827250000001,"mdsmap_epoch":71,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":14.022534464} 2020-11-16T22:18:59.491 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6185, "mdsmap_epoch": 71, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 1.466782725, "state": "up:active", "uptime": 14.022534464, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:18:59.491 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:18:59.634 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":6188,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.63635259,"mdsmap_epoch":71,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":14.143485571999999} 2020-11-16T22:18:59.635 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6188, "mdsmap_epoch": 71, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 2.63635259, "state": "up:active", "uptime": 14.143485572, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:18:59.635 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:18:59.769 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":6642,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.0477141670000005,"mdsmap_epoch":71,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":14.302671534} 2020-11-16T22:18:59.769 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6642, "mdsmap_epoch": 71, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 9.047714167, "state": "up:active", "uptime": 14.302671534, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:18:59.769 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:18:59.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 Processor -- start 2020-11-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- start start 2020-11-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7eff180b72e0 0x7eff180b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7eff180b64b0 0x7eff180b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff18147f90 con 0x7eff180acee0 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff180af020 con 0x7eff180b72e0 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff180b4810 con 0x7eff180b64b0 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7eff180b64b0 0x7eff180b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7eff180b64b0 0x7eff180b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33206/0 (socket says 172.21.15.185:33206) 2020-11-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1535174321 learned_addr learned my addr 172.21.15.185:0/1535174321 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:59.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1d1fe700 1 --2- 172.21.15.185:0/1535174321 >> v2:172.21.15.96:3300/0 conn(0x7eff180b72e0 0x7eff180b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff0ffff700 1 -- 172.21.15.185:0/1535174321 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516482116 0 0) 0x7eff18147f90 con 0x7eff180acee0 2020-11-16T22:18:59.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff0ffff700 1 -- 172.21.15.185:0/1535174321 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff04003240 con 0x7eff180acee0 2020-11-16T22:18:59.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1535174321 >> v2:172.21.15.96:3300/0 conn(0x7eff180b72e0 msgr2=0x7eff180b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:18:59.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 --2- 172.21.15.185:0/1535174321 >> v2:172.21.15.96:3300/0 conn(0x7eff180b72e0 0x7eff180b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1535174321 >> v1:172.21.15.96:6789/0 conn(0x7eff180acee0 legacy=0x7eff18144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1535174321 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18148460 con 0x7eff180b64b0 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1c9fd700 1 --2- 172.21.15.185:0/1535174321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff180b6aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7eff180b71a0 tx=0x7eff10006eb0).ready entity=mon.1 client_cookie=d152dc21689c2d63 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff0ffff700 1 -- 172.21.15.185:0/1535174321 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10009720 con 0x7eff180b64b0 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff0ffff700 1 -- 172.21.15.185:0/1535174321 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff10009880 con 0x7eff180b64b0 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff0ffff700 1 -- 172.21.15.185:0/1535174321 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10009c00 con 0x7eff180b64b0 2020-11-16T22:18:59.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- 172.21.15.185:0/1535174321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 msgr2=0x7eff180b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 --2- 172.21.15.185:0/1535174321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff180b6aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7eff180b71a0 tx=0x7eff10006eb0).stop 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- 172.21.15.185:0/1535174321 shutdown_connections 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 --2- 172.21.15.185:0/1535174321 >> v2:172.21.15.96:3300/0 conn(0x7eff180b72e0 0x7eff180b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 --2- 172.21.15.185:0/1535174321 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff180b6aa0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- 172.21.15.185:0/1535174321 >> 172.21.15.185:0/1535174321 conn(0x7eff18067860 msgr2=0x7eff18067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- 172.21.15.185:0/1535174321 shutdown_connections 2020-11-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.916+0000 7eff1e200700 1 -- 172.21.15.185:0/1535174321 wait complete. 2020-11-16T22:18:59.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 Processor -- start 2020-11-16T22:18:59.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 -- start start 2020-11-16T22:18:59.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff180acee0 0x7eff181466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff180b72e0 0x7eff181400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181405a0 con 0x7eff180acee0 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff180b2ab0 con 0x7eff180b64b0 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.920+0000 7eff1e200700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff180b2d50 con 0x7eff180b72e0 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33212/0 (socket says 172.21.15.185:33212) 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1908454005 learned_addr learned my addr 172.21.15.185:0/1908454005 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1d9ff700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff180b72e0 0x7eff181400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1d1fe700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff180acee0 0x7eff181466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:18:59.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff180b72e0 msgr2=0x7eff181400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff180b72e0 0x7eff181400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1908454005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff180acee0 msgr2=0x7eff181466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff180acee0 0x7eff181466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 -- 172.21.15.185:0/1908454005 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180b2ff0 con 0x7eff180b64b0 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1c9fd700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7eff10006ef0 tx=0x7eff10015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:18:59.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10011b40 con 0x7eff180b64b0 2020-11-16T22:18:59.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff10011ca0 con 0x7eff180b64b0 2020-11-16T22:18:59.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff180b3290 con 0x7eff180b64b0 2020-11-16T22:18:59.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1001e670 con 0x7eff180b64b0 2020-11-16T22:18:59.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff18146c50 con 0x7eff180b64b0 2020-11-16T22:18:59.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1805d200 con 0x7eff180b64b0 2020-11-16T22:18:59.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.924+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7eff10025030 con 0x7eff180b64b0 2020-11-16T22:18:59.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.928+0000 7eff0d7fa700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 0x7efef0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:18:59.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.928+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff1006fa80 con 0x7eff180b64b0 2020-11-16T22:18:59.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.928+0000 7eff1d1fe700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 0x7efef0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:19:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.928+0000 7eff1d1fe700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 0x7efef0057520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7eff14006fd0 tx=0x7eff140091a0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:19:00.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:18:59.932+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7eff10045e10 con 0x7eff180b64b0 2020-11-16T22:19:00.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff180b04a0 con 0x7eff180b64b0 2020-11-16T22:19:00.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff0d7fa700 1 -- 172.21.15.185:0/1908454005 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+2518 (secure 0 0 0) 0x7eff1002d070 con 0x7eff180b64b0 2020-11-16T22:19:00.119 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:19:00.120 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:18:59.037585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:19:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 msgr2=0x7efef0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:19:00.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 0x7efef0057520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7eff14006fd0 tx=0x7eff140091a0).stop 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 msgr2=0x7eff1813fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7eff10006ef0 tx=0x7eff10015000).stop 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 shutdown_connections 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efef0055060 0x7efef0057520 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7eff180b72e0 0x7eff181400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff180acee0 0x7eff181466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:19:00.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 --2- 172.21.15.185:0/1908454005 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7eff180b64b0 0x7eff1813fbc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:19:00.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 >> 172.21.15.185:0/1908454005 conn(0x7eff18067860 msgr2=0x7eff180b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:19:00.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 shutdown_connections 2020-11-16T22:19:00.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:19:00.136+0000 7eff1e200700 1 -- 172.21.15.185:0/1908454005 wait complete. 2020-11-16T22:19:00.123 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 71 2020-11-16T22:19:00.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:19:00.134 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:19:00.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:19:00.206 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:19:00.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:19:00.475 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:19:00.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:19:00.515 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:19:00.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:19:00.597 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:19:00.635 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:00.772 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:19:00.773 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:19:00.773 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:01.351 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:19:01.373+0000 7f5b056f8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:19:01.351 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:19:01.373+0000 7f5b056f8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:19:02.775 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:02.916 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:19:02.916 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:19:02.917 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:04.918 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:05.059 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:19:05.059 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:19:05.060 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:07.061 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:07.199 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:19:07.200 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:19:07.200 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:09.201 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:09.338 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:19:09.339 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:19:09.339 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:11.341 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:19:11.482 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:19:11.483 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:19:11.483 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:19:11.484 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:19:11.627 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:19:11.627 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:19:11.628 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:19:13.629 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:19:13.766 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:19:13.767 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:19:13.768 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:19:15.768 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:20:37.470 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:20:37.476 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:20:37.478 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:20:37.605 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:20:37.810 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:20:38.014 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:20:38.015 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:20:38.154 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:20:38.358 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:20:38.359 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:20:38.360 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:20:38.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 Processor -- start 2020-11-16T22:20:38.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- start start 2020-11-16T22:20:38.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc7d80b7470 0x7fc7d80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:38.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d8143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:38.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d8067510 con 0x7fc7d8067070 2020-11-16T22:20:38.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d80677b0 con 0x7fc7d80b7470 2020-11-16T22:20:38.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b17a0 con 0x7fc7d80682b0 2020-11-16T22:20:38.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd812700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc7d8067070 0x7fc7d80b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54890/0 (socket says 172.21.15.185:54890) 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd812700 1 -- 172.21.15.185:0/1704593418 learned_addr learned my addr 172.21.15.185:0/1704593418 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d8143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de013700 1 --2- 172.21.15.185:0/1704593418 >> v2:172.21.15.96:3300/0 conn(0x7fc7d80b7470 0x7fc7d80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 -- 172.21.15.185:0/1704593418 >> v2:172.21.15.96:3300/0 conn(0x7fc7d80b7470 msgr2=0x7fc7d80b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 --2- 172.21.15.185:0/1704593418 >> v2:172.21.15.96:3300/0 conn(0x7fc7d80b7470 0x7fc7d80b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:38.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 -- 172.21.15.185:0/1704593418 >> v1:172.21.15.185:6789/0 conn(0x7fc7d8067070 legacy=0x7fc7d80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:38.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 -- 172.21.15.185:0/1704593418 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80b1a40 con 0x7fc7d80682b0 2020-11-16T22:20:38.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd011700 1 -- 172.21.15.185:0/1704593418 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322642323 0 0) 0x7fc7d8067510 con 0x7fc7d8067070 2020-11-16T22:20:38.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7de814700 1 --2- 172.21.15.185:0/1704593418 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d8143a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc7d400b9a0 tx=0x7fc7d4006df0).ready entity=mon.2 client_cookie=e5d97aa8baa56c1c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:38.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd011700 1 -- 172.21.15.185:0/1704593418 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40097b0 con 0x7fc7d80682b0 2020-11-16T22:20:38.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd011700 1 -- 172.21.15.185:0/1704593418 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d4009910 con 0x7fc7d80682b0 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7dd011700 1 -- 172.21.15.185:0/1704593418 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4009c90 con 0x7fc7d80682b0 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- 172.21.15.185:0/1704593418 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 msgr2=0x7fc7d8143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 --2- 172.21.15.185:0/1704593418 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d8143a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc7d400b9a0 tx=0x7fc7d4006df0).stop 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- 172.21.15.185:0/1704593418 shutdown_connections 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 --2- 172.21.15.185:0/1704593418 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d8143a20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:38.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 --2- 172.21.15.185:0/1704593418 >> v2:172.21.15.96:3300/0 conn(0x7fc7d80b7470 0x7fc7d80b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:38.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- 172.21.15.185:0/1704593418 >> 172.21.15.185:0/1704593418 conn(0x7fc7d80637b0 msgr2=0x7fc7d80b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:38.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- 172.21.15.185:0/1704593418 shutdown_connections 2020-11-16T22:20:38.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.703+0000 7fc7df015700 1 -- 172.21.15.185:0/1704593418 wait complete. 2020-11-16T22:20:38.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 Processor -- start 2020-11-16T22:20:38.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- start start 2020-11-16T22:20:38.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:38.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:38.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 0x7fc7d8141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d813db20 con 0x7fc7d80b7470 2020-11-16T22:20:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8143a50 con 0x7fc7d8067070 2020-11-16T22:20:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8143cf0 con 0x7fc7d80682b0 2020-11-16T22:20:39.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33226/0 (socket says 172.21.15.185:33226) 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 -- 172.21.15.185:0/517432842 learned_addr learned my addr 172.21.15.185:0/517432842 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7dd812700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de814700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 0x7fc7d8141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 -- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 msgr2=0x7fc7d80b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:39.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d80b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 -- 172.21.15.185:0/517432842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 msgr2=0x7fc7d8141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 0x7fc7d8141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 -- 172.21.15.185:0/517432842 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d813d680 con 0x7fc7d8067070 2020-11-16T22:20:39.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de814700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 0x7fc7d8141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:20:39.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7dd812700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d80b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:39.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7de013700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc7cc00db90 tx=0x7fc7cc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc009790 con 0x7fc7d8067070 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0098f0 con 0x7fc7d8067070 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc009c70 con 0x7fc7d8067070 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d80b6250 con 0x7fc7d8067070 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d80b6c40 con 0x7fc7d8067070 2020-11-16T22:20:39.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.707+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d813d880 con 0x7fc7d8067070 2020-11-16T22:20:39.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.711+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc7cc013480 con 0x7fc7d8067070 2020-11-16T22:20:39.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.711+0000 7fc7ca7fc700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 0x7fc7b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:39.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.711+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7cc06e150 con 0x7fc7d8067070 2020-11-16T22:20:39.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.711+0000 7fc7dd812700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 0x7fc7b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:39.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.711+0000 7fc7dd812700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 0x7fc7b0057520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc7d813c2d0 tx=0x7fc7d0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:39.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.719+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc7d813d880 con 0x7fc7d8067070 2020-11-16T22:20:39.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7d805d200 con 0x7fc7d8067070 2020-11-16T22:20:39.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7ca7fc700 1 -- 172.21.15.185:0/517432842 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2524 (secure 0 0 0) 0x7fc7cc044350 con 0x7fc7d8067070 2020-11-16T22:20:39.527 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:20:39.527 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:19:21.246259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:20:39.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 msgr2=0x7fc7b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:39.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 0x7fc7b0057520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fc7d813c2d0 tx=0x7fc7d0009040).stop 2020-11-16T22:20:39.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 msgr2=0x7fc7d813bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:39.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc7cc00db90 tx=0x7fc7cc008040).stop 2020-11-16T22:20:39.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 shutdown_connections 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc7b0055060 0x7fc7b0057520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc7d8067070 0x7fc7d813bde0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7d80b7470 0x7fc7d8141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 --2- 172.21.15.185:0/517432842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc7d80682b0 0x7fc7d80b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 >> 172.21.15.185:0/517432842 conn(0x7fc7d80637b0 msgr2=0x7fc7d80abf40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 shutdown_connections 2020-11-16T22:20:39.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:38.915+0000 7fc7df015700 1 -- 172.21.15.185:0/517432842 wait complete. 2020-11-16T22:20:39.732 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 74 2020-11-16T22:20:39.732 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:20:39.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 Processor -- start 2020-11-16T22:20:39.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 -- start start 2020-11-16T22:20:39.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 0x7f7c640ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:39.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7c640b4470 0x7f7c640b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:39.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640b7290 con 0x7f7c640b5d70 2020-11-16T22:20:40.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640b4bf0 con 0x7f7c640b4470 2020-11-16T22:20:40.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c69251700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640b74c0 con 0x7f7c641475d0 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c637fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7c640b4470 0x7f7c640b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54912/0 (socket says 172.21.15.185:54912) 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.875+0000 7f7c68a50700 1 -- 172.21.15.185:0/2907578773 learned_addr learned my addr 172.21.15.185:0/2907578773 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c63fff700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 0x7f7c640ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2907578773 >> v2:172.21.15.185:3300/0 conn(0x7f7c640b4470 msgr2=0x7f7c640b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.185:3300/0 conn(0x7f7c640b4470 0x7f7c640b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2907578773 >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 msgr2=0x7f7c640ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 0x7f7c640ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2907578773 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640b7760 con 0x7f7c641475d0 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c63fff700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 0x7f7c640ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:40.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2907578773 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f7c64080bf0 tx=0x7f7c5c009290).ready entity=mon.2 client_cookie=40caf64796b3103a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:40.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c62ffd700 1 -- 172.21.15.185:0/2907578773 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00bbc0 con 0x7f7c641475d0 2020-11-16T22:20:40.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c62ffd700 1 -- 172.21.15.185:0/2907578773 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c00bd20 con 0x7f7c641475d0 2020-11-16T22:20:40.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c62ffd700 1 -- 172.21.15.185:0/2907578773 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c011610 con 0x7f7c641475d0 2020-11-16T22:20:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- 172.21.15.185:0/2907578773 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 msgr2=0x7f7c64147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- 172.21.15.185:0/2907578773 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f7c64080bf0 tx=0x7f7c5c009290).stop 2020-11-16T22:20:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- 172.21.15.185:0/2907578773 shutdown_connections 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- 172.21.15.185:0/2907578773 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c641475d0 0x7f7c64147a80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.96:3300/0 conn(0x7f7c640b5d70 0x7f7c640ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- 172.21.15.185:0/2907578773 >> v2:172.21.15.185:3300/0 conn(0x7f7c640b4470 0x7f7c640b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- 172.21.15.185:0/2907578773 >> 172.21.15.185:0/2907578773 conn(0x7f7c6405ba80 msgr2=0x7f7c6405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- 172.21.15.185:0/2907578773 shutdown_connections 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- 172.21.15.185:0/2907578773 wait complete. 2020-11-16T22:20:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 Processor -- start 2020-11-16T22:20:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- start start 2020-11-16T22:20:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c640b4470 0x7f7c640ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c640b5d70 0x7f7c640b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64143f10 con 0x7f7c640b4470 2020-11-16T22:20:40.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c641441b0 con 0x7f7c641475d0 2020-11-16T22:20:40.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c69251700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64144450 con 0x7f7c640b5d70 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c640b5d70 0x7f7c640b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33244/0 (socket says 172.21.15.185:33244) 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2295431822 learned_addr learned my addr 172.21.15.185:0/2295431822 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c63fff700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c640b4470 0x7f7c640ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c640b5d70 msgr2=0x7f7c640b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:40.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c640b5d70 0x7f7c640b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2295431822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c640b4470 msgr2=0x7f7c640ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:40.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c640b4470 0x7f7c640ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 -- 172.21.15.185:0/2295431822 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c641446f0 con 0x7f7c641475d0 2020-11-16T22:20:40.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.879+0000 7f7c68a50700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7c5c0092d0 tx=0x7f7c5c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:40.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 -- 172.21.15.185:0/2295431822 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c003290 con 0x7f7c641475d0 2020-11-16T22:20:40.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 -- 172.21.15.185:0/2295431822 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c011e60 con 0x7f7c641475d0 2020-11-16T22:20:40.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c640aed50 con 0x7f7c641475d0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 -- 172.21.15.185:0/2295431822 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c010da0 con 0x7f7c641475d0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c640ba9c0 con 0x7f7c641475d0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 -- 172.21.15.185:0/2295431822 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7c5c023030 con 0x7f7c641475d0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 0x7f7c40057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c5bfff700 1 -- 172.21.15.185:0/2295431822 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7c5c06f4b0 con 0x7f7c641475d0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c63fff700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 0x7f7c40057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.883+0000 7f7c63fff700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 0x7f7c40057520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f7c54006fd0 tx=0x7f7c54008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:40.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 Processor -- start 2020-11-16T22:20:40.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 -- start start 2020-11-16T22:20:40.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 0xf4d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:40.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xdda4b0 con 0xf4eda0 2020-11-16T22:20:40.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe8e930 con 0xd307b0 2020-11-16T22:20:40.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.887+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xd4e6f0 con 0xcc2bb0 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33248/0 (socket says 172.21.15.185:33248) 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 -- 172.21.15.185:0/928386918 learned_addr learned my addr 172.21.15.185:0/928386918 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c68a50700 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 0xf4d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:40.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 -- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 msgr2=0xf4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:40.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 -- 172.21.15.185:0/928386918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 msgr2=0xf4d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:40.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 0xf4d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:40.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 -- 172.21.15.185:0/928386918 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd495a0 con 0xd307b0 2020-11-16T22:20:40.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c68a50700 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 0xf4d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:41.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c63fff700 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f7c54012dd0 tx=0x7f7c54012000).ready entity=mon.1 client_cookie=de730659a98358af server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:41.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c597fa700 1 -- 172.21.15.185:0/928386918 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54012530 con 0xd307b0 2020-11-16T22:20:41.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c597fa700 1 -- 172.21.15.185:0/928386918 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c54011d80 con 0xd307b0 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c597fa700 1 -- 172.21.15.185:0/928386918 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5401e620 con 0xd307b0 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 msgr2=0xf4e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f7c54012dd0 tx=0x7f7c54012000).stop 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- 172.21.15.185:0/928386918 shutdown_connections 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf4e8b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf4eda0 0xf4d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 --2- 172.21.15.185:0/928386918 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- 172.21.15.185:0/928386918 >> 172.21.15.185:0/928386918 conn(0xefee40 msgr2=0xf4bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- 172.21.15.185:0/928386918 shutdown_connections 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- 172.21.15.185:0/928386918 wait complete. 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 -- start start 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.891+0000 7f7c78f44740 1 Processor -- start 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xf51e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xf7cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 0xf7d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7b6e0 con 0xcc2bb0 2020-11-16T22:20:41.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7b820 con 0xf4eda0 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe7ca90 con 0xd307b0 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 10 client.0 ll_register_callbacks cb 0xe71ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xf7cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 0xf7d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xf7cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54932/0 (socket says 172.21.15.185:54932) 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c637fe700 1 -- 172.21.15.185:0/1486098348 learned_addr learned my addr 172.21.15.185:0/1486098348 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:41.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xf51e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 -- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 msgr2=0xf7cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:41.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xf7cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 -- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 msgr2=0xf51e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:41.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xf51e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 -- 172.21.15.185:0/1486098348 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe6c280 con 0xf4eda0 2020-11-16T22:20:41.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 0xf7d100 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f7c5c049180 tx=0x7f7c5c06f040).ready entity=mon.1 client_cookie=4f68020a2d1d0cf0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:41.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:41.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00be90 con 0xf4eda0 2020-11-16T22:20:41.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7c5c0485f0 con 0xf4eda0 2020-11-16T22:20:41.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf2f6c0 con 0xf4eda0 2020-11-16T22:20:41.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c5c0487b0 con 0xf4eda0 2020-11-16T22:20:41.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 10 client.7000 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:41.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0xca1fd0 con 0xf4eda0 2020-11-16T22:20:41.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf2f6c0 con 0xf4eda0 2020-11-16T22:20:41.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f7c5c039020 con 0xf4eda0 2020-11-16T22:20:41.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 10 client.7000 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:41.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 10 client.7000 resolve_mds: resolved ID 'c' to GID 6642 2020-11-16T22:20:41.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 20 client.7000 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:41.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4f970 0xf96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 4 client.7000 mds_command: new command op to 6642 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:41.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 --> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xd6cc50 con 0xf4f970 2020-11-16T22:20:41.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4f970 0xf96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c3a7fc700 10 client.7000 ms_handle_connect on v2:172.21.15.96:6832/974001631 2020-11-16T22:20:41.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.895+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4f970 0xf96450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b838f33364ec718b server_cookie=75e70cc350537c32 in_seq=0 out_seq=0 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1486098348 <== mds.0 v2:172.21.15.96:6832/974001631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xd6cc50 con 0xf4f970 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c3a7fc700 10 client.7000 handle_command_reply: tid=0 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 client.7000 shutdown 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 msgr2=0xf7d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 0xf7d100 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f7c5c049180 tx=0x7f7c5c06f040).stop 2020-11-16T22:20:41.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 shutdown_connections 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4f970 0xf96450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xf4eda0 0xf7d100 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c3a7fc700 0 client.7000 ms_handle_reset on v2:172.21.15.96:6832/974001631 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xf51e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1486098348 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xf7cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 >> 172.21.15.185:0/1486098348 conn(0xefee40 msgr2=0xf4a330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:41.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 shutdown_connections 2020-11-16T22:20:41.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 1 -- 172.21.15.185:0/1486098348 wait complete. 2020-11-16T22:20:41.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.899+0000 7f7c78f44740 20 client.7000 trim_cache size 0 max 16384 2020-11-16T22:20:41.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 Processor -- start 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- start start 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 0xf4a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xe53e40 con 0xefee40 2020-11-16T22:20:41.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xf85330 con 0xd307b0 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xf4dac0 con 0xcc2bb0 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54938/0 (socket says 172.21.15.185:54938) 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 -- 172.21.15.185:0/1642415591 learned_addr learned my addr 172.21.15.185:0/1642415591 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 0xf4a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:41.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 -- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 msgr2=0xf49380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:41.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf49380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 -- 172.21.15.185:0/1642415591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 msgr2=0xf4a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:42.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 0xf4a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 -- 172.21.15.185:0/1642415591 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30008160 con 0xcc2bb0 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 0xf4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c637fe700 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f7c4c00cdc0 tx=0x7f7c4c006df0).ready entity=mon.2 client_cookie=96cb43161a180c94 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c59ffb700 1 -- 172.21.15.185:0/1642415591 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01a070 con 0xcc2bb0 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c59ffb700 1 -- 172.21.15.185:0/1642415591 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c015b30 con 0xcc2bb0 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c59ffb700 1 -- 172.21.15.185:0/1642415591 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00c6b0 con 0xcc2bb0 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 msgr2=0xf4fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:42.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f7c4c00cdc0 tx=0x7f7c4c006df0).stop 2020-11-16T22:20:42.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- 172.21.15.185:0/1642415591 shutdown_connections 2020-11-16T22:20:42.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xd307b0 0xf49380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xefee40 0xf4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1642415591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xcc2bb0 0xf4fd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- 172.21.15.185:0/1642415591 >> 172.21.15.185:0/1642415591 conn(0x7f7c30002320 msgr2=0xf4e490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:42.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- 172.21.15.185:0/1642415591 shutdown_connections 2020-11-16T22:20:42.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- 172.21.15.185:0/1642415591 wait complete. 2020-11-16T22:20:42.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 -- start start 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 Processor -- start 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xfe4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xfe53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.919+0000 7f7c78f44740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xf839b0 con 0xcc2bb0 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xf4e220 con 0xefee40 2020-11-16T22:20:42.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xf81f30 con 0xd307b0 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 10 client.0 ll_register_callbacks cb 0xf48c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xfe53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c637fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xfe53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54946/0 (socket says 172.21.15.185:54946) 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c637fe700 1 -- 172.21.15.185:0/1178721461 learned_addr learned my addr 172.21.15.185:0/1178721461 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:42.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33272/0 (socket says 172.21.15.185:33272) 2020-11-16T22:20:42.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xfe4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:42.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 -- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 msgr2=0xfe53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:42.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xfe53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 -- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 msgr2=0xfe4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xfe4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 -- 172.21.15.185:0/1178721461 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xce0720 con 0xefee40 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c68a50700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f7c5c049180 tx=0x7f7c5c02b000).ready entity=mon.1 client_cookie=fed7b49a0bc91bc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00be90 con 0xefee40 2020-11-16T22:20:42.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7c5c0485f0 con 0xefee40 2020-11-16T22:20:42.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c54011d80 con 0xefee40 2020-11-16T22:20:42.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c5c006b20 con 0xefee40 2020-11-16T22:20:42.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 10 client.7006 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:42.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0xcebe00 con 0xefee40 2020-11-16T22:20:42.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c54011d80 con 0xefee40 2020-11-16T22:20:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f7c5c042050 con 0xefee40 2020-11-16T22:20:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 10 client.7006 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 10 client.7006 resolve_mds: resolved ID 'c' to GID 6642 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 20 client.7006 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4eda0 0xfe1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 4 client.7006 mds_command: new command op to 6642 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 --> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0xd73f50 con 0xf4eda0 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4eda0 0xfe1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c63fff700 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4eda0 0xfe1d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bbc3ac3a67b48633 server_cookie=122c90d00efca06b in_seq=0 out_seq=0 2020-11-16T22:20:42.743 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.923+0000 7f7c3a7fc700 10 client.7006 ms_handle_connect on v2:172.21.15.96:6832/974001631 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.943+0000 7f7c3a7fc700 1 -- 172.21.15.185:0/1178721461 <== mds.0 v2:172.21.15.96:6832/974001631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0xd73f50 con 0xf4eda0 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.943+0000 7f7c3a7fc700 10 client.7006 handle_command_reply: tid=0 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.943+0000 7f7c78f44740 1 client.7006 shutdown 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 msgr2=0xfe1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f7c5c049180 tx=0x7f7c5c02b000).stop 2020-11-16T22:20:42.744 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 shutdown_connections 2020-11-16T22:20:42.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:6832/974001631,v1:172.21.15.96:6833/974001631] conn(0xf4eda0 0xfe1d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xefee40 0xfe1860 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c3a7fc700 0 client.7006 ms_handle_reset on v2:172.21.15.96:6832/974001631 2020-11-16T22:20:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xcc2bb0 0xfe4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 --2- 172.21.15.185:0/1178721461 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xd307b0 0xfe53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 >> 172.21.15.185:0/1178721461 conn(0x7f7c30002320 msgr2=0xf49380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 shutdown_connections 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 1 -- 172.21.15.185:0/1178721461 wait complete. 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c78f44740 20 client.7006 trim_cache size 0 max 16384 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 msgr2=0x7f7c40057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 0x7f7c40057520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f7c54006fd0 tx=0x7f7c54008040).stop 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 msgr2=0x7f7c640af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7c5c0092d0 tx=0x7f7c5c016000).stop 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 shutdown_connections 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7c40055060 0x7f7c40057520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7c641475d0 0x7f7c640af1f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c640b4470 0x7f7c640ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 --2- 172.21.15.185:0/2295431822 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7c640b5d70 0x7f7c640b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 >> 172.21.15.185:0/2295431822 conn(0x7f7c6405ba80 msgr2=0x7f7c64140520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 shutdown_connections 2020-11-16T22:20:43.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:39.947+0000 7f7c69251700 1 -- 172.21.15.185:0/2295431822 wait complete. 2020-11-16T22:20:43.151 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:20:43.151 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:20:43.151 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:20:43.151 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:20:43.152 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:20:43.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 Processor -- start 2020-11-16T22:20:43.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 -- start start 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0380b64b0 0x7fa0380b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa038147f90 con 0x7fa0380b6aa0 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa038148460 con 0x7fa0380b72e0 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03f03f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380b6eb0 con 0x7fa0380b64b0 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03e03d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa0380b72e0 0x7fa038144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:54946/0 (socket says 172.21.15.185:54946) 2020-11-16T22:20:43.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03e03d700 1 -- 172.21.15.185:0/358460294 learned_addr learned my addr 172.21.15.185:0/358460294 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:43.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d83c700 1 --2- 172.21.15.185:0/358460294 >> v2:172.21.15.96:3300/0 conn(0x7fa0380b64b0 0x7fa0380b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d03b700 1 -- 172.21.15.185:0/358460294 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967326666 0 0) 0x7fa038148460 con 0x7fa0380b72e0 2020-11-16T22:20:43.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d03b700 1 -- 172.21.15.185:0/358460294 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa020003240 con 0x7fa0380b72e0 2020-11-16T22:20:43.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d83c700 1 -- 172.21.15.185:0/358460294 >> v1:172.21.15.185:6789/0 conn(0x7fa0380b72e0 legacy=0x7fa038144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d83c700 1 -- 172.21.15.185:0/358460294 >> v1:172.21.15.96:6789/0 conn(0x7fa0380b6aa0 legacy=0x7fa03813dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d83c700 1 -- 172.21.15.185:0/358460294 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380b4810 con 0x7fa0380b64b0 2020-11-16T22:20:43.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.295+0000 7fa03d83c700 1 --2- 172.21.15.185:0/358460294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b76c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa02800b9a0 tx=0x7fa028006df0).ready entity=mon.0 client_cookie=a4e1cecf361c6b27 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:43.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d03b700 1 -- 172.21.15.185:0/358460294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280097b0 con 0x7fa0380b64b0 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d03b700 1 -- 172.21.15.185:0/358460294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028009910 con 0x7fa0380b64b0 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d03b700 1 -- 172.21.15.185:0/358460294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028009c90 con 0x7fa0380b64b0 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- 172.21.15.185:0/358460294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 msgr2=0x7fa0380b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 --2- 172.21.15.185:0/358460294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b76c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa02800b9a0 tx=0x7fa028006df0).stop 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- 172.21.15.185:0/358460294 shutdown_connections 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 --2- 172.21.15.185:0/358460294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b76c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- 172.21.15.185:0/358460294 >> 172.21.15.185:0/358460294 conn(0x7fa038067860 msgr2=0x7fa038067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- 172.21.15.185:0/358460294 shutdown_connections 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- 172.21.15.185:0/358460294 wait complete. 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 Processor -- start 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- start start 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 0x7fa0380afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381543a0 con 0x7fa0380b64b0 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa038153510 con 0x7fa0380b6aa0 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03f03f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa038153770 con 0x7fa0380b72e0 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d83c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d83c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33284/0 (socket says 172.21.15.185:33284) 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03d83c700 1 -- 172.21.15.185:0/3024439527 learned_addr learned my addr 172.21.15.185:0/3024439527 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03e03d700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.299+0000 7fa03e83e700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 0x7fa0380afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 -- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 msgr2=0x7fa0380afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 0x7fa0380afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 -- 172.21.15.185:0/3024439527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 msgr2=0x7fa0380b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 -- 172.21.15.185:0/3024439527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa038153a10 con 0x7fa0380b6aa0 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03e83e700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 0x7fa0380afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:43.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03d83c700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fa028006e30 tx=0x7fa028015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028006760 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0380b3960 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028011d50 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02801e670 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0380b0270 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa038153ba0 con 0x7fa0380b6aa0 2020-11-16T22:20:43.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa028025030 con 0x7fa0380b6aa0 2020-11-16T22:20:43.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 0x7fa014057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa02806f270 con 0x7fa0380b6aa0 2020-11-16T22:20:43.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.303+0000 7fa03e03d700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 0x7fa014057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.307+0000 7fa03e03d700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 0x7fa014057520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fa038140890 tx=0x7fa0340027c0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:43.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.311+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa028045580 con 0x7fa0380b6aa0 2020-11-16T22:20:43.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.515+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa03805d200 con 0x7fa0380b6aa0 2020-11-16T22:20:43.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.515+0000 7fa0267fc700 1 -- 172.21.15.185:0/3024439527 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2524 (secure 0 0 0) 0x7fa028023020 con 0x7fa0380b6aa0 2020-11-16T22:20:43.507 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:20:43.508 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:19:21.246259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:20:43.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 msgr2=0x7fa014057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 0x7fa014057520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fa038140890 tx=0x7fa0340027c0).stop 2020-11-16T22:20:43.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 msgr2=0x7fa038153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fa028006e30 tx=0x7fa028015000).stop 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 shutdown_connections 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa014055060 0x7fa014057520 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0380b72e0 0x7fa0380afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0380b64b0 0x7fa0380b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 --2- 172.21.15.185:0/3024439527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0380b6aa0 0x7fa038153eb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 >> 172.21.15.185:0/3024439527 conn(0x7fa038067860 msgr2=0x7fa038144790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:43.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 shutdown_connections 2020-11-16T22:20:43.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.519+0000 7fa03f03f700 1 -- 172.21.15.185:0/3024439527 wait complete. 2020-11-16T22:20:43.511 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 74 2020-11-16T22:20:43.522 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:20:43.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 Processor -- start 2020-11-16T22:20:43.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 -- start start 2020-11-16T22:20:43.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd3100b5d70 0x7fd3100ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100b7290 con 0x7fd3101475d0 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100b4bf0 con 0x7fd3100b5d70 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd314c19700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100b74c0 con 0x7fd3100b4470 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54970/0 (socket says 172.21.15.185:54970) 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 -- 172.21.15.185:0/70569367 learned_addr learned my addr 172.21.15.185:0/70569367 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:43.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/70569367 >> v2:172.21.15.96:3300/0 conn(0x7fd3100b5d70 0x7fd3100ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 -- 172.21.15.185:0/70569367 >> v2:172.21.15.96:3300/0 conn(0x7fd3100b5d70 msgr2=0x7fd3100ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 --2- 172.21.15.185:0/70569367 >> v2:172.21.15.96:3300/0 conn(0x7fd3100b5d70 0x7fd3100ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 -- 172.21.15.185:0/70569367 >> v1:172.21.15.96:6789/0 conn(0x7fd3101475d0 legacy=0x7fd310147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30e7fc700 1 -- 172.21.15.185:0/70569367 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806839259 0 0) 0x7fd3100b7290 con 0x7fd3101475d0 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 -- 172.21.15.185:0/70569367 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100b7760 con 0x7fd3100b4470 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30effd700 1 --2- 172.21.15.185:0/70569367 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd304006fd0 tx=0x7fd304006eb0).ready entity=mon.2 client_cookie=2054647487cd4ba3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:43.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30e7fc700 1 -- 172.21.15.185:0/70569367 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304009810 con 0x7fd3100b4470 2020-11-16T22:20:43.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30e7fc700 1 -- 172.21.15.185:0/70569367 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd304009970 con 0x7fd3100b4470 2020-11-16T22:20:43.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.655+0000 7fd30e7fc700 1 -- 172.21.15.185:0/70569367 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304009cf0 con 0x7fd3100b4470 2020-11-16T22:20:43.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- 172.21.15.185:0/70569367 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 msgr2=0x7fd3100b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- 172.21.15.185:0/70569367 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd304006fd0 tx=0x7fd304006eb0).stop 2020-11-16T22:20:43.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- 172.21.15.185:0/70569367 shutdown_connections 2020-11-16T22:20:43.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- 172.21.15.185:0/70569367 >> v2:172.21.15.96:3300/0 conn(0x7fd3100b5d70 0x7fd3100ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- 172.21.15.185:0/70569367 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3100b4470 0x7fd3100b7c00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- 172.21.15.185:0/70569367 >> 172.21.15.185:0/70569367 conn(0x7fd31005ba80 msgr2=0x7fd31005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:43.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- 172.21.15.185:0/70569367 shutdown_connections 2020-11-16T22:20:43.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- 172.21.15.185:0/70569367 wait complete. 2020-11-16T22:20:43.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 Processor -- start 2020-11-16T22:20:43.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- start start 2020-11-16T22:20:43.865 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3100b5d70 0x7fd310156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3101475d0 0x7fd310157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100aee40 con 0x7fd3100b5d70 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100af0e0 con 0x7fd3100b4470 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd314c19700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd310156a70 con 0x7fd3101475d0 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3101475d0 0x7fd310157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33296/0 (socket says 172.21.15.185:33296) 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 -- 172.21.15.185:0/3769913156 learned_addr learned my addr 172.21.15.185:0/3769913156 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30effd700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3100b5d70 0x7fd310156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 -- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3101475d0 msgr2=0x7fd310157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3101475d0 0x7fd310157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 -- 172.21.15.185:0/3769913156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3100b5d70 msgr2=0x7fd310156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:43.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3100b5d70 0x7fd310156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:43.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 -- 172.21.15.185:0/3769913156 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100b9e00 con 0x7fd3100b4470 2020-11-16T22:20:43.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fd30000eba0 tx=0x7fd300006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:43.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.659+0000 7fd2f77fe700 1 -- 172.21.15.185:0/3769913156 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000b3a0 con 0x7fd3100b4470 2020-11-16T22:20:43.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.663+0000 7fd2f77fe700 1 -- 172.21.15.185:0/3769913156 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30000c9b0 con 0x7fd3100b4470 2020-11-16T22:20:43.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.663+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3101159d0 con 0x7fd3100b4470 2020-11-16T22:20:44.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.663+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3100b1620 con 0x7fd3100b4470 2020-11-16T22:20:44.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.663+0000 7fd2f77fe700 1 -- 172.21.15.185:0/3769913156 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000cd30 con 0x7fd3100b4470 2020-11-16T22:20:44.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f77fe700 1 -- 172.21.15.185:0/3769913156 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd300012600 con 0x7fd3100b4470 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 Processor -- start 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 -- start start 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f77fe700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 0x7fd2e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f77fe700 1 -- 172.21.15.185:0/3769913156 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd30006e1a0 con 0x7fd3100b4470 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30ffff700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 0x7fd2e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30ffff700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 0x7fd2e4057520 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd3080027a0 tx=0x7fd308007c10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:44.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24b27b0 0x26d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x255c4b0 con 0x24b27b0 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2610930 con 0x26d0da0 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d06f0 con 0x2444bb0 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54982/0 (socket says 172.21.15.185:54982) 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 -- 172.21.15.185:0/2674711840 learned_addr learned my addr 172.21.15.185:0/2674711840 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30ffff700 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24b27b0 0x26d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 -- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 msgr2=0x26cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 -- 172.21.15.185:0/2674711840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24b27b0 msgr2=0x26d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24b27b0 0x26d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 -- 172.21.15.185:0/2674711840 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24cb5a0 con 0x2444bb0 2020-11-16T22:20:44.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd30effd700 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd304010000 tx=0x7fd304015000).ready entity=mon.2 client_cookie=1b4fbbfbb23ea883 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:44.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f4ff9700 1 -- 172.21.15.185:0/2674711840 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304006800 con 0x2444bb0 2020-11-16T22:20:44.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f4ff9700 1 -- 172.21.15.185:0/2674711840 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd304006960 con 0x2444bb0 2020-11-16T22:20:44.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.667+0000 7fd2f4ff9700 1 -- 172.21.15.185:0/2674711840 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304009cf0 con 0x2444bb0 2020-11-16T22:20:44.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 msgr2=0x26cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:44.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd304010000 tx=0x7fd304015000).stop 2020-11-16T22:20:44.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/2674711840 shutdown_connections 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24b27b0 0x26d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- 172.21.15.185:0/2674711840 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2444bb0 0x26cee90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/2674711840 >> 172.21.15.185:0/2674711840 conn(0x2680e40 msgr2=0x26cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/2674711840 shutdown_connections 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/2674711840 wait complete. 2020-11-16T22:20:44.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- start start 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 Processor -- start 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x26d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x25fd6e0 con 0x2444bb0 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x25fd820 con 0x26d0da0 2020-11-16T22:20:44.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x25fea90 con 0x24b27b0 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 10 client.0 ll_register_callbacks cb 0x25f3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x26d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33314/0 (socket says 172.21.15.185:33314) 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 -- 172.21.15.185:0/232869325 learned_addr learned my addr 172.21.15.185:0/232869325 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x26d1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54988/0 (socket says 172.21.15.185:54988) 2020-11-16T22:20:44.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 -- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 msgr2=0x26d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x26d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 -- 172.21.15.185:0/232869325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 msgr2=0x26d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 -- 172.21.15.185:0/232869325 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25ee280 con 0x26d0da0 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:44.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd30ffff700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fd308016bf0 tx=0x7fd308018040).ready entity=mon.1 client_cookie=7e685f5a72f8a991 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd2e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30801b040 con 0x26d0da0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd308021c60 con 0x26d0da0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26b16c0 con 0x26d0da0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3080147c0 con 0x26d0da0 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 10 client.7021 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:44.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.671+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x2423fd0 con 0x26d0da0 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26b16c0 con 0x26d0da0 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd308027020 con 0x26d0da0 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 10 client.7021 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 10 client.7021 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 20 client.7021 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26feff0 0x2714d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 4 client.7021 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x24eec50 con 0x26feff0 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26feff0 0x2714d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26feff0 0x2714d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=63b7dcd569288319 server_cookie=eea1b137b8706774 in_seq=0 out_seq=0 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 10 client.7021 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/232869325 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x24eec50 con 0x26feff0 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 10 client.7021 handle_command_reply: tid=0 2020-11-16T22:20:44.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 client.7021 shutdown 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 msgr2=0x26d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fd308016bf0 tx=0x7fd308018040).stop 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 shutdown_connections 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26feff0 0x2714d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x26d0da0 0x26d1970 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd2e1ffb700 0 client.7021 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 --2- 172.21.15.185:0/232869325 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x26d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 >> 172.21.15.185:0/232869325 conn(0x2680e40 msgr2=0x26cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 shutdown_connections 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 1 -- 172.21.15.185:0/232869325 wait complete. 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.675+0000 7fd32490c740 20 client.7021 trim_cache size 0 max 16384 2020-11-16T22:20:44.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.691+0000 7fd32490c740 1 Processor -- start 2020-11-16T22:20:44.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- start start 2020-11-16T22:20:44.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2680e40 0x2705e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x27179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d5e40 con 0x2680e40 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b1300 con 0x2444bb0 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26d2c10 con 0x24b27b0 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x27179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x27179e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:54996/0 (socket says 172.21.15.185:54996) 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33322/0 (socket says 172.21.15.185:33322) 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30ffff700 1 -- 172.21.15.185:0/3940104780 learned_addr learned my addr 172.21.15.185:0/3940104780 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2680e40 0x2705e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 -- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 msgr2=0x27179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x27179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 -- 172.21.15.185:0/3940104780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2680e40 msgr2=0x2705e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2680e40 0x2705e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 -- 172.21.15.185:0/3940104780 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0081d0 con 0x2444bb0 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd30effd700 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fd3100b3090 tx=0x7fd304020000).ready entity=mon.1 client_cookie=95a1e2219784e736 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd2f57fa700 1 -- 172.21.15.185:0/3940104780 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304030070 con 0x2444bb0 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd2f57fa700 1 -- 172.21.15.185:0/3940104780 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3040067e0 con 0x2444bb0 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd2f57fa700 1 -- 172.21.15.185:0/3940104780 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304009cf0 con 0x2444bb0 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 msgr2=0x26cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:44.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fd3100b3090 tx=0x7fd304020000).stop 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- 172.21.15.185:0/3940104780 shutdown_connections 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x27179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2680e40 0x2705e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 --2- 172.21.15.185:0/3940104780 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2444bb0 0x26cc200 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- 172.21.15.185:0/3940104780 >> 172.21.15.185:0/3940104780 conn(0x7fd2dc002320 msgr2=0x26cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:44.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- 172.21.15.185:0/3940104780 shutdown_connections 2020-11-16T22:20:44.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- 172.21.15.185:0/3940104780 wait complete. 2020-11-16T22:20:44.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 -- start start 2020-11-16T22:20:44.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.695+0000 7fd32490c740 1 Processor -- start 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2680e40 0x2760a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26cf010 con 0x2444bb0 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26fde20 con 0x2680e40 2020-11-16T22:20:44.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26fe740 con 0x24b27b0 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 10 client.0 ll_register_callbacks cb 0x26ca9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2680e40 0x2760a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55002/0 (socket says 172.21.15.185:55002) 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 -- 172.21.15.185:0/3188698086 learned_addr learned my addr 172.21.15.185:0/3188698086 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:44.700 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 -- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2680e40 msgr2=0x2760a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2680e40 0x2760a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 -- 172.21.15.185:0/3188698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 msgr2=0x26fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:44.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 -- 172.21.15.185:0/3188698086 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2462720 con 0x24b27b0 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30effd700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fd304006ef0 tx=0x7fd304030000).ready entity=mon.2 client_cookie=863f30f024514a9f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304009cf0 con 0x24b27b0 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3040118f0 con 0x24b27b0 2020-11-16T22:20:44.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24f5f50 con 0x24b27b0 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd304009ea0 con 0x24b27b0 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 10 client.6356 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x246de00 con 0x24b27b0 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24f5f50 con 0x24b27b0 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fd304047070 con 0x24b27b0 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 10 client.6356 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:44.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 10 client.6356 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:44.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 20 client.6356 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:45.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26d0da0 0x2760f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:45.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 4 client.6356 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x24d31d0 con 0x26d0da0 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26d0da0 0x2760f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd2e1ffb700 10 client.6356 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.699+0000 7fd30f7fe700 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26d0da0 0x2760f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=a3f5d921b32f9d8e server_cookie=b10eeb07d097cb1f in_seq=0 out_seq=0 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd2e1ffb700 1 -- 172.21.15.185:0/3188698086 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x24d31d0 con 0x26d0da0 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd2e1ffb700 10 client.6356 handle_command_reply: tid=0 2020-11-16T22:20:45.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 client.6356 shutdown 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 msgr2=0x270ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fd304006ef0 tx=0x7fd304030000).stop 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 shutdown_connections 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x26d0da0 0x2760f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2680e40 0x2760a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd2e1ffb700 0 client.6356 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:45.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2444bb0 0x26fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 --2- 172.21.15.185:0/3188698086 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24b27b0 0x270ce90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 >> 172.21.15.185:0/3188698086 conn(0x7fd2dc002320 msgr2=0x26cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:45.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 shutdown_connections 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 1 -- 172.21.15.185:0/3188698086 wait complete. 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd32490c740 20 client.6356 trim_cache size 0 max 16384 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 msgr2=0x7fd2e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 0x7fd2e4057520 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fd3080027a0 tx=0x7fd308007c10).stop 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 msgr2=0x7fd3100b9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:45.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fd30000eba0 tx=0x7fd300006df0).stop 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 shutdown_connections 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd2e4055060 0x7fd2e4057520 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd3100b4470 0x7fd3100b9910 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.715+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd3101475d0 0x7fd310157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.719+0000 7fd314c19700 1 --2- 172.21.15.185:0/3769913156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd3100b5d70 0x7fd310156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.719+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 >> 172.21.15.185:0/3769913156 conn(0x7fd31005ba80 msgr2=0x7fd310147a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:45.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.719+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 shutdown_connections 2020-11-16T22:20:45.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:43.719+0000 7fd314c19700 1 -- 172.21.15.185:0/3769913156 wait complete. 2020-11-16T22:20:45.514 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:20:45.515 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:20:45.515 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:20:45.515 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:20:45.516 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:20:45.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 Processor -- start 2020-11-16T22:20:45.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 -- start start 2020-11-16T22:20:45.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:45.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9c6c0b1d30 0x7f9c6c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:45.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6c061fe0 con 0x7f9c6c0ad110 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b2b60 con 0x7f9c6c0b1d30 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c72dd2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b0fe0 con 0x7f9c6c0b3090 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9c6c0b1d30 0x7f9c6c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9c6c0b1d30 0x7f9c6c14b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33330/0 (socket says 172.21.15.185:33330) 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 -- 172.21.15.185:0/4263475055 learned_addr learned my addr 172.21.15.185:0/4263475055 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:45.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c71dd0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:45.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 -- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 msgr2=0x7f9c6c0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:45.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 --2- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 -- 172.21.15.185:0/4263475055 >> v1:172.21.15.96:6789/0 conn(0x7f9c6c0ad110 legacy=0x7f9c6c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:45.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 -- 172.21.15.185:0/4263475055 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c0b1280 con 0x7f9c6c0b1d30 2020-11-16T22:20:45.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c715cf700 1 --2- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c14b950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9c60009d50 tx=0x7f9c60009290).ready entity=mon.1 client_cookie=acb88adbdc6975f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:45.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c70dce700 1 -- 172.21.15.185:0/4263475055 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000bb80 con 0x7f9c6c0b1d30 2020-11-16T22:20:45.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c70dce700 1 -- 172.21.15.185:0/4263475055 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000bce0 con 0x7f9c6c0b1d30 2020-11-16T22:20:45.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.655+0000 7f9c70dce700 1 -- 172.21.15.185:0/4263475055 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60011610 con 0x7f9c6c0b1d30 2020-11-16T22:20:45.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0b1d30 msgr2=0x7f9c6c14b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:45.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c14b950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9c60009d50 tx=0x7f9c60009290).stop 2020-11-16T22:20:45.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- 172.21.15.185:0/4263475055 shutdown_connections 2020-11-16T22:20:45.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/4263475055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c14b950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- 172.21.15.185:0/4263475055 >> 172.21.15.185:0/4263475055 conn(0x7f9c6c060130 msgr2=0x7f9c6c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- 172.21.15.185:0/4263475055 shutdown_connections 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- 172.21.15.185:0/4263475055 wait complete. 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 Processor -- start 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.659+0000 7f9c72dd2700 1 -- start start 2020-11-16T22:20:45.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:46.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:46.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:46.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c10fa30 con 0x7f9c6c0b1d30 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1579a0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c157c40 con 0x7f9c6c0b3090 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33336/0 (socket says 172.21.15.185:33336) 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 -- 172.21.15.185:0/3251455290 learned_addr learned my addr 172.21.15.185:0/3251455290 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:46.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c725d1700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:46.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c715cf700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:46.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 -- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 msgr2=0x7f9c6c0b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:46.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 -- 172.21.15.185:0/3251455290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c6c0b1d30 msgr2=0x7f9c6c0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:46.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c0ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 -- 172.21.15.185:0/3251455290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c157ee0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c71dd0700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f9c6400eba0 tx=0x7f9c64006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:46.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400c9b0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6400cb10 con 0x7f9c6c0ad110 2020-11-16T22:20:46.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c0aed20 con 0x7f9c6c0ad110 2020-11-16T22:20:46.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400ce90 con 0x7f9c6c0ad110 2020-11-16T22:20:46.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c10fca0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c10fe90 con 0x7f9c6c0ad110 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.663+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9c64012600 con 0x7f9c6c0ad110 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.667+0000 7f9c69ffb700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 0x7f9c48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.667+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c6406e1b0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.667+0000 7f9c715cf700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 0x7f9c48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:46.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.667+0000 7f9c715cf700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 0x7f9c48057520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f9c600092d0 tx=0x7f9c60016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:46.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.675+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9c640443b0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.879+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c6c10f850 con 0x7f9c6c0ad110 2020-11-16T22:20:46.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.879+0000 7f9c69ffb700 1 -- 172.21.15.185:0/3251455290 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f9c640479d0 con 0x7f9c6c0ad110 2020-11-16T22:20:46.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 msgr2=0x7f9c48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:46.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 0x7f9c48057520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f9c600092d0 tx=0x7f9c60016000).stop 2020-11-16T22:20:46.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 msgr2=0x7f9c6c158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:46.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f9c6400eba0 tx=0x7f9c64006df0).stop 2020-11-16T22:20:46.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 shutdown_connections 2020-11-16T22:20:46.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9c48055060 0x7f9c48057520 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9c6c0ad110 0x7f9c6c158380 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9c6c0b3090 0x7f9c6c0b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 --2- 172.21.15.185:0/3251455290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c6c0b1d30 0x7f9c6c0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:46.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 >> 172.21.15.185:0/3251455290 conn(0x7f9c6c060130 msgr2=0x7f9c6c14da90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:46.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 shutdown_connections 2020-11-16T22:20:46.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:45.883+0000 7f9c72dd2700 1 -- 172.21.15.185:0/3251455290 wait complete. 2020-11-16T22:20:46.686 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 74 2020-11-16T22:20:46.686 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:20:46.687 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:19:21.246259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:20:46.687 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:20:46.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.831+0000 7f2323fff700 1 Processor -- start 2020-11-16T22:20:46.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.831+0000 7f2323fff700 1 -- start start 2020-11-16T22:20:46.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23240b3920 0x7f2324148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:46.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240ac120 con 0x7f23240b13d0 2020-11-16T22:20:46.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240ac3c0 con 0x7f23240b3080 2020-11-16T22:20:46.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f232414c6f0 con 0x7f23240b3920 2020-11-16T22:20:47.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23237fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f23240b3920 0x7f2324148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2322ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f23240b3080 0x7f23240ac860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55016/0 (socket says 172.21.15.185:55016) 2020-11-16T22:20:47.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2322ffd700 1 -- 172.21.15.185:0/2308527756 learned_addr learned my addr 172.21.15.185:0/2308527756 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:47.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23237fe700 1 -- 172.21.15.185:0/2308527756 >> v1:172.21.15.185:6789/0 conn(0x7f23240b3080 legacy=0x7f23240ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23227fc700 1 -- 172.21.15.185:0/2308527756 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222590912 0 0) 0x7f23240ac3c0 con 0x7f23240b3080 2020-11-16T22:20:47.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23237fe700 1 -- 172.21.15.185:0/2308527756 >> v1:172.21.15.96:6789/0 conn(0x7f23240b13d0 legacy=0x7f23240b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23237fe700 1 -- 172.21.15.185:0/2308527756 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23240b3460 con 0x7f23240b3920 2020-11-16T22:20:47.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23237fe700 1 --2- 172.21.15.185:0/2308527756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b3920 0x7f2324148750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f231c009c90 tx=0x7f231c0091d0).ready entity=mon.1 client_cookie=36976aaec35d1ee2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:47.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23227fc700 1 -- 172.21.15.185:0/2308527756 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00bbc0 con 0x7f23240b3920 2020-11-16T22:20:47.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23227fc700 1 -- 172.21.15.185:0/2308527756 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c00bd20 con 0x7f23240b3920 2020-11-16T22:20:47.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f23227fc700 1 -- 172.21.15.185:0/2308527756 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c011610 con 0x7f23240b3920 2020-11-16T22:20:47.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- 172.21.15.185:0/2308527756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b3920 msgr2=0x7f2324148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- 172.21.15.185:0/2308527756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b3920 0x7f2324148750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f231c009c90 tx=0x7f231c0091d0).stop 2020-11-16T22:20:47.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- 172.21.15.185:0/2308527756 shutdown_connections 2020-11-16T22:20:47.045 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- 172.21.15.185:0/2308527756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b3920 0x7f2324148750 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:47.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- 172.21.15.185:0/2308527756 >> 172.21.15.185:0/2308527756 conn(0x7f2324060130 msgr2=0x7f2324060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:47.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- 172.21.15.185:0/2308527756 shutdown_connections 2020-11-16T22:20:47.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- 172.21.15.185:0/2308527756 wait complete. 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 Processor -- start 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- start start 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 0x7f23241449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 0x7f2324144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232410fc20 con 0x7f23240b3920 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324145380 con 0x7f23240b13d0 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2323fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2324113850 con 0x7f23240b3080 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2322ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f2322ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33350/0 (socket says 172.21.15.185:33350) 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 -- 172.21.15.185:0/3439712790 learned_addr learned my addr 172.21.15.185:0/3439712790 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.835+0000 7f231a7fc700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 0x7f23241449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f23237fe700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 0x7f2324144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 -- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 msgr2=0x7f23241449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 0x7f23241449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:47.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 -- 172.21.15.185:0/3439712790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 msgr2=0x7f2324144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 0x7f2324144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 -- 172.21.15.185:0/3439712790 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23241104e0 con 0x7f23240b13d0 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f23237fe700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 0x7f2324144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f231a7fc700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 0x7f23241449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2322ffd700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f231400c790 tx=0x7f2314007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:47.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f231b7fe700 1 -- 172.21.15.185:0/3439712790 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140072e0 con 0x7f23240b13d0 2020-11-16T22:20:47.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f231b7fe700 1 -- 172.21.15.185:0/3439712790 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2314007440 con 0x7f23240b13d0 2020-11-16T22:20:47.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2324110780 con 0x7f23240b13d0 2020-11-16T22:20:47.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f231b7fe700 1 -- 172.21.15.185:0/3439712790 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314005840 con 0x7f23240b13d0 2020-11-16T22:20:47.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23240adb00 con 0x7f23240b13d0 2020-11-16T22:20:47.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.839+0000 7f231b7fe700 1 -- 172.21.15.185:0/3439712790 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2314010070 con 0x7f23240b13d0 2020-11-16T22:20:47.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231b7fe700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 0x7f2300057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231b7fe700 1 -- 172.21.15.185:0/3439712790 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f231406d4d0 con 0x7f23240b13d0 2020-11-16T22:20:47.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 0x7f2300057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 0x7f2300057520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f23240ada20 tx=0x7f230c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 Processor -- start 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 -- start start 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25807b0 0x279e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x279eda0 0x279d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x262a4b0 con 0x25807b0 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26de930 con 0x279eda0 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x259e6f0 con 0x2512bb0 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55036/0 (socket says 172.21.15.185:55036) 2020-11-16T22:20:47.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f23237fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x279eda0 0x279d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 -- 172.21.15.185:0/1901197078 learned_addr learned my addr 172.21.15.185:0/1901197078 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:47.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2322ffd700 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25807b0 0x279e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:47.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 -- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x279eda0 msgr2=0x279d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x279eda0 0x279d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 -- 172.21.15.185:0/1901197078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25807b0 msgr2=0x279e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25807b0 0x279e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 -- 172.21.15.185:0/1901197078 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25995a0 con 0x2512bb0 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f231a7fc700 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f230c015d20 tx=0x7f230c011000).ready entity=mon.2 client_cookie=11b4a9f491331bf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f22fbfff700 1 -- 172.21.15.185:0/1901197078 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c01a070 con 0x2512bb0 2020-11-16T22:20:47.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f22fbfff700 1 -- 172.21.15.185:0/1901197078 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c015e20 con 0x2512bb0 2020-11-16T22:20:47.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f22fbfff700 1 -- 172.21.15.185:0/1901197078 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c012d80 con 0x2512bb0 2020-11-16T22:20:47.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 -- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 msgr2=0x279ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:47.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.843+0000 7f2337e11740 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f230c015d20 tx=0x7f230c011000).stop 2020-11-16T22:20:47.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/1901197078 shutdown_connections 2020-11-16T22:20:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x279eda0 0x279d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25807b0 0x279e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- 172.21.15.185:0/1901197078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x279ce90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/1901197078 >> 172.21.15.185:0/1901197078 conn(0x274ee40 msgr2=0x279bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/1901197078 shutdown_connections 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/1901197078 wait complete. 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- start start 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 Processor -- start 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x27a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 0x27cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26cb6e0 con 0x279eda0 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26cb820 con 0x25807b0 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26cca90 con 0x2512bb0 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 10 client.0 ll_register_callbacks cb 0x26c1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:48.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33366/0 (socket says 172.21.15.185:33366) 2020-11-16T22:20:48.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 -- 172.21.15.185:0/359134591 learned_addr learned my addr 172.21.15.185:0/359134591 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:48.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2322ffd700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x27a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f23237fe700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 0x27cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 -- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 msgr2=0x27a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x27a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 -- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 msgr2=0x27cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 0x27cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 -- 172.21.15.185:0/359134591 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26bc280 con 0x25807b0 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f23237fe700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 0x27cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:48.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f231a7fc700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f230c015cf0 tx=0x7f230c015000).ready entity=mon.1 client_cookie=252176ccdb77a5ff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:48.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:48.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c027070 con 0x25807b0 2020-11-16T22:20:48.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f230c007020 con 0x25807b0 2020-11-16T22:20:48.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x277f6c0 con 0x25807b0 2020-11-16T22:20:48.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f230c0064f0 con 0x25807b0 2020-11-16T22:20:48.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 10 client.7048 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:48.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x24f1fd0 con 0x25807b0 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x277f6c0 con 0x25807b0 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f230c006ca0 con 0x25807b0 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 10 client.7048 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 10 client.7048 resolve_mds: resolved ID 'b' to GID 6185 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 20 client.7048 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x27d4480 0x27e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 4 client.7048 mds_command: new command op to 6185 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 --> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x25bcc50 con 0x27d4480 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.847+0000 7f2322ffd700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x27d4480 0x27e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f22f97fa700 10 client.7048 ms_handle_connect on v2:172.21.15.96:6834/3171513205 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2322ffd700 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x27d4480 0x27e5c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=8ed11ec24b32c9c3 server_cookie=bf5673498bff02d9 in_seq=0 out_seq=0 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f22f97fa700 1 -- 172.21.15.185:0/359134591 <== mds.2 v2:172.21.15.96:6834/3171513205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x25bcc50 con 0x27d4480 2020-11-16T22:20:48.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f22f97fa700 10 client.7048 handle_command_reply: tid=0 2020-11-16T22:20:48.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 client.7048 shutdown 2020-11-16T22:20:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 msgr2=0x27cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f230c015cf0 tx=0x7f230c015000).stop 2020-11-16T22:20:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 shutdown_connections 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x27d4480 0x27e5c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x279eda0 0x27cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2512bb0 0x27a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 --2- 172.21.15.185:0/359134591 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25807b0 0x27cccb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f22f97fa700 0 client.7048 ms_handle_reset on v2:172.21.15.96:6834/3171513205 2020-11-16T22:20:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 >> 172.21.15.185:0/359134591 conn(0x274ee40 msgr2=0x279ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 shutdown_connections 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 1 -- 172.21.15.185:0/359134591 wait complete. 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.851+0000 7f2337e11740 20 client.7048 trim_cache size 0 max 16384 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 Processor -- start 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 -- start start 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x279bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26a3e40 con 0x274ee40 2020-11-16T22:20:48.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x27a1680 con 0x2512bb0 2020-11-16T22:20:48.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26fb680 con 0x25807b0 2020-11-16T22:20:48.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55050/0 (socket says 172.21.15.185:55050) 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 -- 172.21.15.185:0/3887599426 learned_addr learned my addr 172.21.15.185:0/3887599426 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27d4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33376/0 (socket says 172.21.15.185:33376) 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2322ffd700 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x279bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 -- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 msgr2=0x27d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:48.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27d4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 -- 172.21.15.185:0/3887599426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 msgr2=0x279bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:48.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x279bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:48.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 -- 172.21.15.185:0/3887599426 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0008160 con 0x25807b0 2020-11-16T22:20:48.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2322ffd700 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x279bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:49.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f23237fe700 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f2324063040 tx=0x7f231c016000).ready entity=mon.2 client_cookie=814f726e0887adfb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:49.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2318ff9700 1 -- 172.21.15.185:0/3887599426 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c011610 con 0x25807b0 2020-11-16T22:20:49.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2318ff9700 1 -- 172.21.15.185:0/3887599426 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c023040 con 0x25807b0 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.871+0000 7f2318ff9700 1 -- 172.21.15.185:0/3887599426 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c011be0 con 0x25807b0 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 msgr2=0x279cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f2324063040 tx=0x7f231c016000).stop 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- 172.21.15.185:0/3887599426 shutdown_connections 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x279cd00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x279bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- 172.21.15.185:0/3887599426 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- 172.21.15.185:0/3887599426 >> 172.21.15.185:0/3887599426 conn(0x7f22f0002320 msgr2=0x279dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- 172.21.15.185:0/3887599426 shutdown_connections 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- 172.21.15.185:0/3887599426 wait complete. 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- start start 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 Processor -- start 2020-11-16T22:20:49.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:49.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:49.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x2833d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:49.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2834e40 con 0x274ee40 2020-11-16T22:20:49.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x27cec60 con 0x2512bb0 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x28359b0 con 0x25807b0 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2337e11740 10 client.0 ll_register_callbacks cb 0x2798a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55056/0 (socket says 172.21.15.185:55056) 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 -- 172.21.15.185:0/2928407982 learned_addr learned my addr 172.21.15.185:0/2928407982 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:49.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f2322ffd700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:49.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f23237fe700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x2833d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:49.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 -- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 msgr2=0x27e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:49.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 -- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 msgr2=0x2833d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:49.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x2833d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 -- 172.21.15.185:0/2928407982 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2530720 con 0x25807b0 2020-11-16T22:20:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f231a7fc700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f230c008fd0 tx=0x7f230c027000).ready entity=mon.2 client_cookie=729a175dfa81576 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:49.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.875+0000 7f22f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0154c0 con 0x25807b0 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f230c024070 con 0x25807b0 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f230c015e20 con 0x25807b0 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f230c007000 con 0x25807b0 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 10 client.6377 fetch_fsmap learned FSMap version 74 2020-11-16T22:20:49.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x253be00 con 0x25807b0 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c015e20 con 0x25807b0 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 74) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f230c039070 con 0x25807b0 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 10 client.6377 fetch_fsmap finished waiting for FSMap version 74 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 10 client.6377 resolve_mds: resolved ID 'b' to GID 6185 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 20 client.6377 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x279eda0 0x2834200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:49.500 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 4 client.6377 mds_command: new command op to 6185 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:20:49.501 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 --> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x25c3f50 con 0x279eda0 2020-11-16T22:20:49.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2322ffd700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x279eda0 0x2834200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:49.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f22f97fa700 10 client.6377 ms_handle_connect on v2:172.21.15.96:6834/3171513205 2020-11-16T22:20:49.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.879+0000 7f2322ffd700 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x279eda0 0x2834200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=277e0b52ea3b6acd server_cookie=43158b90eaa694ad in_seq=0 out_seq=0 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f22f97fa700 1 -- 172.21.15.185:0/2928407982 <== mds.2 v2:172.21.15.96:6834/3171513205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x25c3f50 con 0x279eda0 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f22f97fa700 10 client.6377 handle_command_reply: tid=0 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 client.6377 shutdown 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 msgr2=0x27e9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f230c008fd0 tx=0x7f230c027000).stop 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 shutdown_connections 2020-11-16T22:20:49.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:6834/3171513205,v1:172.21.15.96:6835/3171513205] conn(0x279eda0 0x2834200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2512bb0 0x27e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f22f97fa700 0 client.6377 ms_handle_reset on v2:172.21.15.96:6834/3171513205 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x274ee40 0x2833d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 --2- 172.21.15.185:0/2928407982 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25807b0 0x27e9730 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 >> 172.21.15.185:0/2928407982 conn(0x7f22f0002320 msgr2=0x27d4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:49.704 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 shutdown_connections 2020-11-16T22:20:49.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 1 -- 172.21.15.185:0/2928407982 wait complete. 2020-11-16T22:20:49.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2337e11740 20 client.6377 trim_cache size 0 max 16384 2020-11-16T22:20:49.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 msgr2=0x7f2300057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 0x7f2300057520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f23240ada20 tx=0x7f230c009040).stop 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 msgr2=0x7f2324146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f231400c790 tx=0x7f2314007a60).stop 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 shutdown_connections 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2300055060 0x7f2300057520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f23240b13d0 0x7f2324146c70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23240b3920 0x7f2324144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 --2- 172.21.15.185:0/3439712790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f23240b3080 0x7f23241449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 >> 172.21.15.185:0/3439712790 conn(0x7f2324060130 msgr2=0x7f232414ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 shutdown_connections 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:46.899+0000 7f2323fff700 1 -- 172.21.15.185:0/3439712790 wait complete. 2020-11-16T22:20:49.912 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:20:49.913 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:20:49.913 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:20:49.913 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:20:49.913 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:20:50.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 Processor -- start 2020-11-16T22:20:50.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- start start 2020-11-16T22:20:50.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2bd00b7510 0x7f2bd00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bd0060130 0x7f2bd0060ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd013ba80 0x7f2bd013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd0060fa0 con 0x7f2bd0060130 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd0141ec0 con 0x7f2bd00b7510 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01420f0 con 0x7f2bd013ba80 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2bd00b7510 0x7f2bd00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd58b5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd013ba80 0x7f2bd013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2bd00b7510 0x7f2bd00b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33386/0 (socket says 172.21.15.185:33386) 2020-11-16T22:20:50.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 -- 172.21.15.185:0/3680054007 learned_addr learned my addr 172.21.15.185:0/3680054007 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:50.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd48b3700 1 --2- 172.21.15.185:0/3680054007 >> v2:172.21.15.96:3300/0 conn(0x7f2bd0060130 0x7f2bd0060ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 -- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd013ba80 msgr2=0x7f2bd013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:50.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 --2- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd013ba80 0x7f2bd013bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 -- 172.21.15.185:0/3680054007 >> v2:172.21.15.96:3300/0 conn(0x7f2bd0060130 msgr2=0x7f2bd0060ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:50.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 --2- 172.21.15.185:0/3680054007 >> v2:172.21.15.96:3300/0 conn(0x7f2bd0060130 0x7f2bd0060ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 -- 172.21.15.185:0/3680054007 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0142390 con 0x7f2bd00b7510 2020-11-16T22:20:50.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd50b4700 1 --2- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00b78f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2bd0062250 tx=0x7f2bc0006eb0).ready entity=mon.1 client_cookie=d24f1e26c750147d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:50.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bcffff700 1 -- 172.21.15.185:0/3680054007 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0009810 con 0x7f2bd00b7510 2020-11-16T22:20:50.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bcffff700 1 -- 172.21.15.185:0/3680054007 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0009970 con 0x7f2bd00b7510 2020-11-16T22:20:50.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bcffff700 1 -- 172.21.15.185:0/3680054007 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0009cf0 con 0x7f2bd00b7510 2020-11-16T22:20:50.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 msgr2=0x7f2bd00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:50.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00b78f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2bd0062250 tx=0x7f2bc0006eb0).stop 2020-11-16T22:20:50.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- 172.21.15.185:0/3680054007 shutdown_connections 2020-11-16T22:20:50.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd013ba80 0x7f2bd013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/3680054007 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00b78f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/3680054007 >> v2:172.21.15.96:3300/0 conn(0x7f2bd0060130 0x7f2bd0060ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- 172.21.15.185:0/3680054007 >> 172.21.15.185:0/3680054007 conn(0x7f2bd005ba80 msgr2=0x7f2bd005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- 172.21.15.185:0/3680054007 shutdown_connections 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.254+0000 7f2bd60b6700 1 -- 172.21.15.185:0/3680054007 wait complete. 2020-11-16T22:20:50.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 Processor -- start 2020-11-16T22:20:50.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- start start 2020-11-16T22:20:50.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 0x7f2bd0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bd013ba80 0x7f2bd00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00adbb0 con 0x7f2bd013ba80 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b0da0 con 0x7f2bd00b7510 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00ad700 con 0x7f2bd0060130 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd50b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 0x7f2bd0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33392/0 (socket says 172.21.15.185:33392) 2020-11-16T22:20:50.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd50b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 0x7f2bd0153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55070/0 (socket says 172.21.15.185:55070) 2020-11-16T22:20:50.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 -- 172.21.15.185:0/1260775193 learned_addr learned my addr 172.21.15.185:0/1260775193 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:50.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd58b5700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bd013ba80 0x7f2bd00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:50.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 -- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 msgr2=0x7f2bd0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:50.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 0x7f2bd0153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 -- 172.21.15.185:0/1260775193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bd013ba80 msgr2=0x7f2bd00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bd013ba80 0x7f2bd00aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:50.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 -- 172.21.15.185:0/1260775193 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00b11d0 con 0x7f2bd00b7510 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd48b3700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f2bc400b920 tx=0x7f2bc4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400f5e0 con 0x7f2bd00b7510 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc400f740 con 0x7f2bd00b7510 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400fae0 con 0x7f2bd00b7510 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd00adde0 con 0x7f2bd00b7510 2020-11-16T22:20:50.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.258+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd0141300 con 0x7f2bd00b7510 2020-11-16T22:20:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd013c260 con 0x7f2bd00b7510 2020-11-16T22:20:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2bc4012420 con 0x7f2bd00b7510 2020-11-16T22:20:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bcd7fa700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 0x7f2ba8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2bc406d9a0 con 0x7f2bd00b7510 2020-11-16T22:20:50.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bd50b4700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 0x7f2ba8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:51.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.262+0000 7f2bd50b4700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 0x7f2ba8057520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f2bc000b1b0 tx=0x7f2bc0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:51.066 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.270+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2bd013c260 con 0x7f2bd00b7510 2020-11-16T22:20:51.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.470+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd013c260 con 0x7f2bd00b7510 2020-11-16T22:20:51.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.470+0000 7f2bcd7fa700 1 -- 172.21.15.185:0/1260775193 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f2bd013c260 con 0x7f2bd00b7510 2020-11-16T22:20:51.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 msgr2=0x7f2ba8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:51.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 0x7f2ba8057520 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f2bc000b1b0 tx=0x7f2bc0015000).stop 2020-11-16T22:20:51.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 msgr2=0x7f2bd00ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:51.069 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f2bc400b920 tx=0x7f2bc4008040).stop 2020-11-16T22:20:51.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 shutdown_connections 2020-11-16T22:20:51.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2ba8055060 0x7f2ba8057520 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:51.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bd013ba80 0x7f2bd00aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:51.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bd0060130 0x7f2bd0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:51.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 --2- 172.21.15.185:0/1260775193 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bd00b7510 0x7f2bd00ad210 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:51.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 >> 172.21.15.185:0/1260775193 conn(0x7f2bd005ba80 msgr2=0x7f2bd00b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:51.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 shutdown_connections 2020-11-16T22:20:51.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:50.474+0000 7f2bd60b6700 1 -- 172.21.15.185:0/1260775193 wait complete. 2020-11-16T22:20:51.275 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 74 2020-11-16T22:20:51.275 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:20:51.275 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:19:21.246259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:20:51.276 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 2020-11-16T22:20:51.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 Processor -- start 2020-11-16T22:20:51.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 -- start start 2020-11-16T22:20:51.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5be40b3020 0x7f5be40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:51.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be40acdf0 con 0x7f5be4141150 2020-11-16T22:20:51.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be40ad2c0 con 0x7f5be40abef0 2020-11-16T22:20:51.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be9580700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be41462b0 con 0x7f5be40b3020 2020-11-16T22:20:51.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5be40b3020 0x7f5be40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:51.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be37fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5be40abef0 0x7f5be4145ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55070/0 (socket says 172.21.15.185:55070) 2020-11-16T22:20:51.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.418+0000 7f5be37fe700 1 -- 172.21.15.185:0/1520815115 learned_addr learned my addr 172.21.15.185:0/1520815115 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be3fff700 1 -- 172.21.15.185:0/1520815115 >> v1:172.21.15.185:6789/0 conn(0x7f5be40abef0 legacy=0x7f5be4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be3fff700 1 -- 172.21.15.185:0/1520815115 >> v1:172.21.15.96:6789/0 conn(0x7f5be4141150 legacy=0x7f5be41415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be3fff700 1 -- 172.21.15.185:0/1520815115 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be40ad020 con 0x7f5be40b3020 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be3fff700 1 --2- 172.21.15.185:0/1520815115 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be40b3020 0x7f5be40b17c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5bcc00b9a0 tx=0x7f5bcc006df0).ready entity=mon.1 client_cookie=9d3fa604865cb7a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be2ffd700 1 -- 172.21.15.185:0/1520815115 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0097b0 con 0x7f5be40b3020 2020-11-16T22:20:51.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be2ffd700 1 -- 172.21.15.185:0/1520815115 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc009910 con 0x7f5be40b3020 2020-11-16T22:20:51.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be2ffd700 1 -- 172.21.15.185:0/1520815115 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc009c90 con 0x7f5be40b3020 2020-11-16T22:20:51.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 -- 172.21.15.185:0/1520815115 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be40b3020 msgr2=0x7f5be40b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:51.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 --2- 172.21.15.185:0/1520815115 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be40b3020 0x7f5be40b17c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f5bcc00b9a0 tx=0x7f5bcc006df0).stop 2020-11-16T22:20:51.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 -- 172.21.15.185:0/1520815115 shutdown_connections 2020-11-16T22:20:51.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 --2- 172.21.15.185:0/1520815115 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be40b3020 0x7f5be40b17c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:51.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 -- 172.21.15.185:0/1520815115 >> 172.21.15.185:0/1520815115 conn(0x7f5be4060130 msgr2=0x7f5be4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 -- 172.21.15.185:0/1520815115 shutdown_connections 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.422+0000 7f5be9580700 1 -- 172.21.15.185:0/1520815115 wait complete. 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 Processor -- start 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- start start 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5be40abef0 0x7f5be413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be40b3020 0x7f5be40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:51.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be4141150 0x7f5be40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41538c0 con 0x7f5be40b3020 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be4153b60 con 0x7f5be4141150 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be4153e00 con 0x7f5be40abef0 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5be40abef0 0x7f5be413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be4141150 0x7f5be40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:51.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5be40abef0 0x7f5be413fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55082/0 (socket says 172.21.15.185:55082) 2020-11-16T22:20:51.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 -- 172.21.15.185:0/1362732314 learned_addr learned my addr 172.21.15.185:0/1362732314 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:52.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be40b3020 0x7f5be40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 -- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be4141150 msgr2=0x7f5be40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:52.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5be4141150 0x7f5be40ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 -- 172.21.15.185:0/1362732314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be40b3020 msgr2=0x7f5be40b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be40b3020 0x7f5be40b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 -- 172.21.15.185:0/1362732314 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be41540a0 con 0x7f5be40abef0 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be40b3020 0x7f5be40b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be3fff700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5be40abef0 0x7f5be413fc20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f5be40688f0 tx=0x7f5bcc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:52.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5bdbfff700 1 -- 172.21.15.185:0/1362732314 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc01e460 con 0x7f5be40abef0 2020-11-16T22:20:52.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5bdbfff700 1 -- 172.21.15.185:0/1362732314 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc01e5c0 con 0x7f5be40abef0 2020-11-16T22:20:52.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- 172.21.15.185:0/1362732314 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be413e370 con 0x7f5be40abef0 2020-11-16T22:20:52.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5bdbfff700 1 -- 172.21.15.185:0/1362732314 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc01e940 con 0x7f5be40abef0 2020-11-16T22:20:52.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.426+0000 7f5be9580700 1 -- 172.21.15.185:0/1362732314 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be41534a0 con 0x7f5be40abef0 2020-11-16T22:20:52.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5bdbfff700 1 -- 172.21.15.185:0/1362732314 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5bcc025030 con 0x7f5be40abef0 2020-11-16T22:20:52.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5bdbfff700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5bb4055060 0x7f5bb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5bb4055060 0x7f5bb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5bdbfff700 1 -- 172.21.15.185:0/1362732314 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bcc06f760 con 0x7f5be40abef0 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1362732314 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5bb4055060 0x7f5bb4057520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f5be4142b30 tx=0x7f5bd4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5bf9273740 1 Processor -- start 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.430+0000 7f5bf9273740 1 -- start start 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf667f0 0x11848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xef8bf0 0x1182ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x10104f0 con 0xf667f0 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c4970 con 0x1184de0 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xf84730 con 0xef8bf0 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33416/0 (socket says 172.21.15.185:33416) 2020-11-16T22:20:52.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2155158803 learned_addr learned my addr 172.21.15.185:0/2155158803 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:52.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be37fe700 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xef8bf0 0x1182ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be3fff700 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf667f0 0x11848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xef8bf0 msgr2=0x1182ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xef8bf0 0x1182ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2155158803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf667f0 msgr2=0x11848f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf667f0 0x11848f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2155158803 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xf7f5e0 con 0x1184de0 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc009040).ready entity=mon.1 client_cookie=a1093d41c706ebce server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:52.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bd97fa700 1 -- 172.21.15.185:0/2155158803 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00fc90 con 0x1184de0 2020-11-16T22:20:52.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bd97fa700 1 -- 172.21.15.185:0/2155158803 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc00fdf0 con 0x1184de0 2020-11-16T22:20:52.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bd97fa700 1 -- 172.21.15.185:0/2155158803 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc014680 con 0x1184de0 2020-11-16T22:20:52.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 msgr2=0x11833c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:52.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc009040).stop 2020-11-16T22:20:52.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- 172.21.15.185:0/2155158803 shutdown_connections 2020-11-16T22:20:52.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11833c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xf667f0 0x11848f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2155158803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xef8bf0 0x1182ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- 172.21.15.185:0/2155158803 >> 172.21.15.185:0/2155158803 conn(0x1134e80 msgr2=0x1181e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- 172.21.15.185:0/2155158803 shutdown_connections 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- 172.21.15.185:0/2155158803 wait complete. 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 -- start start 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.434+0000 7f5bf9273740 1 Processor -- start 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1187e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x10b1720 con 0xef8bf0 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x10b1860 con 0x1184de0 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x10b2ad0 con 0xf667f0 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 10 client.0 ll_register_callbacks cb 0x10a7b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:52.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55098/0 (socket says 172.21.15.185:55098) 2020-11-16T22:20:52.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33424/0 (socket says 172.21.15.185:33424) 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be37fe700 1 -- 172.21.15.185:0/2456297457 learned_addr learned my addr 172.21.15.185:0/2456297457 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be3fff700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1187e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 msgr2=0x11b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2456297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 msgr2=0x1187e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1187e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:52.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 -- 172.21.15.185:0/2456297457 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x10a22c0 con 0x1184de0 2020-11-16T22:20:52.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc017000).ready entity=mon.1 client_cookie=da48b25acc4b8fb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:52.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:52.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00fc90 con 0x1184de0 2020-11-16T22:20:52.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bdc0135c0 con 0x1184de0 2020-11-16T22:20:52.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1165700 con 0x1184de0 2020-11-16T22:20:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bdc013990 con 0x1184de0 2020-11-16T22:20:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 10 client.7075 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0xed8010 con 0x1184de0 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1165700 con 0x1184de0 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f5bdc01d030 con 0x1184de0 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 10 client.7075 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 10 client.7075 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 20 client.7075 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x11ba4c0 0x11c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 4 client.7075 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xfa2c90 con 0x11ba4c0 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be3fff700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x11ba4c0 0x11c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5bd27fc700 10 client.7075 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.438+0000 7f5be3fff700 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x11ba4c0 0x11c86b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=8b8d7086b779b511 server_cookie=a6ee8fdd5496654d in_seq=0 out_seq=0 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bd27fc700 1 -- 172.21.15.185:0/2456297457 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xfa2c90 con 0x11ba4c0 2020-11-16T22:20:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bd27fc700 10 client.7075 handle_command_reply: tid=0 2020-11-16T22:20:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 client.7075 shutdown 2020-11-16T22:20:53.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 msgr2=0x11b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:53.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc017000).stop 2020-11-16T22:20:53.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 shutdown_connections 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x11ba4c0 0x11c86b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1184de0 0x11b3180 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bd27fc700 0 client.7075 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1187e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 --2- 172.21.15.185:0/2456297457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 >> 172.21.15.185:0/2456297457 conn(0x1134e80 msgr2=0x1182e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:53.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 shutdown_connections 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 1 -- 172.21.15.185:0/2456297457 wait complete. 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.442+0000 7f5bf9273740 20 client.7075 trim_cache size 0 max 16384 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 Processor -- start 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- start start 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 0x1180fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1180240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.265 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1089e80 con 0xef8bf0 2020-11-16T22:20:53.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x117a000 con 0x1134e80 2020-11-16T22:20:53.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x11b9cd0 con 0xf667f0 2020-11-16T22:20:53.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 0x1180fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 0x1180fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33428/0 (socket says 172.21.15.185:33428) 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55106/0 (socket says 172.21.15.185:55106) 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be3fff700 1 -- 172.21.15.185:0/1819503965 learned_addr learned my addr 172.21.15.185:0/1819503965 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1180240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 -- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 msgr2=0x1180fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:53.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 0x1180fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 -- 172.21.15.185:0/1819503965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 msgr2=0x1180240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:53.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1180240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 -- 172.21.15.185:0/1819503965 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x117a470 con 0xf667f0 2020-11-16T22:20:53.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be37fe700 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1180240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:53.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc020000).ready entity=mon.2 client_cookie=cebcb27fd1e67d8d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:53.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bd9ffb700 1 -- 172.21.15.185:0/1819503965 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc032070 con 0xf667f0 2020-11-16T22:20:53.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bd9ffb700 1 -- 172.21.15.185:0/1819503965 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc013cd0 con 0xf667f0 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bd9ffb700 1 -- 172.21.15.185:0/1819503965 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00fc90 con 0xf667f0 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 msgr2=0x1182d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5bdc001fb0 tx=0x7f5bdc020000).stop 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- 172.21.15.185:0/1819503965 shutdown_connections 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1134e80 0x1180fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x1182d40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 --2- 172.21.15.185:0/1819503965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xef8bf0 0x1180240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- 172.21.15.185:0/1819503965 >> 172.21.15.185:0/1819503965 conn(0x7f5bac002320 msgr2=0x11840a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- 172.21.15.185:0/1819503965 shutdown_connections 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.462+0000 7f5bf9273740 1 -- 172.21.15.185:0/1819503965 wait complete. 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- start start 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 Processor -- start 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1134e80 0x11cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:53.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ee090 con 0x1134e80 2020-11-16T22:20:53.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1224bf0 con 0xef8bf0 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x11ced30 con 0xf667f0 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 10 client.0 ll_register_callbacks cb 0x117eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33434/0 (socket says 172.21.15.185:33434) 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 -- 172.21.15.185:0/805060761 learned_addr learned my addr 172.21.15.185:0/805060761 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:53.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be37fe700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be8d7f700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1134e80 0x11cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:53.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 -- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 msgr2=0x11cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:53.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:53.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 -- 172.21.15.185:0/805060761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1134e80 msgr2=0x11cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:53.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1134e80 0x11cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 -- 172.21.15.185:0/805060761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac008160 con 0xef8bf0 2020-11-16T22:20:54.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be3fff700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f5bcc006e30 tx=0x7f5bcc02a040).ready entity=mon.1 client_cookie=e22d89ab2ea00691 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:54.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc036110 con 0xef8bf0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bcc045410 con 0xef8bf0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bdc00fdf0 con 0xef8bf0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc048580 con 0xef8bf0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 10 client.7081 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0xf16760 con 0xef8bf0 2020-11-16T22:20:54.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc00fdf0 con 0xef8bf0 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f5bcc0486e0 con 0xef8bf0 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 10 client.7081 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 10 client.7081 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 20 client.7081 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1184de0 0x1227250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 4 client.7081 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",}] 2020-11-16T22:20:54.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " }) v1 -- 0xfa9f90 con 0x1184de0 2020-11-16T22:20:54.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.466+0000 7f5be37fe700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1184de0 0x1227250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:54.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bd27fc700 10 client.7081 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:54.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5be37fe700 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1184de0 0x1227250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=608ac74a6c1a24a9 server_cookie=80dc981ba19c5a2c in_seq=0 out_seq=0 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bd27fc700 1 -- 172.21.15.185:0/805060761 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0xfa9f90 con 0x1184de0 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bd27fc700 10 client.7081 handle_command_reply: tid=0 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 client.7081 shutdown 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 msgr2=0x1224700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f5bcc006e30 tx=0x7f5bcc02a040).stop 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 shutdown_connections 2020-11-16T22:20:54.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1184de0 0x1227250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xef8bf0 0x1224700 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bd27fc700 0 client.7081 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:54.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1134e80 0x11cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 --2- 172.21.15.185:0/805060761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xf667f0 0x11cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 >> 172.21.15.185:0/805060761 conn(0x7f5bac002320 msgr2=0x1180240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:54.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 shutdown_connections 2020-11-16T22:20:54.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 1 -- 172.21.15.185:0/805060761 wait complete. 2020-11-16T22:20:54.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:51.470+0000 7f5bf9273740 20 client.7081 trim_cache size 0 max 16384 2020-11-16T22:20:54.487 INFO:teuthology.orchestra.run.smithi185.stderr:Error EXDEV: Not rank 0 2020-11-16T22:20:54.487 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:20:54.488 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2020-11-16T22:20:54.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.630+0000 7f1e16067700 1 Processor -- start 2020-11-16T22:20:54.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.630+0000 7f1e16067700 1 -- start start 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100acdf0 con 0x7f1e100abef0 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100ad2c0 con 0x7f1e100b3020 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101462b0 con 0x7f1e100b17c0 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:54.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15065700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f1e100b3020 0x7f1e10145ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55116/0 (socket says 172.21.15.185:55116) 2020-11-16T22:20:54.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55120/0 (socket says 172.21.15.185:55120) 2020-11-16T22:20:54.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15065700 1 -- 172.21.15.185:0/1368235447 learned_addr learned my addr 172.21.15.185:0/1368235447 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:54.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738717016 0 0) 0x7f1e100ad2c0 con 0x7f1e100b3020 2020-11-16T22:20:54.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003240 con 0x7f1e100b3020 2020-11-16T22:20:54.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799856411 0 0) 0x7f1e100acdf0 con 0x7f1e100abef0 2020-11-16T22:20:54.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e100ad2c0 con 0x7f1e100abef0 2020-11-16T22:20:54.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 -- 172.21.15.185:0/1368235447 >> v1:172.21.15.185:6789/0 conn(0x7f1e100b3020 legacy=0x7f1e10145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 -- 172.21.15.185:0/1368235447 >> v1:172.21.15.96:6789/0 conn(0x7f1e100abef0 legacy=0x7f1e10141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 -- 172.21.15.185:0/1368235447 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100ad020 con 0x7f1e100b17c0 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e15866700 1 --2- 172.21.15.185:0/1368235447 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f1e0800b9d0 tx=0x7f1e08006df0).ready entity=mon.2 client_cookie=6bea33df0a3fd266 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08009680 con 0x7f1e100b17c0 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080097e0 con 0x7f1e100b17c0 2020-11-16T22:20:54.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e0ffff700 1 -- 172.21.15.185:0/1368235447 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08009b60 con 0x7f1e100b17c0 2020-11-16T22:20:54.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- 172.21.15.185:0/1368235447 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 msgr2=0x7f1e10141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:54.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 --2- 172.21.15.185:0/1368235447 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f1e0800b9d0 tx=0x7f1e08006df0).stop 2020-11-16T22:20:54.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- 172.21.15.185:0/1368235447 shutdown_connections 2020-11-16T22:20:54.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 --2- 172.21.15.185:0/1368235447 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100b17c0 0x7f1e10141860 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- 172.21.15.185:0/1368235447 >> 172.21.15.185:0/1368235447 conn(0x7f1e10060130 msgr2=0x7f1e10060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- 172.21.15.185:0/1368235447 shutdown_connections 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.634+0000 7f1e16067700 1 -- 172.21.15.185:0/1368235447 wait complete. 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 Processor -- start 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- start start 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100abef0 0x7f1e101552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e100b17c0 0x7f1e10151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:54.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1e100b3020 0x7f1e10148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10155870 con 0x7f1e100b17c0 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10141dd0 con 0x7f1e100b3020 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10154910 con 0x7f1e100abef0 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1e100b3020 0x7f1e10148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15065700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100abef0 0x7f1e101552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:54.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1e100b3020 0x7f1e10148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33452/0 (socket says 172.21.15.185:33452) 2020-11-16T22:20:54.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 -- 172.21.15.185:0/3226131070 learned_addr learned my addr 172.21.15.185:0/3226131070 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:54.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e14864700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e100b17c0 0x7f1e10151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:54.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 -- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100abef0 msgr2=0x7f1e101552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:54.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100abef0 0x7f1e101552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:54.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 -- 172.21.15.185:0/3226131070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e100b17c0 msgr2=0x7f1e10151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:55.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e100b17c0 0x7f1e10151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 -- 172.21.15.185:0/3226131070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10154bb0 con 0x7f1e100b3020 2020-11-16T22:20:55.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15065700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1e100abef0 0x7f1e101552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e15866700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1e100b3020 0x7f1e10148800 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f1e0800b150 tx=0x7f1e08015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e0d7fa700 1 -- 172.21.15.185:0/3226131070 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011d40 con 0x7f1e100b3020 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e0d7fa700 1 -- 172.21.15.185:0/3226131070 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0801e460 con 0x7f1e100b3020 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- 172.21.15.185:0/3226131070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e10154e50 con 0x7f1e100b3020 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e0d7fa700 1 -- 172.21.15.185:0/3226131070 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011ea0 con 0x7f1e100b3020 2020-11-16T22:20:55.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.638+0000 7f1e16067700 1 -- 172.21.15.185:0/3226131070 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e10151830 con 0x7f1e100b3020 2020-11-16T22:20:55.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.642+0000 7f1e0d7fa700 1 -- 172.21.15.185:0/3226131070 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1e08025030 con 0x7f1e100b3020 2020-11-16T22:20:55.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.642+0000 7f1e0d7fa700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1dec055060 0x7f1dec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.642+0000 7f1e0d7fa700 1 -- 172.21.15.185:0/3226131070 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e0806f3b0 con 0x7f1e100b3020 2020-11-16T22:20:55.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.642+0000 7f1e15065700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1dec055060 0x7f1dec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.642+0000 7f1e15065700 1 --2- 172.21.15.185:0/3226131070 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1dec055060 0x7f1dec057520 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f1e00005920 tx=0x7f1e00009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:55.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 Processor -- start 2020-11-16T22:20:55.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 -- start start 2020-11-16T22:20:55.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a22e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x18b04a0 con 0x18067a0 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1964920 con 0x1a24d90 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x18246e0 con 0x1798ba0 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a22e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33460/0 (socket says 172.21.15.185:33460) 2020-11-16T22:20:55.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 -- 172.21.15.185:0/3708284638 learned_addr learned my addr 172.21.15.185:0/3708284638 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:55.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15065700 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 -- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 msgr2=0x1a22e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:55.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a22e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 -- 172.21.15.185:0/3708284638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 msgr2=0x1a248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:55.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 -- 172.21.15.185:0/3708284638 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x181f590 con 0x1a24d90 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.646+0000 7f1e15866700 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e08048f40 tx=0x7f1e08011000).ready entity=mon.1 client_cookie=72cdbd70776da29 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e067fc700 1 -- 172.21.15.185:0/3708284638 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08048870 con 0x1a24d90 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e067fc700 1 -- 172.21.15.185:0/3708284638 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080489d0 con 0x1a24d90 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e067fc700 1 -- 172.21.15.185:0/3708284638 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08048b40 con 0x1a24d90 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 msgr2=0x1a23370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e08048f40 tx=0x7f1e08011000).stop 2020-11-16T22:20:55.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- 172.21.15.185:0/3708284638 shutdown_connections 2020-11-16T22:20:55.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a23370 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/3708284638 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a22e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- 172.21.15.185:0/3708284638 >> 172.21.15.185:0/3708284638 conn(0x19d4e30 msgr2=0x1a21dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:55.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- 172.21.15.185:0/3708284638 shutdown_connections 2020-11-16T22:20:55.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- 172.21.15.185:0/3708284638 wait complete. 2020-11-16T22:20:55.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- start start 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 Processor -- start 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1a52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19516d0 con 0x1798ba0 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1951810 con 0x1a24d90 2020-11-16T22:20:55.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1952a80 con 0x18067a0 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e25d5a740 10 client.0 ll_register_callbacks cb 0x1947ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55140/0 (socket says 172.21.15.185:55140) 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e14864700 1 -- 172.21.15.185:0/1853510532 learned_addr learned my addr 172.21.15.185:0/1853510532 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e15866700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a6c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.650+0000 7f1e15065700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1a52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:55.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 -- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 msgr2=0x1a52ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:55.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 -- 172.21.15.185:0/1853510532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 msgr2=0x1a52b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:55.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1a52b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 -- 172.21.15.185:0/1853510532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1942270 con 0x1a24d90 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e14864700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15866700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a6c080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f1e08029000 tx=0x7f1e0806f030).ready entity=mon.1 client_cookie=8e075d06d2c0a87 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0801e9f0 con 0x1a24d90 2020-11-16T22:20:55.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e0801eb50 con 0x1a24d90 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a056b0 con 0x1a24d90 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e080479e0 con 0x1a24d90 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 10 client.7093 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x1777fc0 con 0x1a24d90 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a056b0 con 0x1a24d90 2020-11-16T22:20:55.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f1e08039070 con 0x1a24d90 2020-11-16T22:20:56.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 10 client.7093 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 10 client.7093 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 20 client.7093 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a21410 0x1a58940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 4 client.7093 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1842c40 con 0x1a21410 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15065700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a21410 0x1a58940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 10 client.7093 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e15065700 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a21410 0x1a58940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=115017f97d6bee08 server_cookie=3f59a6bdb03e3134 in_seq=0 out_seq=0 2020-11-16T22:20:56.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/1853510532 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1842c40 con 0x1a21410 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 10 client.7093 handle_command_reply: tid=0 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 client.7093 shutdown 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 msgr2=0x1a6c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a6c080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f1e08029000 tx=0x7f1e0806f030).stop 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 shutdown_connections 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a21410 0x1a58940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a24d90 0x1a6c080 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1a52b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/1853510532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1a52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1ddf7fe700 0 client.7093 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 >> 172.21.15.185:0/1853510532 conn(0x19d4e30 msgr2=0x1a22e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.654+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 shutdown_connections 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.658+0000 7f1e25d5a740 1 -- 172.21.15.185:0/1853510532 wait complete. 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.658+0000 7f1e25d5a740 20 client.7093 trim_cache size 0 max 16384 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 Processor -- start 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 -- start start 2020-11-16T22:20:56.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1a21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a52a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1929e30 con 0x18067a0 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a5ac60 con 0x19d4e30 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.674+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a1fdd0 con 0x1798ba0 2020-11-16T22:20:56.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1a21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55148/0 (socket says 172.21.15.185:55148) 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e14864700 1 -- 172.21.15.185:0/884954189 learned_addr learned my addr 172.21.15.185:0/884954189 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15866700 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a52a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 -- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 msgr2=0x1a24230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 -- 172.21.15.185:0/884954189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 msgr2=0x1a52a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a52a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 -- 172.21.15.185:0/884954189 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4008160 con 0x19d4e30 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e14864700 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e15065700 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1a21c70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f1e0000d970 tx=0x7f1e00017040).ready entity=mon.1 client_cookie=66a411e42096896f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e06ffd700 1 -- 172.21.15.185:0/884954189 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e000214d0 con 0x19d4e30 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e06ffd700 1 -- 172.21.15.185:0/884954189 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0001b040 con 0x19d4e30 2020-11-16T22:20:56.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e06ffd700 1 -- 172.21.15.185:0/884954189 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00021840 con 0x19d4e30 2020-11-16T22:20:56.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 msgr2=0x1a21c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1a21c70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f1e0000d970 tx=0x7f1e00017040).stop 2020-11-16T22:20:56.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- 172.21.15.185:0/884954189 shutdown_connections 2020-11-16T22:20:56.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1a21c70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18067a0 0x1a52a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/884954189 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1798ba0 0x1a24230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- 172.21.15.185:0/884954189 >> 172.21.15.185:0/884954189 conn(0x7f1de4002320 msgr2=0x1a57780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- 172.21.15.185:0/884954189 shutdown_connections 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- 172.21.15.185:0/884954189 wait complete. 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 -- start start 2020-11-16T22:20:56.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.678+0000 7f1e25d5a740 1 Processor -- start 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1abe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1ac2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a52ff0 con 0x1798ba0 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6f490 con 0x19d4e30 2020-11-16T22:20:56.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a57f20 con 0x18067a0 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 10 client.0 ll_register_callbacks cb 0x1a227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55154/0 (socket says 172.21.15.185:55154) 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 -- 172.21.15.185:0/2257253034 learned_addr learned my addr 172.21.15.185:0/2257253034 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e15866700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1ac2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e15065700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1abe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 -- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 msgr2=0x1ac2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1ac2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 -- 172.21.15.185:0/2257253034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 msgr2=0x1abe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1abe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 -- 172.21.15.185:0/2257253034 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x17b6710 con 0x18067a0 2020-11-16T22:20:56.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e14864700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1df8016d00 tx=0x7f1df8007a00).ready entity=mon.2 client_cookie=48c5c0733aa5cb7c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df801b070 con 0x18067a0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e080489d0 con 0x18067a0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1df8015c90 con 0x18067a0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df8015e50 con 0x18067a0 2020-11-16T22:20:56.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 10 client.6416 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x17c1df0 con 0x18067a0 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080489d0 con 0x18067a0 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f1df8020a40 con 0x18067a0 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 10 client.6416 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 10 client.6416 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 20 client.6416 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a24d90 0x1ac2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 4 client.6416 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,a,b,o,r,t,",}] 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " s c r u b a b o r t " }) v1 -- 0x1849f40 con 0x1a24d90 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e15065700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a24d90 0x1ac2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 10 client.6416 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e15065700 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a24d90 0x1ac2900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=934942c110587169 server_cookie=716dd0ab7b87d0ab in_seq=0 out_seq=0 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 1 -- 172.21.15.185:0/2257253034 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x1849f40 con 0x1a24d90 2020-11-16T22:20:56.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 10 client.6416 handle_command_reply: tid=0 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 client.6416 shutdown 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 msgr2=0x1abe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1df8016d00 tx=0x7f1df8007a00).stop 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 shutdown_connections 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x1a24d90 0x1ac2900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19d4e30 0x1ac2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1798ba0 0x1abe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 --2- 172.21.15.185:0/2257253034 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18067a0 0x1abe680 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 >> 172.21.15.185:0/2257253034 conn(0x7f1de4002320 msgr2=0x1a574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1ddf7fe700 0 client.6416 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 shutdown_connections 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.682+0000 7f1e25d5a740 1 -- 172.21.15.185:0/2257253034 wait complete. 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:54.686+0000 7f1e25d5a740 20 client.6416 trim_cache size 0 max 16384 2020-11-16T22:20:56.499 INFO:teuthology.orchestra.run.smithi185.stderr:Error EXDEV: Not rank 0 2020-11-16T22:20:56.500 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:20:56.500 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2020-11-16T22:20:56.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 Processor -- start 2020-11-16T22:20:56.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 -- start start 2020-11-16T22:20:56.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd5580b3920 0x7fd558148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55814c660 con 0x7fd5580b3080 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5580ac120 con 0x7fd5580b3920 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55c84f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580ac3c0 con 0x7fd5580b13d0 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55160/0 (socket says 172.21.15.185:55160) 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 -- 172.21.15.185:0/1197736959 learned_addr learned my addr 172.21.15.185:0/1197736959 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55759e700 1 --2- 172.21.15.185:0/1197736959 >> v2:172.21.15.185:3300/0 conn(0x7fd5580b3920 0x7fd558148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 -- 172.21.15.185:0/1197736959 >> v2:172.21.15.185:3300/0 conn(0x7fd5580b3920 msgr2=0x7fd558148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 --2- 172.21.15.185:0/1197736959 >> v2:172.21.15.185:3300/0 conn(0x7fd5580b3920 0x7fd558148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 -- 172.21.15.185:0/1197736959 >> v1:172.21.15.185:6789/0 conn(0x7fd5580b3080 legacy=0x7fd5580ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 -- 172.21.15.185:0/1197736959 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580b20e0 con 0x7fd5580b13d0 2020-11-16T22:20:56.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.646+0000 7fd55659c700 1 --2- 172.21.15.185:0/1197736959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fd540009d50 tx=0x7fd540009290).ready entity=mon.2 client_cookie=d5e2771c366d6f9f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd555d9b700 1 -- 172.21.15.185:0/1197736959 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000bbc0 con 0x7fd5580b13d0 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd555d9b700 1 -- 172.21.15.185:0/1197736959 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000bd20 con 0x7fd5580b13d0 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd555d9b700 1 -- 172.21.15.185:0/1197736959 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540011610 con 0x7fd5580b13d0 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- 172.21.15.185:0/1197736959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 msgr2=0x7fd5580b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 --2- 172.21.15.185:0/1197736959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fd540009d50 tx=0x7fd540009290).stop 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- 172.21.15.185:0/1197736959 shutdown_connections 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 --2- 172.21.15.185:0/1197736959 >> v2:172.21.15.185:3300/0 conn(0x7fd5580b3920 0x7fd558148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 --2- 172.21.15.185:0/1197736959 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b13d0 0x7fd5580b1bf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- 172.21.15.185:0/1197736959 >> 172.21.15.185:0/1197736959 conn(0x7fd558060130 msgr2=0x7fd558060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- 172.21.15.185:0/1197736959 shutdown_connections 2020-11-16T22:20:56.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- 172.21.15.185:0/1197736959 wait complete. 2020-11-16T22:20:56.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 Processor -- start 2020-11-16T22:20:56.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.650+0000 7fd55c84f700 1 -- start start 2020-11-16T22:20:56.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5580b13d0 0x7fd5581104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5580b3080 0x7fd5581591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b3920 0x7fd558140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55810fb20 con 0x7fd5580b3080 2020-11-16T22:20:56.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558112b40 con 0x7fd5580b13d0 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55815be50 con 0x7fd5580b3920 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5580b13d0 0x7fd5581104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5580b13d0 0x7fd5581104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33490/0 (socket says 172.21.15.185:33490) 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 -- 172.21.15.185:0/2741182078 learned_addr learned my addr 172.21.15.185:0/2741182078 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55759e700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b3920 0x7fd558140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55659c700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5580b3080 0x7fd5581591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 -- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b3920 msgr2=0x7fd558140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd5580b3920 0x7fd558140900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 -- 172.21.15.185:0/2741182078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5580b3080 msgr2=0x7fd5581591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5580b3080 0x7fd5581591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 -- 172.21.15.185:0/2741182078 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558159a90 con 0x7fd5580b13d0 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55659c700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5580b3080 0x7fd5581591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd5580b13d0 0x7fd5581104f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd54800c6e0 tx=0x7fd548007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd54effd700 1 -- 172.21.15.185:0/2741182078 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548007020 con 0x7fd5580b13d0 2020-11-16T22:20:56.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd54effd700 1 -- 172.21.15.185:0/2741182078 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548007180 con 0x7fd5580b13d0 2020-11-16T22:20:56.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 -- 172.21.15.185:0/2741182078 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55815bb70 con 0x7fd5580b13d0 2020-11-16T22:20:56.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd54effd700 1 -- 172.21.15.185:0/2741182078 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800e560 con 0x7fd5580b13d0 2020-11-16T22:20:56.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd55c84f700 1 -- 172.21.15.185:0/2741182078 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55810fd60 con 0x7fd5580b13d0 2020-11-16T22:20:56.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.654+0000 7fd54effd700 1 -- 172.21.15.185:0/2741182078 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd548010070 con 0x7fd5580b13d0 2020-11-16T22:20:56.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd54effd700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd528055060 0x7fd528057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd55659c700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd528055060 0x7fd528057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd54effd700 1 -- 172.21.15.185:0/2741182078 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd54806d130 con 0x7fd5580b13d0 2020-11-16T22:20:56.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd55659c700 1 --2- 172.21.15.185:0/2741182078 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd528055060 0x7fd528057520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd558067a20 tx=0x7fd540016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd56bae1740 1 Processor -- start 2020-11-16T22:20:56.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.658+0000 7fd56bae1740 1 -- start start 2020-11-16T22:20:56.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x232a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x232ad90 0x2329370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21b64a0 con 0x210c7a0 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x226a920 con 0x232ad90 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x212a6e0 con 0x209eba0 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55174/0 (socket says 172.21.15.185:55174) 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 -- 172.21.15.185:0/2739206910 learned_addr learned my addr 172.21.15.185:0/2739206910 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55759e700 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x232ad90 0x2329370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x232a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 -- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x232ad90 msgr2=0x2329370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x232ad90 0x2329370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 -- 172.21.15.185:0/2739206910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 msgr2=0x232a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x232a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 -- 172.21.15.185:0/2739206910 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2125590 con 0x209eba0 2020-11-16T22:20:56.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd55659c700 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fd540011000 tx=0x7fd54001b000).ready entity=mon.2 client_cookie=de6fc5ba7e810c10 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd547fff700 1 -- 172.21.15.185:0/2739206910 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540025420 con 0x209eba0 2020-11-16T22:20:56.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd547fff700 1 -- 172.21.15.185:0/2739206910 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd540025580 con 0x209eba0 2020-11-16T22:20:56.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd547fff700 1 -- 172.21.15.185:0/2739206910 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540025900 con 0x209eba0 2020-11-16T22:20:56.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 msgr2=0x2328e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fd540011000 tx=0x7fd54001b000).stop 2020-11-16T22:20:56.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- 172.21.15.185:0/2739206910 shutdown_connections 2020-11-16T22:20:56.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x232ad90 0x2329370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x232a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2739206910 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x209eba0 0x2328e80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- 172.21.15.185:0/2739206910 >> 172.21.15.185:0/2739206910 conn(0x22dae30 msgr2=0x2327dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- 172.21.15.185:0/2739206910 shutdown_connections 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- 172.21.15.185:0/2739206910 wait complete. 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 -- start start 2020-11-16T22:20:56.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.662+0000 7fd56bae1740 1 Processor -- start 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x2358bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 0x23590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x22576d0 con 0x210c7a0 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2257810 con 0x209eba0 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2258a80 con 0x232ad90 2020-11-16T22:20:56.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 10 client.0 ll_register_callbacks cb 0x224dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd55759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 0x23590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33504/0 (socket says 172.21.15.185:33504) 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd55759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 0x23590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55184/0 (socket says 172.21.15.185:55184) 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 -- 172.21.15.185:0/2648616490 learned_addr learned my addr 172.21.15.185:0/2648616490 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd55659c700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x2358bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 -- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 msgr2=0x23590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 0x23590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 -- 172.21.15.185:0/2648616490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 msgr2=0x2358bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x2358bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 -- 172.21.15.185:0/2648616490 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2248270 con 0x209eba0 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd556d9d700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fd5480452b0 tx=0x7fd548012000).ready entity=mon.1 client_cookie=e9becdb0b46c886b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:56.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548001ea0 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd548043070 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x230b6b0 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd548046160 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 10 client.7114 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x207dfc0 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x230b6b0 con 0x209eba0 2020-11-16T22:20:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fd54806b900 con 0x209eba0 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 10 client.7114 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 10 client.7114 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 20 client.7114 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2360ad0 0x2371bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 4 client.7114 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2148c40 con 0x2360ad0 2020-11-16T22:20:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd55659c700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2360ad0 0x2371bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 10 client.7114 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd55659c700 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2360ad0 0x2371bc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=3ae8ed4bb8c33bb2 server_cookie=3061ec2c18297da5 in_seq=0 out_seq=0 2020-11-16T22:20:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 1 -- 172.21.15.185:0/2648616490 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2148c40 con 0x2360ad0 2020-11-16T22:20:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd5457fa700 10 client.7114 handle_command_reply: tid=0 2020-11-16T22:20:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 client.7114 shutdown 2020-11-16T22:20:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.666+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 msgr2=0x232dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fd5480452b0 tx=0x7fd548012000).stop 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 shutdown_connections 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2360ad0 0x2371bc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x209eba0 0x232dd30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x232ad90 0x23590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 --2- 172.21.15.185:0/2648616490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x210c7a0 0x2358bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd5457fa700 0 client.7114 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 >> 172.21.15.185:0/2648616490 conn(0x22dae30 msgr2=0x2328e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 shutdown_connections 2020-11-16T22:20:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 1 -- 172.21.15.185:0/2648616490 wait complete. 2020-11-16T22:20:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.670+0000 7fd56bae1740 20 client.7114 trim_cache size 0 max 16384 2020-11-16T22:20:56.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.686+0000 7fd56bae1740 1 Processor -- start 2020-11-16T22:20:56.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.686+0000 7fd56bae1740 1 -- start start 2020-11-16T22:20:56.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x23270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x235a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x222fe30 con 0x209eba0 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2324dd0 con 0x22dae30 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x23248f0 con 0x210c7a0 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd55759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x235a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd55659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x23270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33514/0 (socket says 172.21.15.185:33514) 2020-11-16T22:20:56.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 -- 172.21.15.185:0/1314787558 learned_addr learned my addr 172.21.15.185:0/1314787558 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd55759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x235a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55192/0 (socket says 172.21.15.185:55192) 2020-11-16T22:20:56.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 -- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 msgr2=0x235a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x235a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 -- 172.21.15.185:0/1314787558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 msgr2=0x23270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x23270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 -- 172.21.15.185:0/1314787558 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200081d0 con 0x22dae30 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd54806cfd0 tx=0x7fd54801c000).ready entity=mon.1 client_cookie=a707f461da5d4134 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd54cc96700 1 -- 172.21.15.185:0/1314787558 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548001ea0 con 0x22dae30 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd54cc96700 1 -- 172.21.15.185:0/1314787558 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548043070 con 0x22dae30 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd54cc96700 1 -- 172.21.15.185:0/1314787558 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548046160 con 0x22dae30 2020-11-16T22:20:56.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 msgr2=0x231fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd54806cfd0 tx=0x7fd54801c000).stop 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- 172.21.15.185:0/1314787558 shutdown_connections 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x231fa90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x235a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1314787558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x23270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- 172.21.15.185:0/1314787558 >> 172.21.15.185:0/1314787558 conn(0x7fd520002320 msgr2=0x232c730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- 172.21.15.185:0/1314787558 shutdown_connections 2020-11-16T22:20:56.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- 172.21.15.185:0/1314787558 wait complete. 2020-11-16T22:20:56.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 -- start start 2020-11-16T22:20:56.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.690+0000 7fd56bae1740 1 Processor -- start 2020-11-16T22:20:56.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x2371e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x2327140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x2327630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2327b20 con 0x209eba0 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2368780 con 0x22dae30 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2368c50 con 0x210c7a0 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x2327140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 10 client.0 ll_register_callbacks cb 0x2324a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x2327140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55196/0 (socket says 172.21.15.185:55196) 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55659c700 1 -- 172.21.15.185:0/1770724214 learned_addr learned my addr 172.21.15.185:0/1770724214 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x2327630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x2371e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 -- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 msgr2=0x2327140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x2327140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 -- 172.21.15.185:0/1770724214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 msgr2=0x2371e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x2371e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 -- 172.21.15.185:0/1770724214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x20bc710 con 0x22dae30 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd55759e700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x2327630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fd550006ec0 tx=0x7fd550017040).ready entity=mon.1 client_cookie=e23d3090b58db578 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55001a070 con 0x22dae30 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd550014810 con 0x22dae30 2020-11-16T22:20:56.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd540025580 con 0x22dae30 2020-11-16T22:20:56.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd550014be0 con 0x22dae30 2020-11-16T22:20:56.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 10 client.7120 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:56.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x20c7df0 con 0x22dae30 2020-11-16T22:20:56.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd540025580 con 0x22dae30 2020-11-16T22:20:56.693 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fd550014d40 con 0x22dae30 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 10 client.7120 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 10 client.7120 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 20 client.7120 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x232ad90 0x2358500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 4 client.7120 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.694+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0x214ff40 con 0x232ad90 2020-11-16T22:20:56.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x232ad90 0x2358500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd5457fa700 10 client.7120 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd556d9d700 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x232ad90 0x2358500 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=7d490d9c99170f24 server_cookie=13f1ea2b0ec2078a in_seq=0 out_seq=0 2020-11-16T22:20:56.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd5457fa700 1 -- 172.21.15.185:0/1770724214 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x214ff40 con 0x232ad90 2020-11-16T22:20:56.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd5457fa700 10 client.7120 handle_command_reply: tid=0 2020-11-16T22:20:56.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 client.7120 shutdown 2020-11-16T22:20:56.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 msgr2=0x2327630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x2327630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fd550006ec0 tx=0x7fd550017040).stop 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 shutdown_connections 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x232ad90 0x2358500 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x22dae30 0x2327630 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd5457fa700 0 client.7120 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x209eba0 0x2371e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 --2- 172.21.15.185:0/1770724214 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x210c7a0 0x2327140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 >> 172.21.15.185:0/1770724214 conn(0x7fd520002320 msgr2=0x235a300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 shutdown_connections 2020-11-16T22:20:56.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 1 -- 172.21.15.185:0/1770724214 wait complete. 2020-11-16T22:20:56.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.698+0000 7fd56bae1740 20 client.7120 trim_cache size 0 max 16384 2020-11-16T22:20:56.698 INFO:teuthology.orchestra.run.smithi185.stderr:Error EXDEV: Not rank 0 2020-11-16T22:20:56.698 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:20:56.699 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2020-11-16T22:20:56.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.838+0000 7f8bd759e700 1 Processor -- start 2020-11-16T22:20:56.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.838+0000 7f8bd759e700 1 -- start start 2020-11-16T22:20:56.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 0x7f8bd8065b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 0x7f8bd80b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd8067130 con 0x7f8bd80ac3e0 2020-11-16T22:20:56.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd8065300 con 0x7f8bd80657a0 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8068640 con 0x7f8bd8068180 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd659c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 0x7f8bd8065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55202/0 (socket says 172.21.15.185:55202) 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/1817302515 learned_addr learned my addr 172.21.15.185:0/1817302515 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 0x7f8bd80b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/1817302515 >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 msgr2=0x7f8bd8065b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 0x7f8bd8065b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/1817302515 >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 msgr2=0x7f8bd80b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 0x7f8bd80b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/1817302515 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8067360 con 0x7f8bd8068180 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 0x7f8bd80b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd659c700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 0x7f8bd8065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:56.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1817302515 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8bd8064930 tx=0x7f8bc0006eb0).ready entity=mon.2 client_cookie=619c6ddcc371439b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd559a700 1 -- 172.21.15.185:0/1817302515 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0009810 con 0x7f8bd8068180 2020-11-16T22:20:56.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd559a700 1 -- 172.21.15.185:0/1817302515 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0009970 con 0x7f8bd8068180 2020-11-16T22:20:56.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd559a700 1 -- 172.21.15.185:0/1817302515 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0009cf0 con 0x7f8bd8068180 2020-11-16T22:20:56.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- 172.21.15.185:0/1817302515 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 msgr2=0x7f8bd80abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- 172.21.15.185:0/1817302515 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8bd8064930 tx=0x7f8bc0006eb0).stop 2020-11-16T22:20:56.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- 172.21.15.185:0/1817302515 shutdown_connections 2020-11-16T22:20:56.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.96:3300/0 conn(0x7f8bd80ac3e0 0x7f8bd80b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- 172.21.15.185:0/1817302515 >> v2:172.21.15.185:3300/0 conn(0x7f8bd80657a0 0x7f8bd8065b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- 172.21.15.185:0/1817302515 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd80abef0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- 172.21.15.185:0/1817302515 >> 172.21.15.185:0/1817302515 conn(0x7f8bd8060130 msgr2=0x7f8bd8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- 172.21.15.185:0/1817302515 shutdown_connections 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- 172.21.15.185:0/1817302515 wait complete. 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 Processor -- start 2020-11-16T22:20:56.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- start start 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80657a0 0x7f8bd812c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd81297e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8bd80ac3e0 0x7f8bd8129cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80b57e0 con 0x7f8bd80657a0 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80ad550 con 0x7f8bd80ac3e0 2020-11-16T22:20:56.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80ad7f0 con 0x7f8bd8068180 2020-11-16T22:20:56.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd81297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd81297e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55210/0 (socket says 172.21.15.185:55210) 2020-11-16T22:20:56.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/2277966177 learned_addr learned my addr 172.21.15.185:0/2277966177 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8bd80ac3e0 0x7f8bd8129cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.842+0000 7f8bd659c700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80657a0 0x7f8bd812c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8bd80ac3e0 msgr2=0x7f8bd8129cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8bd80ac3e0 0x7f8bd8129cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/2277966177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80657a0 msgr2=0x7f8bd812c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80657a0 0x7f8bd812c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/2277966177 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80ada90 con 0x7f8bd8068180 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8bd8068180 0x7f8bd81297e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f8bc0009e10 tx=0x7f8bc0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 -- 172.21.15.185:0/2277966177 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011c80 con 0x7f8bd8068180 2020-11-16T22:20:56.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 -- 172.21.15.185:0/2277966177 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0011de0 con 0x7f8bd8068180 2020-11-16T22:20:56.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd759e700 1 -- 172.21.15.185:0/2277966177 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd812ca70 con 0x7f8bd8068180 2020-11-16T22:20:56.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 -- 172.21.15.185:0/2277966177 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc001e690 con 0x7f8bd8068180 2020-11-16T22:20:56.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd759e700 1 -- 172.21.15.185:0/2277966177 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd812a1c0 con 0x7f8bd8068180 2020-11-16T22:20:56.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 -- 172.21.15.185:0/2277966177 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8bc0025030 con 0x7f8bd8068180 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8bac055060 0x7f8bac057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bce7fc700 1 -- 172.21.15.185:0/2277966177 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8bc006f150 con 0x7f8bd8068180 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.846+0000 7f8bd659c700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8bac055060 0x7f8bac057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd659c700 1 --2- 172.21.15.185:0/2277966177 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8bac055060 0x7f8bac057520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f8bc8006fd0 tx=0x7f8bc8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 Processor -- start 2020-11-16T22:20:56.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 -- start start 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25577b0 0x27758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 0x2774380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26014b0 con 0x2775da0 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b5930 con 0x25577b0 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x25756f0 con 0x24e9bb0 2020-11-16T22:20:56.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55218/0 (socket says 172.21.15.185:55218) 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/460238096 learned_addr learned my addr 172.21.15.185:0/460238096 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd659c700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25577b0 0x27758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 0x2774380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25577b0 msgr2=0x27758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25577b0 0x27758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/460238096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 msgr2=0x2774380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 0x2774380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/460238096 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25705a0 con 0x24e9bb0 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.850+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 0x2774380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f8bc0048f20 tx=0x7f8bc0011000).ready entity=mon.2 client_cookie=7b183f2efd591718 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bb37fe700 1 -- 172.21.15.185:0/460238096 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00474a0 con 0x24e9bb0 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bb37fe700 1 -- 172.21.15.185:0/460238096 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0045810 con 0x24e9bb0 2020-11-16T22:20:56.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bb37fe700 1 -- 172.21.15.185:0/460238096 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0048800 con 0x24e9bb0 2020-11-16T22:20:56.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 msgr2=0x2773e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f8bc0048f20 tx=0x7f8bc0011000).stop 2020-11-16T22:20:56.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- 172.21.15.185:0/460238096 shutdown_connections 2020-11-16T22:20:56.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x25577b0 0x27758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2775da0 0x2774380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/460238096 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x2773e90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- 172.21.15.185:0/460238096 >> 172.21.15.185:0/460238096 conn(0x2725e40 msgr2=0x2772dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- 172.21.15.185:0/460238096 shutdown_connections 2020-11-16T22:20:56.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- 172.21.15.185:0/460238096 wait complete. 2020-11-16T22:20:56.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- start start 2020-11-16T22:20:56.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 Processor -- start 2020-11-16T22:20:56.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 0x27b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 0x27b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26a26e0 con 0x25577b0 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26a2820 con 0x2775da0 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26a3a90 con 0x24e9bb0 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8beb3ab740 10 client.0 ll_register_callbacks cb 0x2698ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 0x27b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55222/0 (socket says 172.21.15.185:55222) 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 0x27b9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33548/0 (socket says 172.21.15.185:33548) 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd659c700 1 -- 172.21.15.185:0/1071627437 learned_addr learned my addr 172.21.15.185:0/1071627437 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 0x27b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd659c700 1 -- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 msgr2=0x27b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.854+0000 7f8bd659c700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 0x27b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd659c700 1 -- 172.21.15.185:0/1071627437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 msgr2=0x27b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd659c700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 0x27b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd659c700 1 -- 172.21.15.185:0/1071627437 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2693280 con 0x24e9bb0 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 0x27b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd659c700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f8bc8014d40 tx=0x7f8bc8013000).ready entity=mon.2 client_cookie=ec97a6c2fe2fc230 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800f930 con 0x24e9bb0 2020-11-16T22:20:56.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27566c0 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8bc8012a30 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc8012e00 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 10 client.6446 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x24c8fd0 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27566c0 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f8bc8021a90 con 0x24e9bb0 2020-11-16T22:20:56.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 10 client.6446 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 10 client.6446 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 20 client.6446 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2776fc0 0x27a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 4 client.6446 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2593c50 con 0x2776fc0 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2776fc0 0x27a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 10 client.6446 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2776fc0 0x27a3820 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=bb6825397729fc8b server_cookie=758f47484ce6754a in_seq=0 out_seq=0 2020-11-16T22:20:56.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/1071627437 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2593c50 con 0x2776fc0 2020-11-16T22:20:56.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 10 client.6446 handle_command_reply: tid=0 2020-11-16T22:20:56.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 client.6446 shutdown 2020-11-16T22:20:56.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 msgr2=0x27ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f8bc8014d40 tx=0x7f8bc8013000).stop 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 shutdown_connections 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2776fc0 0x27a3820 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2775da0 0x27b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x24e9bb0 0x27ac2e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8bb0ff9700 0 client.6446 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/1071627437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x25577b0 0x27b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 >> 172.21.15.185:0/1071627437 conn(0x2725e40 msgr2=0x2773e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 shutdown_connections 2020-11-16T22:20:56.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 1 -- 172.21.15.185:0/1071627437 wait complete. 2020-11-16T22:20:56.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.858+0000 7f8beb3ab740 20 client.6446 trim_cache size 0 max 16384 2020-11-16T22:20:56.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.878+0000 7f8beb3ab740 1 Processor -- start 2020-11-16T22:20:56.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- start start 2020-11-16T22:20:56.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2725e40 0x27bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24e9bb0 0x2773d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x267ae40 con 0x2725e40 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2757200 con 0x24e9bb0 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2773ae0 con 0x25577b0 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55230/0 (socket says 172.21.15.185:55230) 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24e9bb0 0x2773d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 -- 172.21.15.185:0/491662444 learned_addr learned my addr 172.21.15.185:0/491662444 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd659c700 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2725e40 0x27bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 -- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24e9bb0 msgr2=0x2773d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24e9bb0 0x2773d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 -- 172.21.15.185:0/491662444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2725e40 msgr2=0x27bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2725e40 0x27bd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 -- 172.21.15.185:0/491662444 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00081d0 con 0x25577b0 2020-11-16T22:20:56.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f8bd000cd50 tx=0x7f8bd0008040).ready entity=mon.2 client_cookie=348c2dc3ebf13fdb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bb3fff700 1 -- 172.21.15.185:0/491662444 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0017d90 con 0x25577b0 2020-11-16T22:20:56.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bb3fff700 1 -- 172.21.15.185:0/491662444 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0018450 con 0x25577b0 2020-11-16T22:20:56.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8bb3fff700 1 -- 172.21.15.185:0/491662444 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00185e0 con 0x25577b0 2020-11-16T22:20:56.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 msgr2=0x2778890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f8bd000cd50 tx=0x7f8bd0008040).stop 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- 172.21.15.185:0/491662444 shutdown_connections 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x2778890 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2725e40 0x27bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/491662444 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x24e9bb0 0x2773d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- 172.21.15.185:0/491662444 >> 172.21.15.185:0/491662444 conn(0x7f8ba0002320 msgr2=0x2771200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- 172.21.15.185:0/491662444 shutdown_connections 2020-11-16T22:20:56.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- 172.21.15.185:0/491662444 wait complete. 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 -- start start 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.882+0000 7f8beb3ab740 1 Processor -- start 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24e9bb0 0x27b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x27aea00 con 0x24e9bb0 2020-11-16T22:20:56.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b9bc0 con 0x2725e40 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x27b9df0 con 0x25577b0 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 10 client.0 ll_register_callbacks cb 0x276fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33564/0 (socket says 172.21.15.185:33564) 2020-11-16T22:20:56.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55238/0 (socket says 172.21.15.185:55238) 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd6d9d700 1 -- 172.21.15.185:0/221665990 learned_addr learned my addr 172.21.15.185:0/221665990 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd659c700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24e9bb0 0x27b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 msgr2=0x280f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/221665990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24e9bb0 msgr2=0x27b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24e9bb0 0x27b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 -- 172.21.15.185:0/221665990 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2507720 con 0x25577b0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd6d9d700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd5d9b700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f8bc0048ef0 tx=0x7f8bc003c280).ready entity=mon.2 client_cookie=9c9a9411de766f62 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00069f0 con 0x25577b0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8bc0045980 con 0x25577b0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bc0045810 con 0x25577b0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc004b030 con 0x25577b0 2020-11-16T22:20:56.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 10 client.6452 fetch_fsmap learned FSMap version 75 2020-11-16T22:20:56.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x2512e00 con 0x25577b0 2020-11-16T22:20:56.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0045810 con 0x25577b0 2020-11-16T22:20:56.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 75) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f8bc0048060 con 0x25577b0 2020-11-16T22:20:56.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 10 client.6452 fetch_fsmap finished waiting for FSMap version 75 2020-11-16T22:20:56.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 10 client.6452 resolve_mds: resolved ID 'a' to GID 6188 2020-11-16T22:20:56.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 20 client.6452 populate_metadata read hostname 'smithi185' 2020-11-16T22:20:56.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2775da0 0x280fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:56.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 4 client.6452 mds_command: new command op to 6188 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,r,e,s,u,m,e,",}] 2020-11-16T22:20:56.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 --> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] -- command(tid 0: { " p r e f i x " : " s c r u b r e s u m e " }) v1 -- 0x259af50 con 0x2775da0 2020-11-16T22:20:56.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd659c700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2775da0 0x280fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:56.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bd659c700 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2775da0 0x280fa60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=6d196d398c3fb746 server_cookie=aeb598546d740653 in_seq=0 out_seq=0 2020-11-16T22:20:56.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 10 client.6452 ms_handle_connect on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 1 -- 172.21.15.185:0/221665990 <== mds.1 v2:172.21.15.185:6834/3676791890 1 ==== command_reply(tid 0: -18 Not rank 0) v1 ==== 18+0+0 (crc 0 0 0) 0x259af50 con 0x2775da0 2020-11-16T22:20:56.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8bb0ff9700 10 client.6452 handle_command_reply: tid=0 2020-11-16T22:20:56.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.886+0000 7f8beb3ab740 1 client.6452 shutdown 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 msgr2=0x27b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f8bc0048ef0 tx=0x7f8bc003c280).stop 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 shutdown_connections 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:6834/3676791890,v1:172.21.15.185:6835/3676791890] conn(0x2775da0 0x280fa60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2725e40 0x280f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8bb0ff9700 0 client.6452 ms_handle_reset on v2:172.21.15.185:6834/3676791890 2020-11-16T22:20:56.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x24e9bb0 0x27b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 --2- 172.21.15.185:0/221665990 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25577b0 0x27b2a10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 >> 172.21.15.185:0/221665990 conn(0x7f8ba0002320 msgr2=0x2778b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 shutdown_connections 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 1 -- 172.21.15.185:0/221665990 wait complete. 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:56.890+0000 7f8beb3ab740 20 client.6452 trim_cache size 0 max 16384 2020-11-16T22:20:56.891 INFO:teuthology.orchestra.run.smithi185.stderr:Error EXDEV: Not rank 0 2020-11-16T22:20:56.892 DEBUG:teuthology.orchestra.run:got remote process result: 18 2020-11-16T22:20:56.892 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:20:56.893 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:56.930 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:56.947 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:56.965 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:20:56.965 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:20:56.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:20:56.966 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:20:57.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:20:57.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:20:57.058 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:20:57.074 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:20:57.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.218+0000 7efd53fff700 1 Processor -- start 2020-11-16T22:20:57.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.218+0000 7efd53fff700 1 -- start start 2020-11-16T22:20:57.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd5405bc50 0x7efd54144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efd54144d60 0x7efd5413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd540ad350 con 0x7efd5405bc50 2020-11-16T22:20:57.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd540b2690 con 0x7efd54144d60 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540b2930 con 0x7efd540b38a0 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55244/0 (socket says 172.21.15.185:55244) 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 -- 172.21.15.185:0/2894808200 learned_addr learned my addr 172.21.15.185:0/2894808200 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd537fe700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.185:3300/0 conn(0x7efd54144d60 0x7efd5413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd4a7fc700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.96:3300/0 conn(0x7efd5405bc50 0x7efd54144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 -- 172.21.15.185:0/2894808200 >> v2:172.21.15.185:3300/0 conn(0x7efd54144d60 msgr2=0x7efd5413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:57.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.185:3300/0 conn(0x7efd54144d60 0x7efd5413fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 -- 172.21.15.185:0/2894808200 >> v2:172.21.15.96:3300/0 conn(0x7efd5405bc50 msgr2=0x7efd54144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:57.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.96:3300/0 conn(0x7efd5405bc50 0x7efd54144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 -- 172.21.15.185:0/2894808200 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd541443d0 con 0x7efd540b38a0 2020-11-16T22:20:57.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd52ffd700 1 --2- 172.21.15.185:0/2894808200 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7efd540ac710 tx=0x7efd44006eb0).ready entity=mon.2 client_cookie=c706719bd402c616 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd527fc700 1 -- 172.21.15.185:0/2894808200 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44009810 con 0x7efd540b38a0 2020-11-16T22:20:57.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd527fc700 1 -- 172.21.15.185:0/2894808200 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd44009970 con 0x7efd540b38a0 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd527fc700 1 -- 172.21.15.185:0/2894808200 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44009cf0 con 0x7efd540b38a0 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- 172.21.15.185:0/2894808200 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 msgr2=0x7efd540b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- 172.21.15.185:0/2894808200 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7efd540ac710 tx=0x7efd44006eb0).stop 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- 172.21.15.185:0/2894808200 shutdown_connections 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.185:3300/0 conn(0x7efd54144d60 0x7efd5413fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- 172.21.15.185:0/2894808200 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd540b38a0 0x7efd540b2060 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 --2- 172.21.15.185:0/2894808200 >> v2:172.21.15.96:3300/0 conn(0x7efd5405bc50 0x7efd54144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.222+0000 7efd53fff700 1 -- 172.21.15.185:0/2894808200 >> 172.21.15.185:0/2894808200 conn(0x7efd54063df0 msgr2=0x7efd540641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:57.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- 172.21.15.185:0/2894808200 shutdown_connections 2020-11-16T22:20:57.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- 172.21.15.185:0/2894808200 wait complete. 2020-11-16T22:20:57.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 Processor -- start 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- start start 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 0x7efd54132410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efd540b38a0 0x7efd540ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540afa70 con 0x7efd5405bc50 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540afce0 con 0x7efd540b38a0 2020-11-16T22:20:57.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd53fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540ad9c0 con 0x7efd54144d60 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd4a7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efd540b38a0 0x7efd540ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55254/0 (socket says 172.21.15.185:55254) 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 -- 172.21.15.185:0/782944480 learned_addr learned my addr 172.21.15.185:0/782944480 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd52ffd700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 0x7efd54132410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 -- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efd540b38a0 msgr2=0x7efd540ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efd540b38a0 0x7efd540ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 -- 172.21.15.185:0/782944480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 msgr2=0x7efd54132410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 0x7efd54132410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd537fe700 1 -- 172.21.15.185:0/782944480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540adc60 con 0x7efd54144d60 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.226+0000 7efd52ffd700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 0x7efd54132410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd537fe700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7efd4c00ef20 tx=0x7efd4c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c009690 con 0x7efd54144d60 2020-11-16T22:20:57.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c00cbc0 con 0x7efd54144d60 2020-11-16T22:20:57.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0123f0 con 0x7efd54144d60 2020-11-16T22:20:57.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd54132900 con 0x7efd54144d60 2020-11-16T22:20:57.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd54134cd0 con 0x7efd54144d60 2020-11-16T22:20:57.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efd4c022070 con 0x7efd54144d60 2020-11-16T22:20:57.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5405d200 con 0x7efd54144d60 2020-11-16T22:20:57.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 0x7efd30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efd4c06e4d0 con 0x7efd54144d60 2020-11-16T22:20:57.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd52ffd700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 0x7efd30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.230+0000 7efd52ffd700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 0x7efd30057520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7efd4400beb0 tx=0x7efd44015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.234+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efd4c044b00 con 0x7efd54144d60 2020-11-16T22:20:57.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.390+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efd54061e20 con 0x7efd54144d60 2020-11-16T22:20:57.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.398+0000 7efd4b7fe700 1 -- 172.21.15.185:0/782944480 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7efd4c0160a0 con 0x7efd54144d60 2020-11-16T22:20:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 msgr2=0x7efd30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 0x7efd30057520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7efd4400beb0 tx=0x7efd44015000).stop 2020-11-16T22:20:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 msgr2=0x7efd541347e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7efd4c00ef20 tx=0x7efd4c0091d0).stop 2020-11-16T22:20:57.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 shutdown_connections 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efd30055060 0x7efd30057520 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efd54144d60 0x7efd541347e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd5405bc50 0x7efd54132410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 --2- 172.21.15.185:0/782944480 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efd540b38a0 0x7efd540ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 >> 172.21.15.185:0/782944480 conn(0x7efd54063df0 msgr2=0x7efd540ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 shutdown_connections 2020-11-16T22:20:57.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.402+0000 7efd53fff700 1 -- 172.21.15.185:0/782944480 wait complete. 2020-11-16T22:20:57.409 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2020-11-16T22:20:57.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.534+0000 7f1813a60700 1 Processor -- start 2020-11-16T22:20:57.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.534+0000 7f1813a60700 1 -- start start 2020-11-16T22:20:57.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1813a60700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f180c142830 0x7f180c145d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1813a60700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c09b330 con 0x7f180c142c10 2020-11-16T22:20:57.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1813a60700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c099f10 con 0x7f180c0b3200 2020-11-16T22:20:57.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1813a60700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c098d30 con 0x7f180c142830 2020-11-16T22:20:57.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f180c0b3200 0x7f180c099800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55252/0 (socket says 172.21.15.185:55252) 2020-11-16T22:20:57.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 -- 172.21.15.185:0/4073315457 learned_addr learned my addr 172.21.15.185:0/4073315457 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:57.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f18117fc700 1 --2- 172.21.15.185:0/4073315457 >> v2:172.21.15.96:3300/0 conn(0x7f180c142830 0x7f180c145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351801465 0 0) 0x7f180c099f10 con 0x7f180c0b3200 2020-11-16T22:20:57.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f0003240 con 0x7f180c0b3200 2020-11-16T22:20:57.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218735070 0 0) 0x7f180c09b330 con 0x7f180c142c10 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f180c099f10 con 0x7f180c142c10 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 298064178 0 0) 0x7f17f0003240 con 0x7f180c0b3200 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 >> v2:172.21.15.96:3300/0 conn(0x7f180c142830 msgr2=0x7f180c145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 --2- 172.21.15.185:0/4073315457 >> v2:172.21.15.96:3300/0 conn(0x7f180c142830 0x7f180c145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 >> v1:172.21.15.96:6789/0 conn(0x7f180c142c10 legacy=0x7f180c141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c09b090 con 0x7f180c0b3200 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f17fc0014e0 con 0x7f180c0b3200 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 >> v1:172.21.15.185:6789/0 conn(0x7f180c0b3200 legacy=0x7f180c099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f00038c0 0x7f17f0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 0x7f17f0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f0004e50 0x7f17f000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c098d30 con 0x7f17f00038c0 2020-11-16T22:20:57.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000bc20 con 0x7f17f0004e50 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000bd60 con 0x7f17f00044e0 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1811ffd700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f0004e50 0x7f17f000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 0x7f17f0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f18117fc700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f00038c0 0x7f17f0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 -- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f0004e50 msgr2=0x7f17f000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f0004e50 0x7f17f000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 -- 172.21.15.185:0/4073315457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f00038c0 msgr2=0x7f17f0003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17f00038c0 0x7f17f0003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 -- 172.21.15.185:0/4073315457 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f0003240 con 0x7f17f00044e0 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1811ffd700 1 -- 172.21.15.185:0/4073315457 reap_dead start 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1810ffb700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 0x7f17f0004960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f180c061010 tx=0x7f180c1454d0).ready entity=mon.2 client_cookie=ab881feb3db13811 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00ce20 con 0x7f17f00044e0 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0070d0 con 0x7f17f00044e0 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.538+0000 7f1803fff700 1 -- 172.21.15.185:0/4073315457 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007490 con 0x7f17f00044e0 2020-11-16T22:20:57.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 msgr2=0x7f17f0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 0x7f17f0004960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f180c061010 tx=0x7f180c1454d0).stop 2020-11-16T22:20:57.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- 172.21.15.185:0/4073315457 shutdown_connections 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 --2- 172.21.15.185:0/4073315457 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f17f00044e0 0x7f17f0004960 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- 172.21.15.185:0/4073315457 >> 172.21.15.185:0/4073315457 conn(0x7f180c060130 msgr2=0x7f180c060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- 172.21.15.185:0/4073315457 shutdown_connections 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- 172.21.15.185:0/4073315457 wait complete. 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 Processor -- start 2020-11-16T22:20:57.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.542+0000 7f1813a60700 1 -- start start 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f00044e0 0x7f180c0ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f180c0b6690 0x7f180c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0ae4f0 con 0x7f180c0b6690 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0add20 con 0x7f17f00044e0 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0adfb0 con 0x7f180c0b9180 2020-11-16T22:20:57.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f18117fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f00044e0 0x7f180c0ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55274/0 (socket says 172.21.15.185:55274) 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 -- 172.21.15.185:0/2718870486 learned_addr learned my addr 172.21.15.185:0/2718870486 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1810ffb700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f180c0b6690 0x7f180c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 -- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f00044e0 msgr2=0x7f180c0ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f00044e0 0x7f180c0ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 -- 172.21.15.185:0/2718870486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f180c0b6690 msgr2=0x7f180c1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f180c0b6690 0x7f180c1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 -- 172.21.15.185:0/2718870486 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0ae250 con 0x7f180c0b9180 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1811ffd700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f180c0b5be0 tx=0x7f1804007e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804006a80 con 0x7f180c0b9180 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1804006be0 con 0x7f180c0b9180 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c1513b0 con 0x7f180c0b9180 2020-11-16T22:20:57.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804007070 con 0x7f180c0b9180 2020-11-16T22:20:57.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c151d40 con 0x7f180c0b9180 2020-11-16T22:20:57.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c151590 con 0x7f180c0b9180 2020-11-16T22:20:57.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.546+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1804007420 con 0x7f180c0b9180 2020-11-16T22:20:57.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.550+0000 7f18017fa700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 0x7f17e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:57.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.550+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f180406d920 con 0x7f180c0b9180 2020-11-16T22:20:57.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.550+0000 7f18117fc700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 0x7f17e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:57.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.550+0000 7f18117fc700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 0x7f17e4057520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f1808006d00 tx=0x7f1808006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:57.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.554+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f180c151590 con 0x7f180c0b9180 2020-11-16T22:20:57.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:57.762+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) v1 -- 0x7f180c05d4e0 con 0x7f180c0b9180 2020-11-16T22:20:58.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.062+0000 7f18017fa700 1 -- 172.21.15.185:0/2718870486 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v321) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1804043b80 con 0x7f180c0b9180 2020-11-16T22:20:58.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.062+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 msgr2=0x7f17e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.062+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 0x7f17e4057520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f1808006d00 tx=0x7f1808006df0).stop 2020-11-16T22:20:58.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.062+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 msgr2=0x7f180c151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.062+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f180c0b5be0 tx=0x7f1804007e80).stop 2020-11-16T22:20:58.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 shutdown_connections 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f17e4055060 0x7f17e4057520 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f17f00044e0 0x7f180c0ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f180c0b9180 0x7f180c151850 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 --2- 172.21.15.185:0/2718870486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f180c0b6690 0x7f180c1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 >> 172.21.15.185:0/2718870486 conn(0x7f180c060130 msgr2=0x7f180c141c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 shutdown_connections 2020-11-16T22:20:58.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.066+0000 7f1813a60700 1 -- 172.21.15.185:0/2718870486 wait complete. 2020-11-16T22:20:58.073 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:20:58.073 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:20:58.074 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2020-11-16T22:20:58.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 Processor -- start 2020-11-16T22:20:58.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 -- start start 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3bf40b76c0 0x7f3bf40b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf4147f90 con 0x7f3bf40b72e0 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf4148460 con 0x7f3bf40b64b0 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.206+0000 7f3bf929d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40af020 con 0x7f3bf40b76c0 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf8a9c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3bf40b76c0 0x7f3bf40b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:58.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf37fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3bf40b64b0 0x7f3bf413dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55272/0 (socket says 172.21.15.185:55272) 2020-11-16T22:20:58.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf37fe700 1 -- 172.21.15.185:0/563962527 learned_addr learned my addr 172.21.15.185:0/563962527 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf8a9c700 1 -- 172.21.15.185:0/563962527 >> v1:172.21.15.185:6789/0 conn(0x7f3bf40b64b0 legacy=0x7f3bf413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf8a9c700 1 -- 172.21.15.185:0/563962527 >> v1:172.21.15.96:6789/0 conn(0x7f3bf40b72e0 legacy=0x7f3bf4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf8a9c700 1 -- 172.21.15.185:0/563962527 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40b4810 con 0x7f3bf40b76c0 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf8a9c700 1 --2- 172.21.15.185:0/563962527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b76c0 0x7f3bf40b6aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f3bec009c90 tx=0x7f3bec0091d0).ready entity=mon.1 client_cookie=cee460c72b8d333b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/563962527 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00bbc0 con 0x7f3bf40b76c0 2020-11-16T22:20:58.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/563962527 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec00bd20 con 0x7f3bf40b76c0 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/563962527 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec011610 con 0x7f3bf40b76c0 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 -- 172.21.15.185:0/563962527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b76c0 msgr2=0x7f3bf40b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 --2- 172.21.15.185:0/563962527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b76c0 0x7f3bf40b6aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f3bec009c90 tx=0x7f3bec0091d0).stop 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 -- 172.21.15.185:0/563962527 shutdown_connections 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 --2- 172.21.15.185:0/563962527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b76c0 0x7f3bf40b6aa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 -- 172.21.15.185:0/563962527 >> 172.21.15.185:0/563962527 conn(0x7f3bf4067860 msgr2=0x7f3bf4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:58.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 -- 172.21.15.185:0/563962527 shutdown_connections 2020-11-16T22:20:58.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.210+0000 7f3bf929d700 1 -- 172.21.15.185:0/563962527 wait complete. 2020-11-16T22:20:58.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 Processor -- start 2020-11-16T22:20:58.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- start start 2020-11-16T22:20:58.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 0x7f3bf4142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:58.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bf40b76c0 0x7f3bf4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40b32e0 con 0x7f3bf40b76c0 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40b1470 con 0x7f3bf40b72e0 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4141420 con 0x7f3bf40b64b0 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33610/0 (socket says 172.21.15.185:33610) 2020-11-16T22:20:58.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 -- 172.21.15.185:0/208071713 learned_addr learned my addr 172.21.15.185:0/208071713 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf3fff700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 0x7f3bf4142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf8a9c700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bf40b76c0 0x7f3bf4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 -- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 msgr2=0x7f3bf4142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 0x7f3bf4142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 -- 172.21.15.185:0/208071713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bf40b76c0 msgr2=0x7f3bf4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bf40b76c0 0x7f3bf4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:58.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 -- 172.21.15.185:0/208071713 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf4153450 con 0x7f3bf40b72e0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf3fff700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 0x7f3bf4142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf37fe700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f3bdc00c710 tx=0x7f3bdc007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007320 con 0x7f3bf40b72e0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc007480 con 0x7f3bf40b72e0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf41536f0 con 0x7f3bf40b72e0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc005800 con 0x7f3bf40b72e0 2020-11-16T22:20:58.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.214+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf4154100 con 0x7f3bf40b72e0 2020-11-16T22:20:58.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4153290 con 0x7f3bf40b72e0 2020-11-16T22:20:58.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3bdc010030 con 0x7f3bf40b72e0 2020-11-16T22:20:58.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bebfff700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 0x7f3bd0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:58.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bf3fff700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 0x7f3bd0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:58.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bdc06dab0 con 0x7f3bf40b72e0 2020-11-16T22:20:58.213 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.218+0000 7f3bf3fff700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 0x7f3bd0057520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3be4000bc0 tx=0x7f3be4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:58.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.222+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3bdc043d10 con 0x7f3bf40b72e0 2020-11-16T22:20:58.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:58.434+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f3bf4153880 con 0x7f3bf40b72e0 2020-11-16T22:20:59.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bebfff700 1 -- 172.21.15.185:0/208071713 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v322) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3bdc047330 con 0x7f3bf40b72e0 2020-11-16T22:20:59.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 msgr2=0x7f3bd0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 0x7f3bd0057520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f3be4000bc0 tx=0x7f3be4009040).stop 2020-11-16T22:20:59.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 msgr2=0x7f3bf4153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f3bdc00c710 tx=0x7f3bdc007c80).stop 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 shutdown_connections 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bd0055060 0x7f3bd0057520 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bf40b76c0 0x7f3bf4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bf40b64b0 0x7f3bf4142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 --2- 172.21.15.185:0/208071713 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bf40b72e0 0x7f3bf4153b90 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 >> 172.21.15.185:0/208071713 conn(0x7f3bf4067860 msgr2=0x7f3bf413d670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:59.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 shutdown_connections 2020-11-16T22:20:59.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.066+0000 7f3bf929d700 1 -- 172.21.15.185:0/208071713 wait complete. 2020-11-16T22:20:59.077 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:20:59.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 Processor -- start 2020-11-16T22:20:59.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 -- start start 2020-11-16T22:20:59.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fca1c0ae580 0x7fca1c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0ad490 con 0x7fca1c0b7440 2020-11-16T22:20:59.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0ad730 con 0x7fca1c0afc60 2020-11-16T22:20:59.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca20891700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0ad9d0 con 0x7fca1c0ae580 2020-11-16T22:20:59.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca19d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fca1c0afc60 0x7fca1c0ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55288/0 (socket says 172.21.15.185:55288) 2020-11-16T22:20:59.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca19d9b700 1 -- 172.21.15.185:0/870946337 learned_addr learned my addr 172.21.15.185:0/870946337 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:20:59.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1ad9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fca1c0ae580 0x7fca1c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495615909 0 0) 0x7fca1c0ad730 con 0x7fca1c0afc60 2020-11-16T22:20:59.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca00003240 con 0x7fca1c0afc60 2020-11-16T22:20:59.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1ad9d700 1 -- 172.21.15.185:0/870946337 >> v1:172.21.15.185:6789/0 conn(0x7fca1c0afc60 legacy=0x7fca1c0ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.217 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871604344 0 0) 0x7fca1c0ad490 con 0x7fca1c0b7440 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1ad9d700 1 -- 172.21.15.185:0/870946337 >> v1:172.21.15.96:6789/0 conn(0x7fca1c0b7440 legacy=0x7fca1c13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1ad9d700 1 -- 172.21.15.185:0/870946337 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c13e5b0 con 0x7fca1c0ae580 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1ad9d700 1 --2- 172.21.15.185:0/870946337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c0b7820 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fca14009c90 tx=0x7fca140091d0).ready entity=mon.1 client_cookie=368fc99c59bc1801 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.222+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400ba70 con 0x7fca1c0ae580 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1400bbd0 con 0x7fca1c0ae580 2020-11-16T22:20:59.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1959a700 1 -- 172.21.15.185:0/870946337 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14011400 con 0x7fca1c0ae580 2020-11-16T22:20:59.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- 172.21.15.185:0/870946337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 msgr2=0x7fca1c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 --2- 172.21.15.185:0/870946337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c0b7820 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fca14009c90 tx=0x7fca140091d0).stop 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- 172.21.15.185:0/870946337 shutdown_connections 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 --2- 172.21.15.185:0/870946337 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c0b7820 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- 172.21.15.185:0/870946337 >> 172.21.15.185:0/870946337 conn(0x7fca1c067860 msgr2=0x7fca1c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- 172.21.15.185:0/870946337 shutdown_connections 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- 172.21.15.185:0/870946337 wait complete. 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 Processor -- start 2020-11-16T22:20:59.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- start start 2020-11-16T22:20:59.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c13e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 0x7fca1c0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca1c0b7440 0x7fca1c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c13f200 con 0x7fca1c0b7440 2020-11-16T22:20:59.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c153100 con 0x7fca1c0ae580 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca20891700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c1533a0 con 0x7fca1c0afc60 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c13e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c13e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33622/0 (socket says 172.21.15.185:33622) 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1a59c700 1 -- 172.21.15.185:0/1729674350 learned_addr learned my addr 172.21.15.185:0/1729674350 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 0x7fca1c0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca1ad9d700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca1c0b7440 0x7fca1c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 -- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 msgr2=0x7fca1c13e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c13e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 -- 172.21.15.185:0/1729674350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca1c0b7440 msgr2=0x7fca1c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca1c0b7440 0x7fca1c0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.226+0000 7fca19d9b700 1 -- 172.21.15.185:0/1729674350 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c153640 con 0x7fca1c0afc60 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca19d9b700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 0x7fca1c0b3130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fca0c007fd0 tx=0x7fca0c007980).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0072e0 con 0x7fca1c0afc60 2020-11-16T22:20:59.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c007440 con 0x7fca1c0afc60 2020-11-16T22:20:59.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c005820 con 0x7fca1c0afc60 2020-11-16T22:20:59.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c153940 con 0x7fca1c0afc60 2020-11-16T22:20:59.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c153d80 con 0x7fca1c0afc60 2020-11-16T22:20:59.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c145040 con 0x7fca1c0afc60 2020-11-16T22:20:59.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fca0c010070 con 0x7fca1c0afc60 2020-11-16T22:20:59.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 0x7fc9f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca1a59c700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 0x7fc9f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca0c06d0e0 con 0x7fca1c0afc60 2020-11-16T22:20:59.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.230+0000 7fca1a59c700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 0x7fc9f4057520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fca100011a0 tx=0x7fca10007ac0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.238+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fca0c0433f0 con 0x7fca1c0afc60 2020-11-16T22:20:59.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.394+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fca1c05d4e0 con 0x7fca1c0afc60 2020-11-16T22:20:59.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.402+0000 7fca0a7fc700 1 -- 172.21.15.185:0/1729674350 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca0c018020 con 0x7fca1c0afc60 2020-11-16T22:20:59.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 msgr2=0x7fc9f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 0x7fc9f4057520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fca100011a0 tx=0x7fca10007ac0).stop 2020-11-16T22:20:59.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 msgr2=0x7fca1c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 0x7fca1c0b3130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fca0c007fd0 tx=0x7fca0c007980).stop 2020-11-16T22:20:59.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 shutdown_connections 2020-11-16T22:20:59.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc9f4055060 0x7fc9f4057520 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fca1c0ae580 0x7fca1c13e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca1c0b7440 0x7fca1c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 --2- 172.21.15.185:0/1729674350 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fca1c0afc60 0x7fca1c0b3130 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 >> 172.21.15.185:0/1729674350 conn(0x7fca1c067860 msgr2=0x7fca1c0b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 shutdown_connections 2020-11-16T22:20:59.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.406+0000 7fca20891700 1 -- 172.21.15.185:0/1729674350 wait complete. 2020-11-16T22:20:59.412 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:20:59.412 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:59.452 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:59.470 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:20:59.487 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:20:59.487 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:20:59.488 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:20:59.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 Processor -- start 2020-11-16T22:20:59.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 -- start start 2020-11-16T22:20:59.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f85300602d0 0x7f8530144a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85300b7cb0 con 0x7f8530143810 2020-11-16T22:20:59.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85300b7a10 con 0x7f85300602d0 2020-11-16T22:20:59.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f853489f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8530067470 con 0x7f85300b7570 2020-11-16T22:20:59.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f852effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f852effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55308/0 (socket says 172.21.15.185:55308) 2020-11-16T22:20:59.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f852effd700 1 -- 172.21.15.185:0/1598094340 learned_addr learned my addr 172.21.15.185:0/1598094340 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:59.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.670+0000 7f852f7fe700 1 --2- 172.21.15.185:0/1598094340 >> v2:172.21.15.96:3300/0 conn(0x7f85300602d0 0x7f8530144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852effd700 1 -- 172.21.15.185:0/1598094340 >> v2:172.21.15.96:3300/0 conn(0x7f85300602d0 msgr2=0x7f8530144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:20:59.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852effd700 1 --2- 172.21.15.185:0/1598094340 >> v2:172.21.15.96:3300/0 conn(0x7f85300602d0 0x7f8530144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852effd700 1 -- 172.21.15.185:0/1598094340 >> v1:172.21.15.96:6789/0 conn(0x7f8530143810 legacy=0x7f8530067bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852effd700 1 -- 172.21.15.185:0/1598094340 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530067710 con 0x7f85300b7570 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852e7fc700 1 -- 172.21.15.185:0/1598094340 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994048156 0 0) 0x7f85300b7cb0 con 0x7f8530143810 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852effd700 1 --2- 172.21.15.185:0/1598094340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8524006fd0 tx=0x7f8524006eb0).ready entity=mon.2 client_cookie=4d4e7ecd8395678 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852e7fc700 1 -- 172.21.15.185:0/1598094340 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524009810 con 0x7f85300b7570 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852e7fc700 1 -- 172.21.15.185:0/1598094340 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524009970 con 0x7f85300b7570 2020-11-16T22:20:59.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f852e7fc700 1 -- 172.21.15.185:0/1598094340 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524009cf0 con 0x7f85300b7570 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- 172.21.15.185:0/1598094340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 msgr2=0x7f8530143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 --2- 172.21.15.185:0/1598094340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8524006fd0 tx=0x7f8524006eb0).stop 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- 172.21.15.185:0/1598094340 shutdown_connections 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 --2- 172.21.15.185:0/1598094340 >> v2:172.21.15.96:3300/0 conn(0x7f85300602d0 0x7f8530144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 --2- 172.21.15.185:0/1598094340 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300b7570 0x7f8530143320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- 172.21.15.185:0/1598094340 >> 172.21.15.185:0/1598094340 conn(0x7f853005ba80 msgr2=0x7f853005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:20:59.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- 172.21.15.185:0/1598094340 shutdown_connections 2020-11-16T22:20:59.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- 172.21.15.185:0/1598094340 wait complete. 2020-11-16T22:20:59.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 Processor -- start 2020-11-16T22:20:59.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.674+0000 7f853489f700 1 -- start start 2020-11-16T22:20:59.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 0x7f853014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 0x7f8530153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 0x7f8530153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530154070 con 0x7f8530143810 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530157200 con 0x7f85300b7570 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85301531f0 con 0x7f85300602d0 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 0x7f853014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852effd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 0x7f8530153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 0x7f853014b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55312/0 (socket says 172.21.15.185:55312) 2020-11-16T22:20:59.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852f7fe700 1 -- 172.21.15.185:0/102053080 learned_addr learned my addr 172.21.15.185:0/102053080 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:20:59.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 0x7f8530153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 -- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 msgr2=0x7f853014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 0x7f853014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 -- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 msgr2=0x7f8530153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:20:59.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 0x7f8530153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:20:59.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 -- 172.21.15.185:0/102053080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853014b660 con 0x7f8530143810 2020-11-16T22:20:59.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852effd700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 0x7f8530153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:20:59.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f852ffff700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 0x7f8530153b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f852800c8b0 tx=0x7f8528007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800e560 con 0x7f8530143810 2020-11-16T22:20:59.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852800e6c0 con 0x7f8530143810 2020-11-16T22:20:59.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85301574a0 con 0x7f8530143810 2020-11-16T22:20:59.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800ea80 con 0x7f8530143810 2020-11-16T22:20:59.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.678+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85301476f0 con 0x7f8530143810 2020-11-16T22:20:59.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8530157040 con 0x7f8530143810 2020-11-16T22:20:59.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8528010070 con 0x7f8530143810 2020-11-16T22:20:59.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f850f7fe700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 0x7f8508057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:20:59.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f852806e830 con 0x7f8530143810 2020-11-16T22:20:59.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f852f7fe700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 0x7f8508057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:20:59.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.682+0000 7f852f7fe700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 0x7f8508057520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f8520006a90 tx=0x7f8520006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:20:59.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.690+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8528044980 con 0x7f8530143810 2020-11-16T22:20:59.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.898+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85301533b0 con 0x7f8530143810 2020-11-16T22:20:59.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.898+0000 7f850f7fe700 1 -- 172.21.15.185:0/102053080 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f8528047fa0 con 0x7f8530143810 2020-11-16T22:21:00.095 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:00.096 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:20:51.229431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:21:00.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 msgr2=0x7f8508057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:00.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 0x7f8508057520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f8520006a90 tx=0x7f8520006df0).stop 2020-11-16T22:21:00.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 msgr2=0x7f8530153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:00.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 0x7f8530153b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f852800c8b0 tx=0x7f8528007c10).stop 2020-11-16T22:21:00.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 shutdown_connections 2020-11-16T22:21:00.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8508055060 0x7f8508057520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8530143810 0x7f8530153b80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f85300602d0 0x7f853014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 --2- 172.21.15.185:0/102053080 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f85300b7570 0x7f8530153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 >> 172.21.15.185:0/102053080 conn(0x7f853005ba80 msgr2=0x7f85300b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:00.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 shutdown_connections 2020-11-16T22:21:00.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:20:59.902+0000 7f853489f700 1 -- 172.21.15.185:0/102053080 wait complete. 2020-11-16T22:21:00.299 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 75 2020-11-16T22:21:00.300 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:00.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 Processor -- start 2020-11-16T22:21:00.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 -- start start 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f96f80b6aa0 0x7f96f80acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8147f90 con 0x7f96f80b72e0 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f80af020 con 0x7f96f80b6aa0 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fe3eb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80b4810 con 0x7f96f80b64b0 2020-11-16T22:21:00.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fcbe8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:00.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fd3e9700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f96f80b72e0 0x7f96f8144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55314/0 (socket says 172.21.15.185:55314) 2020-11-16T22:21:00.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fd3e9700 1 -- 172.21.15.185:0/1944326260 learned_addr learned my addr 172.21.15.185:0/1944326260 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:00.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.438+0000 7f96fcbe8700 1 --2- 172.21.15.185:0/1944326260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55320/0 (socket says 172.21.15.185:55320) 2020-11-16T22:21:00.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fdbea700 1 --2- 172.21.15.185:0/1944326260 >> v2:172.21.15.96:3300/0 conn(0x7f96f80b6aa0 0x7f96f80acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:00.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fcbe8700 1 -- 172.21.15.185:0/1944326260 >> v2:172.21.15.96:3300/0 conn(0x7f96f80b6aa0 msgr2=0x7f96f80acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:00.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fcbe8700 1 --2- 172.21.15.185:0/1944326260 >> v2:172.21.15.96:3300/0 conn(0x7f96f80b6aa0 0x7f96f80acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fcbe8700 1 -- 172.21.15.185:0/1944326260 >> v1:172.21.15.185:6789/0 conn(0x7f96f80b72e0 legacy=0x7f96f8144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:00.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fcbe8700 1 -- 172.21.15.185:0/1944326260 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8148460 con 0x7f96f80b64b0 2020-11-16T22:21:00.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fcbe8700 1 --2- 172.21.15.185:0/1944326260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f96ec00b9a0 tx=0x7f96ec006df0).ready entity=mon.2 client_cookie=117999de46e3a965 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:00.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96ebfff700 1 -- 172.21.15.185:0/1944326260 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0097b0 con 0x7f96f80b64b0 2020-11-16T22:21:00.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96ebfff700 1 -- 172.21.15.185:0/1944326260 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec009910 con 0x7f96f80b64b0 2020-11-16T22:21:00.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96ebfff700 1 -- 172.21.15.185:0/1944326260 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec009c90 con 0x7f96f80b64b0 2020-11-16T22:21:00.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- 172.21.15.185:0/1944326260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 msgr2=0x7f96f80b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:00.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/1944326260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f96ec00b9a0 tx=0x7f96ec006df0).stop 2020-11-16T22:21:00.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- 172.21.15.185:0/1944326260 shutdown_connections 2020-11-16T22:21:00.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/1944326260 >> v2:172.21.15.96:3300/0 conn(0x7f96f80b6aa0 0x7f96f80acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/1944326260 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b64b0 0x7f96f80b76c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:00.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- 172.21.15.185:0/1944326260 >> 172.21.15.185:0/1944326260 conn(0x7f96f8067860 msgr2=0x7f96f8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:00.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- 172.21.15.185:0/1944326260 shutdown_connections 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- 172.21.15.185:0/1944326260 wait complete. 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 Processor -- start 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.442+0000 7f96fe3eb700 1 -- start start 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f80b6aa0 0x7f96f80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 0x7f96f80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:00.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f813fec0 con 0x7f96f80b6aa0 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8113c60 con 0x7f96f8156c60 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8113f00 con 0x7f96f80b72e0 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fcbe8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 0x7f96f80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33654/0 (socket says 172.21.15.185:33654) 2020-11-16T22:21:00.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 -- 172.21.15.185:0/3400563394 learned_addr learned my addr 172.21.15.185:0/3400563394 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:01.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fd3e9700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f80b6aa0 0x7f96f80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:01.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 -- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 msgr2=0x7f96f80b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:01.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 0x7f96f80b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 -- 172.21.15.185:0/3400563394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f80b6aa0 msgr2=0x7f96f80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f80b6aa0 0x7f96f80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 -- 172.21.15.185:0/3400563394 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f81141a0 con 0x7f96f8156c60 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fcbe8700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 0x7f96f80b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fdbea700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f96f000ec20 tx=0x7f96f0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000c070 con 0x7f96f8156c60 2020-11-16T22:21:01.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f00097d0 con 0x7f96f8156c60 2020-11-16T22:21:01.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8114440 con 0x7f96f8156c60 2020-11-16T22:21:01.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009b70 con 0x7f96f8156c60 2020-11-16T22:21:01.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f80b3840 con 0x7f96f8156c60 2020-11-16T22:21:01.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.446+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f81445b0 con 0x7f96f8156c60 2020-11-16T22:21:01.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.450+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f96f001c400 con 0x7f96f8156c60 2020-11-16T22:21:01.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.450+0000 7f96e97fa700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 0x7f96d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.450+0000 7f96fd3e9700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 0x7f96d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.450+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96f006dd50 con 0x7f96f8156c60 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.450+0000 7f96fd3e9700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 0x7f96d0057520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f96f40010a0 tx=0x7f96f4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.454+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f96f81445b0 con 0x7f96f8156c60 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96f81445b0 con 0x7f96f8156c60 2020-11-16T22:21:01.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96e97fa700 1 -- 172.21.15.185:0/3400563394 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f96f81445b0 con 0x7f96f8156c60 2020-11-16T22:21:01.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 msgr2=0x7f96d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:01.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 0x7f96d0057520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f96f40010a0 tx=0x7f96f4009040).stop 2020-11-16T22:21:01.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 msgr2=0x7f96f81570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:01.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f96f000ec20 tx=0x7f96f0008040).stop 2020-11-16T22:21:01.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 shutdown_connections 2020-11-16T22:21:01.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96d0055060 0x7f96d0057520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f8156c60 0x7f96f81570c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f80b6aa0 0x7f96f80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 --2- 172.21.15.185:0/3400563394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f80b72e0 0x7f96f80b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 >> 172.21.15.185:0/3400563394 conn(0x7f96f8067860 msgr2=0x7f96f8067c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 shutdown_connections 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:00.606+0000 7f96fe3eb700 1 -- 172.21.15.185:0/3400563394 wait complete. 2020-11-16T22:21:01.459 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:01.460 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":65,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:51.735891+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:02.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:18:48.576973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:18:49.752256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:21:02.686 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:21:02.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 Processor -- start 2020-11-16T22:21:02.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 -- start start 2020-11-16T22:21:02.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe668061240 0x7fe6680b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680b4350 con 0x7fe668063fd0 2020-11-16T22:21:02.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680629a0 con 0x7fe6680b4c70 2020-11-16T22:21:02.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe667fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6680b60a0 con 0x7fe668061240 2020-11-16T22:21:02.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe6667fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe6680b4c70 0x7fe668143310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55328/0 (socket says 172.21.15.185:55328) 2020-11-16T22:21:02.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe6667fc700 1 -- 172.21.15.185:0/1712180018 learned_addr learned my addr 172.21.15.185:0/1712180018 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:02.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe6677fe700 1 --2- 172.21.15.185:0/1712180018 >> v2:172.21.15.96:3300/0 conn(0x7fe668061240 0x7fe6680b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344027216 0 0) 0x7fe6680629a0 con 0x7fe6680b4c70 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.830+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe654003240 con 0x7fe6680b4c70 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1933568650 0 0) 0x7fe654003240 con 0x7fe6680b4c70 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 >> v2:172.21.15.96:3300/0 conn(0x7fe668061240 msgr2=0x7fe6680b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 --2- 172.21.15.185:0/1712180018 >> v2:172.21.15.96:3300/0 conn(0x7fe668061240 0x7fe6680b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 >> v1:172.21.15.96:6789/0 conn(0x7fe668063fd0 legacy=0x7fe6680b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668062c40 con 0x7fe6680b4c70 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fe65c0014e0 con 0x7fe6680b4c70 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 >> v1:172.21.15.185:6789/0 conn(0x7fe6680b4c70 legacy=0x7fe668143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe654003730 0x7fe654003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 0x7fe654004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe654004e50 0x7fe65400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b60a0 con 0x7fe654004e50 2020-11-16T22:21:02.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540041d0 con 0x7fe654003730 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65400bc20 con 0x7fe6540044e0 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 0x7fe654004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe666ffd700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe654003730 0x7fe654003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6677fe700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe654004e50 0x7fe65400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 -- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe654003730 msgr2=0x7fe654003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe654003730 0x7fe654003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 -- 172.21.15.185:0/1712180018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe654004e50 msgr2=0x7fe65400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe654004e50 0x7fe65400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 -- 172.21.15.185:0/1712180018 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654003240 con 0x7fe6540044e0 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6677fe700 1 -- 172.21.15.185:0/1712180018 reap_dead start 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe666ffd700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe654003730 0x7fe654003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe6667fc700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 0x7fe654004960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe65c00c470 tx=0x7fe668062820).ready entity=mon.2 client_cookie=c517fb00be632a4d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0070d0 con 0x7fe6540044e0 2020-11-16T22:21:02.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c007230 con 0x7fe6540044e0 2020-11-16T22:21:02.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe665ffb700 1 -- 172.21.15.185:0/1712180018 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0055f0 con 0x7fe6540044e0 2020-11-16T22:21:02.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 -- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 msgr2=0x7fe654004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 0x7fe654004960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe65c00c470 tx=0x7fe668062820).stop 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 -- 172.21.15.185:0/1712180018 shutdown_connections 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 --2- 172.21.15.185:0/1712180018 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6540044e0 0x7fe654004960 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 -- 172.21.15.185:0/1712180018 >> 172.21.15.185:0/1712180018 conn(0x7fe668067940 msgr2=0x7fe668067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 -- 172.21.15.185:0/1712180018 shutdown_connections 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.834+0000 7fe667fff700 1 -- 172.21.15.185:0/1712180018 wait complete. 2020-11-16T22:21:02.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 Processor -- start 2020-11-16T22:21:02.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 -- start start 2020-11-16T22:21:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6540044e0 0x7fe668140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 0x7fe6681534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe668157360 con 0x7fe6540044e0 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b2810 con 0x7fe668145010 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe667fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b2ab0 con 0x7fe6680ae690 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe6667fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe6677fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 0x7fe6681534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe666ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6540044e0 0x7fe668140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe6667fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55348/0 (socket says 172.21.15.185:55348) 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.838+0000 7fe6667fc700 1 -- 172.21.15.185:0/3456183665 learned_addr learned my addr 172.21.15.185:0/3456183665 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 -- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 msgr2=0x7fe668153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 -- 172.21.15.185:0/3456183665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6540044e0 msgr2=0x7fe668140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6540044e0 0x7fe668140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 -- 172.21.15.185:0/3456183665 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668141070 con 0x7fe668145010 2020-11-16T22:21:02.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6667fc700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe6677fe700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 0x7fe6681534f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fe6680677d0 tx=0x7fe660008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660006cb0 con 0x7fe668145010 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66000e8a0 con 0x7fe668145010 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6680ad3c0 con 0x7fe668145010 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000ec40 con 0x7fe668145010 2020-11-16T22:21:02.837 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe668157090 con 0x7fe668145010 2020-11-16T22:21:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6681574d0 con 0x7fe668145010 2020-11-16T22:21:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe66001a070 con 0x7fe668145010 2020-11-16T22:21:02.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 0x7fe644057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe66006e0c0 con 0x7fe668145010 2020-11-16T22:21:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe666ffd700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 0x7fe644057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.842+0000 7fe666ffd700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 0x7fe644057520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe66805fa60 tx=0x7fe658009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:02.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.846+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe6681574d0 con 0x7fe668145010 2020-11-16T22:21:02.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:02.998+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe6681574d0 con 0x7fe668145010 2020-11-16T22:21:02.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe64effd700 1 -- 172.21.15.185:0/3456183665 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v65) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe6681574d0 con 0x7fe668145010 2020-11-16T22:21:02.993 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:21:02.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 msgr2=0x7fe644057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 0x7fe644057520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe66805fa60 tx=0x7fe658009040).stop 2020-11-16T22:21:02.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 msgr2=0x7fe6681534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:02.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 0x7fe6681534f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fe6680677d0 tx=0x7fe660008040).stop 2020-11-16T22:21:02.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 shutdown_connections 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe644055060 0x7fe644057520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe668145010 0x7fe6681534f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6540044e0 0x7fe668140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 --2- 172.21.15.185:0/3456183665 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe6680ae690 0x7fe668153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 >> 172.21.15.185:0/3456183665 conn(0x7fe668067940 msgr2=0x7fe6680b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 shutdown_connections 2020-11-16T22:21:02.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.002+0000 7fe667fff700 1 -- 172.21.15.185:0/3456183665 wait complete. 2020-11-16T22:21:03.015 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:21:03.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 Processor -- start 2020-11-16T22:21:03.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 -- start start 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f35f80b5d70 0x7f35f80ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f80b7290 con 0x7f35f80b4470 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f80b4bf0 con 0x7f35f8143470 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f80b74c0 con 0x7f35f80b5d70 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f35f80b5d70 0x7f35f80ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f35f80b5d70 0x7f35f80ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33678/0 (socket says 172.21.15.185:33678) 2020-11-16T22:21:03.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 -- 172.21.15.185:0/2727032992 learned_addr learned my addr 172.21.15.185:0/2727032992 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:03.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fdc18700 1 --1- 172.21.15.185:0/2727032992 >> v1:172.21.15.185:6789/0 conn(0x7f35f8143470 0x7f35f8143920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55352/0 (socket says 172.21.15.185:55352) 2020-11-16T22:21:03.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 -- 172.21.15.185:0/2727032992 >> v1:172.21.15.185:6789/0 conn(0x7f35f8143470 legacy=0x7f35f8143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 -- 172.21.15.185:0/2727032992 >> v1:172.21.15.96:6789/0 conn(0x7f35f80b4470 legacy=0x7f35f80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 -- 172.21.15.185:0/2727032992 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80b7760 con 0x7f35f80b5d70 2020-11-16T22:21:03.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fd417700 1 --2- 172.21.15.185:0/2727032992 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f80b5d70 0x7f35f80ba800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f35f4006fd0 tx=0x7f35f4006eb0).ready entity=mon.1 client_cookie=cf13d5ff8240ff41 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35e7fff700 1 -- 172.21.15.185:0/2727032992 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40098d0 con 0x7f35f80b5d70 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35e7fff700 1 -- 172.21.15.185:0/2727032992 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4009a30 con 0x7f35f80b5d70 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35e7fff700 1 -- 172.21.15.185:0/2727032992 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4009db0 con 0x7f35f80b5d70 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 -- 172.21.15.185:0/2727032992 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f80b5d70 msgr2=0x7f35f80ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.150+0000 7f35fe419700 1 --2- 172.21.15.185:0/2727032992 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f80b5d70 0x7f35f80ba800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f35f4006fd0 tx=0x7f35f4006eb0).stop 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- 172.21.15.185:0/2727032992 shutdown_connections 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 --2- 172.21.15.185:0/2727032992 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f80b5d70 0x7f35f80ba800 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- 172.21.15.185:0/2727032992 >> 172.21.15.185:0/2727032992 conn(0x7f35f805ba80 msgr2=0x7f35f805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- 172.21.15.185:0/2727032992 shutdown_connections 2020-11-16T22:21:03.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- 172.21.15.185:0/2727032992 wait complete. 2020-11-16T22:21:03.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 Processor -- start 2020-11-16T22:21:03.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- start start 2020-11-16T22:21:03.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f80b4470 0x7f35f813e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35f80b5d70 0x7f35f80ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f80ae160 con 0x7f35f80b4470 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8147c40 con 0x7f35f8143470 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fe419700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81471e0 con 0x7f35f80b5d70 2020-11-16T22:21:03.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fdc18700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fdc18700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33686/0 (socket says 172.21.15.185:33686) 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fdc18700 1 -- 172.21.15.185:0/4248089792 learned_addr learned my addr 172.21.15.185:0/4248089792 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fcc16700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35f80b5d70 0x7f35f80ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.154+0000 7f35fd417700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f80b4470 0x7f35f813e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 -- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35f80b5d70 msgr2=0x7f35f80ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35f80b5d70 0x7f35f80ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 -- 172.21.15.185:0/4248089792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f80b4470 msgr2=0x7f35f813e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f80b4470 0x7f35f813e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 -- 172.21.15.185:0/4248089792 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8147480 con 0x7f35f8143470 2020-11-16T22:21:03.151 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fdc18700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f35f000cac0 tx=0x7f35f0007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0005ac0 con 0x7f35f8143470 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0005c20 con 0x7f35f8143470 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f80ad2e0 con 0x7f35f8143470 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0007110 con 0x7f35f8143470 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f8147930 con 0x7f35f8143470 2020-11-16T22:21:03.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f805d200 con 0x7f35f8143470 2020-11-16T22:21:03.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f35f0010030 con 0x7f35f8143470 2020-11-16T22:21:03.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 0x7f35d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35f006dc80 con 0x7f35f8143470 2020-11-16T22:21:03.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.158+0000 7f35fd417700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 0x7f35d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.162+0000 7f35fd417700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 0x7f35d0057520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f35f400b1b0 tx=0x7f35f4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.166+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f35f0043ee0 con 0x7f35f8143470 2020-11-16T22:21:03.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.318+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35f805c720 con 0x7f35f8143470 2020-11-16T22:21:03.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.318+0000 7f35e57fa700 1 -- 172.21.15.185:0/4248089792 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35f0047500 con 0x7f35f8143470 2020-11-16T22:21:03.312 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:03.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 msgr2=0x7f35d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 0x7f35d0057520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f35f400b1b0 tx=0x7f35f4015000).stop 2020-11-16T22:21:03.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 msgr2=0x7f35f80adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f35f000cac0 tx=0x7f35f0007ef0).stop 2020-11-16T22:21:03.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 shutdown_connections 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f35d0055060 0x7f35d0057520 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f35f8143470 0x7f35f80adc70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f80b4470 0x7f35f813e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 --2- 172.21.15.185:0/4248089792 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f35f80b5d70 0x7f35f80ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 >> 172.21.15.185:0/4248089792 conn(0x7f35f805ba80 msgr2=0x7f35f8141180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 shutdown_connections 2020-11-16T22:21:03.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.322+0000 7f35fe419700 1 -- 172.21.15.185:0/4248089792 wait complete. 2020-11-16T22:21:03.326 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:21:03.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 Processor -- start 2020-11-16T22:21:03.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 -- start start 2020-11-16T22:21:03.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fea54064220 0x7fea5413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 0x7fea5413f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540b3370 con 0x7fea541428e0 2020-11-16T22:21:03.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea540b3610 con 0x7fea540af960 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.458+0000 7fea5a4f1700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea540b38b0 con 0x7fea54064220 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fea54064220 0x7fea5413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fea54064220 0x7fea5413efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33692/0 (socket says 172.21.15.185:33692) 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 -- 172.21.15.185:0/3974926184 learned_addr learned my addr 172.21.15.185:0/3974926184 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea58a8e700 1 --2- 172.21.15.185:0/3974926184 >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 0x7fea5413f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 -- 172.21.15.185:0/3974926184 >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 msgr2=0x7fea5413f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 --2- 172.21.15.185:0/3974926184 >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 0x7fea5413f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 -- 172.21.15.185:0/3974926184 >> v1:172.21.15.96:6789/0 conn(0x7fea541428e0 legacy=0x7fea540b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 -- 172.21.15.185:0/3974926184 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540b6100 con 0x7fea54064220 2020-11-16T22:21:03.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea58a8e700 1 --2- 172.21.15.185:0/3974926184 >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 0x7fea5413f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:03.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea53fff700 1 --2- 172.21.15.185:0/3974926184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea54064220 0x7fea5413efb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fea54066910 tx=0x7fea48006eb0).ready entity=mon.1 client_cookie=f57e46e694cc4014 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea52ffd700 1 -- 172.21.15.185:0/3974926184 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea480098d0 con 0x7fea54064220 2020-11-16T22:21:03.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea52ffd700 1 -- 172.21.15.185:0/3974926184 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48009a30 con 0x7fea54064220 2020-11-16T22:21:03.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea52ffd700 1 -- 172.21.15.185:0/3974926184 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48009db0 con 0x7fea54064220 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- 172.21.15.185:0/3974926184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea54064220 msgr2=0x7fea5413efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/3974926184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea54064220 0x7fea5413efb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fea54066910 tx=0x7fea48006eb0).stop 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- 172.21.15.185:0/3974926184 shutdown_connections 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/3974926184 >> v2:172.21.15.96:3300/0 conn(0x7fea540af960 0x7fea5413f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/3974926184 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea54064220 0x7fea5413efb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- 172.21.15.185:0/3974926184 >> 172.21.15.185:0/3974926184 conn(0x7fea54067a60 msgr2=0x7fea54067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- 172.21.15.185:0/3974926184 shutdown_connections 2020-11-16T22:21:03.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- 172.21.15.185:0/3974926184 wait complete. 2020-11-16T22:21:03.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 Processor -- start 2020-11-16T22:21:03.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.462+0000 7fea5a4f1700 1 -- start start 2020-11-16T22:21:03.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea54064220 0x7fea5413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 0x7fea54140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea541422b0 con 0x7fea54064220 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54144db0 con 0x7fea540af960 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54145050 con 0x7fea541428e0 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea58a8e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 0x7fea54140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33700/0 (socket says 172.21.15.185:33700) 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 -- 172.21.15.185:0/4166918527 learned_addr learned my addr 172.21.15.185:0/4166918527 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea53fff700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea54064220 0x7fea5413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 -- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 msgr2=0x7fea54140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 0x7fea54140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 -- 172.21.15.185:0/4166918527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea54064220 msgr2=0x7fea5413d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea54064220 0x7fea5413d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 -- 172.21.15.185:0/4166918527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea541424c0 con 0x7fea540af960 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea58a8e700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 0x7fea54140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea537fe700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fea4400c6e0 tx=0x7fea44007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44007020 con 0x7fea540af960 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44007180 con 0x7fea540af960 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea440064f0 con 0x7fea540af960 2020-11-16T22:21:03.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5413dbf0 con 0x7fea540af960 2020-11-16T22:21:03.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5413fa80 con 0x7fea540af960 2020-11-16T22:21:03.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea54145210 con 0x7fea540af960 2020-11-16T22:21:03.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.466+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fea44010070 con 0x7fea540af960 2020-11-16T22:21:03.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.470+0000 7fea3bfff700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 0x7fea30057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.470+0000 7fea53fff700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 0x7fea30057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.470+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea4406d980 con 0x7fea540af960 2020-11-16T22:21:03.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.470+0000 7fea53fff700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 0x7fea30057520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fea48006ef0 tx=0x7fea48015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.474+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fea44043b10 con 0x7fea540af960 2020-11-16T22:21:03.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.626+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea5413fc70 con 0x7fea540af960 2020-11-16T22:21:03.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.626+0000 7fea3bfff700 1 -- 172.21.15.185:0/4166918527 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+10 (secure 0 0 0) 0x7fea44020070 con 0x7fea540af960 2020-11-16T22:21:03.620 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 msgr2=0x7fea30057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 0x7fea30057520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fea48006ef0 tx=0x7fea48015000).stop 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 msgr2=0x7fea54140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fea4400c6e0 tx=0x7fea44007b90).stop 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 shutdown_connections 2020-11-16T22:21:03.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fea30055060 0x7fea30057520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fea541428e0 0x7fea54140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea54064220 0x7fea5413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 --2- 172.21.15.185:0/4166918527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fea540af960 0x7fea54140970 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 >> 172.21.15.185:0/4166918527 conn(0x7fea54067a60 msgr2=0x7fea5413c720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 shutdown_connections 2020-11-16T22:21:03.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.630+0000 7fea5a4f1700 1 -- 172.21.15.185:0/4166918527 wait complete. 2020-11-16T22:21:03.640 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:21:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.770+0000 7f45fa0ae700 1 Processor -- start 2020-11-16T22:21:03.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.770+0000 7f45fa0ae700 1 -- start start 2020-11-16T22:21:03.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f45f40b3020 0x7f45f40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40abef0 0x7f45f4145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40acdf0 con 0x7f45f4067890 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f40ad2c0 con 0x7f45f40b3020 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40ad020 con 0x7f45f40abef0 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f98ad700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f45f4067890 0x7f45f4067d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55380/0 (socket says 172.21.15.185:55380) 2020-11-16T22:21:03.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f98ad700 1 -- 172.21.15.185:0/2584717784 learned_addr learned my addr 172.21.15.185:0/2584717784 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:03.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f88ab700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40abef0 0x7f45f4145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> v2:172.21.15.96:3300/0 conn(0x7f45f40b3020 0x7f45f40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353674724 0 0) 0x7f45f40acdf0 con 0x7f45f4067890 2020-11-16T22:21:03.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d8003240 con 0x7f45f4067890 2020-11-16T22:21:03.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1085260922 0 0) 0x7f45d8003240 con 0x7f45f4067890 2020-11-16T22:21:03.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40abef0 msgr2=0x7f45f4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40abef0 0x7f45f4145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 >> v2:172.21.15.96:3300/0 conn(0x7f45f40b3020 msgr2=0x7f45f40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 --2- 172.21.15.185:0/2584717784 >> v2:172.21.15.96:3300/0 conn(0x7f45f40b3020 0x7f45f40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4145460 con 0x7f45f4067890 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f45ec001860 con 0x7f45f4067890 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> v2:172.21.15.96:3300/0 conn(0x7f45f40b3020 0x7f45f40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 >> v1:172.21.15.185:6789/0 conn(0x7f45f4067890 legacy=0x7f45f4067d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45d8003730 0x7f45d8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45d80044e0 0x7f45d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8004e50 0x7f45d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40ad2c0 con 0x7f45d8004e50 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40ad020 con 0x7f45d8003730 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8004240 con 0x7f45d80044e0 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f88ab700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40abef0 0x7f45f4145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:03.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45d8003730 0x7f45d8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f88ab700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45d80044e0 0x7f45d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f98ad700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8004e50 0x7f45d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 -- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45d80044e0 msgr2=0x7f45d8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45d80044e0 0x7f45d8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 -- 172.21.15.185:0/2584717784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8004e50 msgr2=0x7f45d800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8004e50 0x7f45d800b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 -- 172.21.15.185:0/2584717784 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8003240 con 0x7f45d8003730 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f88ab700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45d80044e0 0x7f45d8004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f90ac700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45d8003730 0x7f45d8003b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f45f000be70 tx=0x7f45f0006df0).ready entity=mon.1 client_cookie=4c8aa71893972bb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000b580 con 0x7f45d8003730 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000ed70 con 0x7f45d8003730 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45ebfff700 1 -- 172.21.15.185:0/2584717784 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000eee0 con 0x7f45d8003730 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45d8003730 msgr2=0x7f45d8003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/2584717784 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45d8003730 0x7f45d8003b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f45f000be70 tx=0x7f45f0006df0).stop 2020-11-16T22:21:03.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45f98ad700 1 -- 172.21.15.185:0/2584717784 reap_dead start 2020-11-16T22:21:03.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- 172.21.15.185:0/2584717784 shutdown_connections 2020-11-16T22:21:03.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.774+0000 7f45fa0ae700 1 -- 172.21.15.185:0/2584717784 >> 172.21.15.185:0/2584717784 conn(0x7f45f4060130 msgr2=0x7f45f4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- 172.21.15.185:0/2584717784 shutdown_connections 2020-11-16T22:21:03.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- 172.21.15.185:0/2584717784 wait complete. 2020-11-16T22:21:03.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 Processor -- start 2020-11-16T22:21:03.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- start start 2020-11-16T22:21:03.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8003730 0x7f45f4136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 0x7f45f40655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40ad840 con 0x7f45d8003730 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40672d0 con 0x7f45f40ae800 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45fa0ae700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4067500 con 0x7f45f40af170 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45f88ab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45f98ad700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 0x7f45f40655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45f88ab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33720/0 (socket says 172.21.15.185:33720) 2020-11-16T22:21:03.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45f88ab700 1 -- 172.21.15.185:0/1874467919 learned_addr learned my addr 172.21.15.185:0/1874467919 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.778+0000 7f45f90ac700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8003730 0x7f45f4136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 -- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 msgr2=0x7f45f40aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 -- 172.21.15.185:0/1874467919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8003730 msgr2=0x7f45f4136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8003730 0x7f45f4136ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 -- 172.21.15.185:0/1874467919 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f405d4e0 con 0x7f45f40af170 2020-11-16T22:21:03.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f88ab700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45f98ad700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 0x7f45f40655a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f45f4064a70 tx=0x7f45ec009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec004a90 con 0x7f45f40af170 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec004c50 con 0x7f45f40af170 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f40ae3a0 con 0x7f45f40af170 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec007170 con 0x7f45f40af170 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f41364c0 con 0x7f45f40af170 2020-11-16T22:21:03.779 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f40ae530 con 0x7f45f40af170 2020-11-16T22:21:03.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.782+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f45ec0074e0 con 0x7f45f40af170 2020-11-16T22:21:03.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.786+0000 7f45e97fa700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 0x7f45cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:03.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.786+0000 7f45f90ac700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 0x7f45cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:03.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.786+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f45ec06df10 con 0x7f45f40af170 2020-11-16T22:21:03.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.786+0000 7f45f90ac700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 0x7f45cc057520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f45f0006d00 tx=0x7f45f0019000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:03.782 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.790+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f45ec044170 con 0x7f45f40af170 2020-11-16T22:21:03.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.938+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45f405d200 con 0x7f45f40af170 2020-11-16T22:21:03.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.938+0000 7f45e97fa700 1 -- 172.21.15.185:0/1874467919 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+10 (secure 0 0 0) 0x7f45ec047790 con 0x7f45f40af170 2020-11-16T22:21:03.932 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:03.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 msgr2=0x7f45cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 0x7f45cc057520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f45f0006d00 tx=0x7f45f0019000).stop 2020-11-16T22:21:03.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 msgr2=0x7f45f40655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:03.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 0x7f45f40655a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f45f4064a70 tx=0x7f45ec009040).stop 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 shutdown_connections 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f45cc055060 0x7f45cc057520 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f45f40af170 0x7f45f40655a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8003730 0x7f45f4136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 --2- 172.21.15.185:0/1874467919 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f45f40ae800 0x7f45f40aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 >> 172.21.15.185:0/1874467919 conn(0x7f45f4060130 msgr2=0x7f45f4140f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:03.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 shutdown_connections 2020-11-16T22:21:03.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:03.942+0000 7f45fa0ae700 1 -- 172.21.15.185:0/1874467919 wait complete. 2020-11-16T22:21:03.946 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:04.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.078+0000 7f97effff700 1 Processor -- start 2020-11-16T22:21:04.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.078+0000 7f97effff700 1 -- start start 2020-11-16T22:21:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97effff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97f0067910 0x7f97f0144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97effff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b3020 con 0x7f97f0142f30 2020-11-16T22:21:04.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97effff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b3280 con 0x7f97f00b7610 2020-11-16T22:21:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97effff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00b3520 con 0x7f97f0067910 2020-11-16T22:21:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97eeffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f97f00b7610 0x7f97f01441f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55396/0 (socket says 172.21.15.185:55396) 2020-11-16T22:21:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97eeffd700 1 -- 172.21.15.185:0/583772358 learned_addr learned my addr 172.21.15.185:0/583772358 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:04.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97ee7fc700 1 --2- 172.21.15.185:0/583772358 >> v2:172.21.15.96:3300/0 conn(0x7f97f0067910 0x7f97f0144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628260693 0 0) 0x7f97f00b3280 con 0x7f97f00b7610 2020-11-16T22:21:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d4003240 con 0x7f97f00b7610 2020-11-16T22:21:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97ee7fc700 1 -- 172.21.15.185:0/583772358 >> v1:172.21.15.185:6789/0 conn(0x7f97f00b7610 legacy=0x7f97f01441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97ee7fc700 1 -- 172.21.15.185:0/583772358 >> v1:172.21.15.96:6789/0 conn(0x7f97f0142f30 legacy=0x7f97f0143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466627575 0 0) 0x7f97f00b3020 con 0x7f97f0142f30 2020-11-16T22:21:04.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97ee7fc700 1 -- 172.21.15.185:0/583772358 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0067f40 con 0x7f97f0067910 2020-11-16T22:21:04.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.082+0000 7f97ee7fc700 1 --2- 172.21.15.185:0/583772358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f0144900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f97d800b9a0 tx=0x7f97d8006df0).ready entity=mon.0 client_cookie=51305e62b84909d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.086+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80097b0 con 0x7f97f0067910 2020-11-16T22:21:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.086+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8009910 con 0x7f97f0067910 2020-11-16T22:21:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.086+0000 7f97edffb700 1 -- 172.21.15.185:0/583772358 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8009c90 con 0x7f97f0067910 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- 172.21.15.185:0/583772358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 msgr2=0x7f97f0144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 --2- 172.21.15.185:0/583772358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f0144900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f97d800b9a0 tx=0x7f97d8006df0).stop 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- 172.21.15.185:0/583772358 shutdown_connections 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 --2- 172.21.15.185:0/583772358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f0144900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- 172.21.15.185:0/583772358 >> 172.21.15.185:0/583772358 conn(0x7f97f00637b0 msgr2=0x7f97f00b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:04.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- 172.21.15.185:0/583772358 shutdown_connections 2020-11-16T22:21:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- 172.21.15.185:0/583772358 wait complete. 2020-11-16T22:21:04.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 Processor -- start 2020-11-16T22:21:04.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- start start 2020-11-16T22:21:04.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 0x7f97f0157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0154250 con 0x7f97f0067910 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0142cc0 con 0x7f97f00b7610 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97effff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01569d0 con 0x7f97f0142f30 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97ee7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97ee7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33734/0 (socket says 172.21.15.185:33734) 2020-11-16T22:21:04.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97ee7fc700 1 -- 172.21.15.185:0/3934780813 learned_addr learned my addr 172.21.15.185:0/3934780813 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:04.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97ef7fe700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 0x7f97f0157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.090+0000 7f97eeffd700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 -- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 msgr2=0x7f97f0157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 0x7f97f0157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 -- 172.21.15.185:0/3934780813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 msgr2=0x7f97f014b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f014b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 -- 172.21.15.185:0/3934780813 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0142780 con 0x7f97f00b7610 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ef7fe700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 0x7f97f0157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97ee7fc700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f97d800be60 tx=0x7f97d8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8011d50 con 0x7f97f00b7610 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d801e460 con 0x7f97f00b7610 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d801e7e0 con 0x7f97f00b7610 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f0142a20 con 0x7f97f00b7610 2020-11-16T22:21:04.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f00af1a0 con 0x7f97f00b7610 2020-11-16T22:21:04.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f97d8025030 con 0x7f97f00b7610 2020-11-16T22:21:04.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f005d200 con 0x7f97f00b7610 2020-11-16T22:21:04.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 0x7f97c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f97d806f420 con 0x7f97f00b7610 2020-11-16T22:21:04.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97eeffd700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 0x7f97c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.094+0000 7f97eeffd700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 0x7f97c0057520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f97e00055a0 tx=0x7f97e0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.102+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f97d8045730 con 0x7f97f00b7610 2020-11-16T22:21:04.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.306+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f00b17e0 con 0x7f97f00b7610 2020-11-16T22:21:04.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.306+0000 7f97e6ffd700 1 -- 172.21.15.185:0/3934780813 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f97d8023070 con 0x7f97f00b7610 2020-11-16T22:21:04.298 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:04.299 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:18:50.731986+0000","modified":"2020-11-16T22:20:51.229431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6642,"mds_1":6188,"mds_2":6185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6185":{"gid":6185,"name":"b","rank":2,"incarnation":70,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/3171513205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3171513205},{"type":"v1","addr":"172.21.15.96:6835","nonce":3171513205}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_6188":{"gid":6188,"name":"a","rank":1,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.185:6835/3676791890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":3676791890},{"type":"v1","addr":"172.21.15.185:6835","nonce":3676791890}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_6642":{"gid":6642,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6833/974001631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":974001631},{"type":"v1","addr":"172.21.15.96:6833","nonce":974001631}]},"join_fscid":-1,"export_targets":[1],"features":4540138297136906239,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-16T22:21:04.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 msgr2=0x7f97c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 0x7f97c0057520 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f97e00055a0 tx=0x7f97e0009040).stop 2020-11-16T22:21:04.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 msgr2=0x7f97f0156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f97d800be60 tx=0x7f97d8015000).stop 2020-11-16T22:21:04.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 shutdown_connections 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97c0055060 0x7f97c0057520 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97f0142f30 0x7f97f0157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97f0067910 0x7f97f014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 --2- 172.21.15.185:0/3934780813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97f00b7610 0x7f97f0156e30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 >> 172.21.15.185:0/3934780813 conn(0x7f97f00637b0 msgr2=0x7f97f01472d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 shutdown_connections 2020-11-16T22:21:04.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.310+0000 7f97effff700 1 -- 172.21.15.185:0/3934780813 wait complete. 2020-11-16T22:21:04.304 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 75 2020-11-16T22:21:04.324 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 Processor -- start 2020-11-16T22:21:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 -- start start 2020-11-16T22:21:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc414099d80 0x7fc4140b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc414068060 con 0x7fc4140b4140 2020-11-16T22:21:04.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc414067260 con 0x7fc4140b2980 2020-11-16T22:21:04.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc41b595700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140b2e30 con 0x7fc414099d80 2020-11-16T22:21:04.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc418b30700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc414099d80 0x7fc4140b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc419b32700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc4140b2980 0x7fc4141445b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55412/0 (socket says 172.21.15.185:55412) 2020-11-16T22:21:04.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.450+0000 7fc419b32700 1 -- 172.21.15.185:0/880105894 learned_addr learned my addr 172.21.15.185:0/880105894 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:04.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc403fff700 1 -- 172.21.15.185:0/880105894 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784359058 0 0) 0x7fc414067260 con 0x7fc4140b2980 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc403fff700 1 -- 172.21.15.185:0/880105894 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3fc003240 con 0x7fc4140b2980 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc418b30700 1 -- 172.21.15.185:0/880105894 >> v1:172.21.15.185:6789/0 conn(0x7fc4140b2980 legacy=0x7fc4141445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc418b30700 1 -- 172.21.15.185:0/880105894 >> v1:172.21.15.96:6789/0 conn(0x7fc4140b4140 legacy=0x7fc4140b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc418b30700 1 -- 172.21.15.185:0/880105894 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414067ce0 con 0x7fc414099d80 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc418b30700 1 --2- 172.21.15.185:0/880105894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc414099d80 0x7fc4140b4520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc40400b9a0 tx=0x7fc404006df0).ready entity=mon.1 client_cookie=9e988d0568450250 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc403fff700 1 -- 172.21.15.185:0/880105894 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040097b0 con 0x7fc414099d80 2020-11-16T22:21:04.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc403fff700 1 -- 172.21.15.185:0/880105894 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404009910 con 0x7fc414099d80 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc403fff700 1 -- 172.21.15.185:0/880105894 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404009c90 con 0x7fc414099d80 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- 172.21.15.185:0/880105894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc414099d80 msgr2=0x7fc4140b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 --2- 172.21.15.185:0/880105894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc414099d80 0x7fc4140b4520 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc40400b9a0 tx=0x7fc404006df0).stop 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- 172.21.15.185:0/880105894 shutdown_connections 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 --2- 172.21.15.185:0/880105894 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc414099d80 0x7fc4140b4520 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- 172.21.15.185:0/880105894 >> 172.21.15.185:0/880105894 conn(0x7fc414061240 msgr2=0x7fc414061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:04.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- 172.21.15.185:0/880105894 shutdown_connections 2020-11-16T22:21:04.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- 172.21.15.185:0/880105894 wait complete. 2020-11-16T22:21:04.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 Processor -- start 2020-11-16T22:21:04.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.454+0000 7fc41b595700 1 -- start start 2020-11-16T22:21:04.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc414099d80 0x7fc414148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4140b2980 0x7fc414148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414149350 con 0x7fc4140b2980 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140af900 con 0x7fc4140b4140 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140af4f0 con 0x7fc414099d80 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419331700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc414099d80 0x7fc414148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33746/0 (socket says 172.21.15.185:33746) 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 -- 172.21.15.185:0/2293482102 learned_addr learned my addr 172.21.15.185:0/2293482102 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc418b30700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4140b2980 0x7fc414148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 -- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc414099d80 msgr2=0x7fc414148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc414099d80 0x7fc414148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 -- 172.21.15.185:0/2293482102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4140b2980 msgr2=0x7fc414148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4140b2980 0x7fc414148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 -- 172.21.15.185:0/2293482102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140afb30 con 0x7fc4140b4140 2020-11-16T22:21:04.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc419b32700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fc40c00cac0 tx=0x7fc40c007d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c005b80 con 0x7fc4140b4140 2020-11-16T22:21:04.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c005ce0 con 0x7fc4140b4140 2020-11-16T22:21:04.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007320 con 0x7fc4140b4140 2020-11-16T22:21:04.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140afdd0 con 0x7fc4140b4140 2020-11-16T22:21:04.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4140ae8d0 con 0x7fc4140b4140 2020-11-16T22:21:04.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41405d4e0 con 0x7fc4140b4140 2020-11-16T22:21:04.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.458+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc40c010030 con 0x7fc4140b4140 2020-11-16T22:21:04.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.462+0000 7fc4017fa700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 0x7fc3ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.462+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc40c06dce0 con 0x7fc4140b4140 2020-11-16T22:21:04.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.462+0000 7fc419331700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 0x7fc3ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.462+0000 7fc419331700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 0x7fc3ec057520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc410003300 tx=0x7fc410007b00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.466+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc40c043f40 con 0x7fc4140b4140 2020-11-16T22:21:04.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.614+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4140b0270 con 0x7fc4140b4140 2020-11-16T22:21:04.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.614+0000 7fc4017fa700 1 -- 172.21.15.185:0/2293482102 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16829 (secure 0 0 0) 0x7fc40c047560 con 0x7fc4140b4140 2020-11-16T22:21:04.607 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:04.608 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":65,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:18:51.735891+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:18:48.576973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2020-11-16T22:18:49.752256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:04.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 msgr2=0x7fc3ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 0x7fc3ec057520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fc410003300 tx=0x7fc410007b00).stop 2020-11-16T22:21:04.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 msgr2=0x7fc4140ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fc40c00cac0 tx=0x7fc40c007d60).stop 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 shutdown_connections 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc3ec055060 0x7fc3ec057520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc4140b4140 0x7fc4140ad310 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc414099d80 0x7fc414148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 --2- 172.21.15.185:0/2293482102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4140b2980 0x7fc414148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 >> 172.21.15.185:0/2293482102 conn(0x7fc414061240 msgr2=0x7fc41409aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:04.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 shutdown_connections 2020-11-16T22:21:04.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.618+0000 7fc41b595700 1 -- 172.21.15.185:0/2293482102 wait complete. 2020-11-16T22:21:04.622 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:21:04.623 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:21:04.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 Processor -- start 2020-11-16T22:21:04.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- start start 2020-11-16T22:21:04.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad140b17e0 0x7fad14067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad14098cc0 con 0x7fad140b2160 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1414c4f0 con 0x7fad140b17e0 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad140b3950 con 0x7fad14066240 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad140b17e0 0x7fad14067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad19f8b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fad140b17e0 0x7fad14067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33754/0 (socket says 172.21.15.185:33754) 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad19f8b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55432/0 (socket says 172.21.15.185:55432) 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 -- 172.21.15.185:0/376018048 learned_addr learned my addr 172.21.15.185:0/376018048 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 -- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 msgr2=0x7fad14065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 -- 172.21.15.185:0/376018048 >> v1:172.21.15.96:6789/0 conn(0x7fad140b2160 legacy=0x7fad1414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 -- 172.21.15.185:0/376018048 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140b3bf0 con 0x7fad140b17e0 2020-11-16T22:21:04.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad19f8b700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:04.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1978a700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b17e0 0x7fad14067fb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fad10006fd0 tx=0x7fad10006eb0).ready entity=mon.1 client_cookie=7e8ec1d8a277ddca server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad0bfff700 1 -- 172.21.15.185:0/376018048 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10009810 con 0x7fad140b17e0 2020-11-16T22:21:04.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad0bfff700 1 -- 172.21.15.185:0/376018048 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10009970 con 0x7fad140b17e0 2020-11-16T22:21:04.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad0bfff700 1 -- 172.21.15.185:0/376018048 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10009cf0 con 0x7fad140b17e0 2020-11-16T22:21:04.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b17e0 msgr2=0x7fad14067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b17e0 0x7fad14067fb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fad10006fd0 tx=0x7fad10006eb0).stop 2020-11-16T22:21:04.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- 172.21.15.185:0/376018048 shutdown_connections 2020-11-16T22:21:04.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad14066240 0x7fad14065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/376018048 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b17e0 0x7fad14067fb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- 172.21.15.185:0/376018048 >> 172.21.15.185:0/376018048 conn(0x7fad140604a0 msgr2=0x7fad140608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- 172.21.15.185:0/376018048 shutdown_connections 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.754+0000 7fad1b9ee700 1 -- 172.21.15.185:0/376018048 wait complete. 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 Processor -- start 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- start start 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad14066240 0x7fad14157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.752 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad140b17e0 0x7fad1408f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad141580f0 con 0x7fad14066240 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1409c5c0 con 0x7fad140b2160 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1408f8e0 con 0x7fad140b17e0 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33760/0 (socket says 172.21.15.185:33760) 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 -- 172.21.15.185:0/1715789668 learned_addr learned my addr 172.21.15.185:0/1715789668 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:04.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad18f89700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad140b17e0 0x7fad1408f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1978a700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad14066240 0x7fad14157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 -- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad140b17e0 msgr2=0x7fad1408f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad140b17e0 0x7fad1408f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 -- 172.21.15.185:0/1715789668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad14066240 msgr2=0x7fad14157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad14066240 0x7fad14157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 -- 172.21.15.185:0/1715789668 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1415b310 con 0x7fad140b2160 2020-11-16T22:21:04.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad19f8b700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fad0c00ef50 tx=0x7fad0c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00cd10 con 0x7fad140b2160 2020-11-16T22:21:04.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c00ce70 con 0x7fad140b2160 2020-11-16T22:21:04.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1415b5b0 con 0x7fad140b2160 2020-11-16T22:21:04.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c012620 con 0x7fad140b2160 2020-11-16T22:21:04.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1415bf40 con 0x7fad140b2160 2020-11-16T22:21:04.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.758+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1415b770 con 0x7fad140b2160 2020-11-16T22:21:04.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.762+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fad0c022070 con 0x7fad140b2160 2020-11-16T22:21:04.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.762+0000 7fad097fa700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 0x7facec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:04.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.762+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad0c06e730 con 0x7fad140b2160 2020-11-16T22:21:04.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.762+0000 7fad1978a700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 0x7facec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:04.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.762+0000 7fad1978a700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 0x7facec057520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fad1000beb0 tx=0x7fad10015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:04.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.766+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fad0c044990 con 0x7fad140b2160 2020-11-16T22:21:04.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:04.966+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad1405d4e0 con 0x7fad140b2160 2020-11-16T22:21:05.083 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:21:05.094+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:21:05.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad097fa700 1 -- 172.21.15.185:0/1715789668 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad0c047fb0 con 0x7fad140b2160 2020-11-16T22:21:05.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 msgr2=0x7facec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 0x7facec057520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fad1000beb0 tx=0x7fad10015000).stop 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 msgr2=0x7fad1415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fad0c00ef50 tx=0x7fad0c0091d0).stop 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 shutdown_connections 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7facec055060 0x7facec057520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fad140b2160 0x7fad1415ba50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad14066240 0x7fad14157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 --2- 172.21.15.185:0/1715789668 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fad140b17e0 0x7fad1408f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.106+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 >> 172.21.15.185:0/1715789668 conn(0x7fad140604a0 msgr2=0x7fad14098180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:05.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.110+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 shutdown_connections 2020-11-16T22:21:05.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.110+0000 7fad1b9ee700 1 -- 172.21.15.185:0/1715789668 wait complete. 2020-11-16T22:21:05.103 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:21:05.105 INFO:tasks.ceph.mds.c.smithi096.stderr: -8870> 2020-11-16T22:19:01.373+0000 7f5b056f8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:21:05.106 INFO:tasks.ceph.mds.c.smithi096.stderr: -8868> 2020-11-16T22:19:01.373+0000 7f5b056f8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:21:05.119 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:21:05.126 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:21:05.128 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:21:05.136 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:21:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 Processor -- start 2020-11-16T22:21:05.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 -- start start 2020-11-16T22:21:05.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 0x7fe82013f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:05.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe820060970 con 0x7fe820060130 2020-11-16T22:21:05.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe820060e40 con 0x7fe8200b3830 2020-11-16T22:21:05.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe825196700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b4610 con 0x7fe8200ac820 2020-11-16T22:21:05.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe81ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe8200b3830 0x7fe820143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55448/0 (socket says 172.21.15.185:55448) 2020-11-16T22:21:05.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe81ffff700 1 -- 172.21.15.185:0/3823013452 learned_addr learned my addr 172.21.15.185:0/3823013452 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:05.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.250+0000 7fe81f7fe700 1 --2- 172.21.15.185:0/3823013452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 0x7fe82013f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:05.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685116145 0 0) 0x7fe820060e40 con 0x7fe8200b3830 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe80c003240 con 0x7fe8200b3830 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566580574 0 0) 0x7fe820060970 con 0x7fe820060130 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe820060e40 con 0x7fe820060130 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3823013452 >> v1:172.21.15.185:6789/0 conn(0x7fe8200b3830 legacy=0x7fe820143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3823013452 >> v1:172.21.15.96:6789/0 conn(0x7fe820060130 legacy=0x7fe82013fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3823013452 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820060ba0 con 0x7fe8200ac820 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81f7fe700 1 --2- 172.21.15.185:0/3823013452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 0x7fe82013f840 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fe81400b9d0 tx=0x7fe814006df0).ready entity=mon.2 client_cookie=816977a7e0b99d34 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814014070 con 0x7fe8200ac820 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814009710 con 0x7fe8200ac820 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe81effd700 1 -- 172.21.15.185:0/3823013452 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814009a90 con 0x7fe8200ac820 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 -- 172.21.15.185:0/3823013452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 msgr2=0x7fe82013f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 --2- 172.21.15.185:0/3823013452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 0x7fe82013f840 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fe81400b9d0 tx=0x7fe814006df0).stop 2020-11-16T22:21:05.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 -- 172.21.15.185:0/3823013452 shutdown_connections 2020-11-16T22:21:05.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 --2- 172.21.15.185:0/3823013452 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe8200ac820 0x7fe82013f840 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 -- 172.21.15.185:0/3823013452 >> 172.21.15.185:0/3823013452 conn(0x7fe820063df0 msgr2=0x7fe8200641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:05.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 -- 172.21.15.185:0/3823013452 shutdown_connections 2020-11-16T22:21:05.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.254+0000 7fe825196700 1 -- 172.21.15.185:0/3823013452 wait complete. 2020-11-16T22:21:05.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 Processor -- start 2020-11-16T22:21:05.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 -- start start 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe820060130 0x7fe82014fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8200b3830 0x7fe82013a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8201501d0 con 0x7fe8200b3830 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200aeee0 con 0x7fe8200ac820 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe825196700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200ae9e0 con 0x7fe820060130 2020-11-16T22:21:05.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:05.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33786/0 (socket says 172.21.15.185:33786) 2020-11-16T22:21:05.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3909121102 learned_addr learned my addr 172.21.15.185:0/3909121102 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:05.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81ffff700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe820060130 0x7fe82014fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:05.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe824995700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8200b3830 0x7fe82013a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:05.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe820060130 msgr2=0x7fe82014fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe820060130 0x7fe82014fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3909121102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8200b3830 msgr2=0x7fe82013a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8200b3830 0x7fe82013a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 -- 172.21.15.185:0/3909121102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200aec40 con 0x7fe8200ac820 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.258+0000 7fe81f7fe700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fe81400b150 tx=0x7fe814016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81401f070 con 0x7fe8200ac820 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814024410 con 0x7fe8200ac820 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814024790 con 0x7fe8200ac820 2020-11-16T22:21:05.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8200af6e0 con 0x7fe8200ac820 2020-11-16T22:21:05.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8200accb0 con 0x7fe8200ac820 2020-11-16T22:21:05.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8200af870 con 0x7fe8200ac820 2020-11-16T22:21:05.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe81402b070 con 0x7fe8200ac820 2020-11-16T22:21:05.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 0x7fe7f8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:05.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fe814070d00 con 0x7fe8200ac820 2020-11-16T22:21:05.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.262+0000 7fe81ffff700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 0x7fe7f8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:05.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.266+0000 7fe81ffff700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 0x7fe7f8057520 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe8201104b0 tx=0x7fe810009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:05.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.270+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fe814074030 con 0x7fe8200ac820 2020-11-16T22:21:05.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:05.470+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe82005d200 con 0x7fe8200ac820 2020-11-16T22:21:06.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.114+0000 7fe7fffff700 1 -- 172.21.15.185:0/3909121102 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe814046e60 con 0x7fe8200ac820 2020-11-16T22:21:06.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 msgr2=0x7fe7f8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 0x7fe7f8057520 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fe8201104b0 tx=0x7fe810009040).stop 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 msgr2=0x7fe82013dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fe81400b150 tx=0x7fe814016000).stop 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 shutdown_connections 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe7f8055060 0x7fe7f8057520 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8200b3830 0x7fe82013a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe820060130 0x7fe82014fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 --2- 172.21.15.185:0/3909121102 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe8200ac820 0x7fe82013dde0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 >> 172.21.15.185:0/3909121102 conn(0x7fe820063df0 msgr2=0x7fe8200b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:06.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 shutdown_connections 2020-11-16T22:21:06.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.118+0000 7fe825196700 1 -- 172.21.15.185:0/3909121102 wait complete. 2020-11-16T22:21:06.127 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:21:06.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 Processor -- start 2020-11-16T22:21:06.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 -- start start 2020-11-16T22:21:06.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3c180ac0b0 0x7f3c1813d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:06.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c180ac830 con 0x7f3c18068220 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c180b1810 con 0x7f3c180b7610 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c1c8da700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c180b1ab0 con 0x7f3c180ac0b0 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c177fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3c180b7610 0x7f3c180b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55462/0 (socket says 172.21.15.185:55462) 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c177fe700 1 -- 172.21.15.185:0/1123039901 learned_addr learned my addr 172.21.15.185:0/1123039901 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c17fff700 1 --2- 172.21.15.185:0/1123039901 >> v2:172.21.15.96:3300/0 conn(0x7f3c180ac0b0 0x7f3c1813d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c167fc700 1 -- 172.21.15.185:0/1123039901 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336078369 0 0) 0x7f3c180b1810 con 0x7f3c180b7610 2020-11-16T22:21:06.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.314+0000 7f3c167fc700 1 -- 172.21.15.185:0/1123039901 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c04003240 con 0x7f3c180b7610 2020-11-16T22:21:06.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1123039901 >> v1:172.21.15.185:6789/0 conn(0x7f3c180b7610 legacy=0x7f3c180b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1123039901 >> v1:172.21.15.96:6789/0 conn(0x7f3c18068220 legacy=0x7f3c18143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1123039901 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180ac4d0 con 0x7f3c180ac0b0 2020-11-16T22:21:06.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- 172.21.15.185:0/1123039901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c180ac0b0 0x7f3c1813d220 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f3c10009c90 tx=0x7f3c100091d0).ready entity=mon.0 client_cookie=cf0ca0dc4ffaba1c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:06.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c167fc700 1 -- 172.21.15.185:0/1123039901 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000ba70 con 0x7f3c180ac0b0 2020-11-16T22:21:06.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c167fc700 1 -- 172.21.15.185:0/1123039901 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1000bbd0 con 0x7f3c180ac0b0 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c167fc700 1 -- 172.21.15.185:0/1123039901 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10011400 con 0x7f3c180ac0b0 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1123039901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c180ac0b0 msgr2=0x7f3c1813d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1123039901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c180ac0b0 0x7f3c1813d220 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f3c10009c90 tx=0x7f3c100091d0).stop 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1123039901 shutdown_connections 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1123039901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c180ac0b0 0x7f3c1813d220 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1123039901 >> 172.21.15.185:0/1123039901 conn(0x7f3c18063780 msgr2=0x7f3c180b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:06.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1123039901 shutdown_connections 2020-11-16T22:21:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1123039901 wait complete. 2020-11-16T22:21:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 Processor -- start 2020-11-16T22:21:06.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- start start 2020-11-16T22:21:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c18068220 0x7f3c181530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c180ac0b0 0x7f3c18156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:06.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1813d070 con 0x7f3c18068220 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18113f70 con 0x7f3c180ac0b0 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c1c8da700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c18156800 con 0x7f3c180b7610 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55474/0 (socket says 172.21.15.185:55474) 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1549994813 learned_addr learned my addr 172.21.15.185:0/1549994813 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:06.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c16ffd700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c180ac0b0 0x7f3c18156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c177fe700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c18068220 0x7f3c181530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c180ac0b0 msgr2=0x7f3c18156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c180ac0b0 0x7f3c18156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1549994813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c18068220 msgr2=0x7f3c181530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c18068220 0x7f3c181530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 -- 172.21.15.185:0/1549994813 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18156aa0 con 0x7f3c180b7610 2020-11-16T22:21:06.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.318+0000 7f3c17fff700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f3c180b4860 tx=0x7f3c10016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10010700 con 0x7f3c180b7610 2020-11-16T22:21:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c181535a0 con 0x7f3c180b7610 2020-11-16T22:21:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10010860 con 0x7f3c180b7610 2020-11-16T22:21:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1813fd30 con 0x7f3c180b7610 2020-11-16T22:21:06.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10010be0 con 0x7f3c180b7610 2020-11-16T22:21:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3c10024030 con 0x7f3c180b7610 2020-11-16T22:21:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 0x7f3bf0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c18156c30 con 0x7f3c180b7610 2020-11-16T22:21:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f3c1006f9d0 con 0x7f3c180b7610 2020-11-16T22:21:06.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3c177fe700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 0x7f3bf0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:06.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.322+0000 7f3c177fe700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 0x7f3bf0057520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f3c0c0055a0 tx=0x7f3c0c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:06.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.334+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3c10015020 con 0x7f3c180b7610 2020-11-16T22:21:06.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:06.490+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3c1805d200 con 0x7f3c180b7610 2020-11-16T22:21:07.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.118+0000 7f3bf77fe700 1 -- 172.21.15.185:0/1549994813 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3c10022070 con 0x7f3c180b7610 2020-11-16T22:21:07.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 msgr2=0x7f3bf0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:07.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 0x7f3bf0057520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f3c0c0055a0 tx=0x7f3c0c009040).stop 2020-11-16T22:21:07.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 msgr2=0x7f3c1813f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f3c180b4860 tx=0x7f3c10016000).stop 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 shutdown_connections 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bf0055060 0x7f3bf0057520 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c180b7610 0x7f3c1813f840 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c18068220 0x7f3c181530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 --2- 172.21.15.185:0/1549994813 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c180ac0b0 0x7f3c18156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 >> 172.21.15.185:0/1549994813 conn(0x7f3c18063780 msgr2=0x7f3c180b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:07.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 shutdown_connections 2020-11-16T22:21:07.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.122+0000 7f3c1c8da700 1 -- 172.21.15.185:0/1549994813 wait complete. 2020-11-16T22:21:07.117 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:21:07.129 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:21:07.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.258+0000 7f1201f86700 1 Processor -- start 2020-11-16T22:21:07.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.258+0000 7f1201f86700 1 -- start start 2020-11-16T22:21:07.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 0x7f11fc0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 0x7f11fc144570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0679b0 con 0x7f11fc0b7610 2020-11-16T22:21:07.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc067c50 con 0x7f11fc144a60 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc067ef0 con 0x7f11fc1440f0 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201785700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 0x7f11fc144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201785700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33810/0 (socket says 172.21.15.185:33810) 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201785700 1 -- 172.21.15.185:0/370701847 learned_addr learned my addr 172.21.15.185:0/370701847 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1200f84700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 0x7f11fc0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 -- 172.21.15.185:0/370701847 >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 msgr2=0x7f11fc147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 -- 172.21.15.185:0/370701847 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 msgr2=0x7f11fc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 0x7f11fc0b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 -- 172.21.15.185:0/370701847 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc068190 con 0x7f11fc1440f0 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201785700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1200f84700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 0x7f11fc0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:07.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fbfff700 1 --2- 172.21.15.185:0/370701847 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 0x7f11fc144570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f11fc061260 tx=0x7f11e8006df0).ready entity=mon.2 client_cookie=241d37d56404cae9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:07.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fb7fe700 1 -- 172.21.15.185:0/370701847 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80097b0 con 0x7f11fc1440f0 2020-11-16T22:21:07.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fb7fe700 1 -- 172.21.15.185:0/370701847 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8009910 con 0x7f11fc1440f0 2020-11-16T22:21:07.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f11fb7fe700 1 -- 172.21.15.185:0/370701847 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8009c90 con 0x7f11fc1440f0 2020-11-16T22:21:07.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- 172.21.15.185:0/370701847 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 msgr2=0x7f11fc144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:07.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- 172.21.15.185:0/370701847 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 0x7f11fc144570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f11fc061260 tx=0x7f11e8006df0).stop 2020-11-16T22:21:07.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- 172.21.15.185:0/370701847 shutdown_connections 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.185:3300/0 conn(0x7f11fc144a60 0x7f11fc147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- 172.21.15.185:0/370701847 >> v2:172.21.15.96:3300/0 conn(0x7f11fc0b7610 0x7f11fc0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 --2- 172.21.15.185:0/370701847 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc1440f0 0x7f11fc144570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- 172.21.15.185:0/370701847 >> 172.21.15.185:0/370701847 conn(0x7f11fc063780 msgr2=0x7f11fc0b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- 172.21.15.185:0/370701847 shutdown_connections 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- 172.21.15.185:0/370701847 wait complete. 2020-11-16T22:21:07.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 Processor -- start 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.262+0000 7f1201f86700 1 -- start start 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 0x7f11fc0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc144a60 0x7f11fc153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc147560 con 0x7f11fc144a60 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc147790 con 0x7f11fc1440f0 2020-11-16T22:21:07.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc147a30 con 0x7f11fc0b7610 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 0x7f11fc0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55488/0 (socket says 172.21.15.185:55488) 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 -- 172.21.15.185:0/1589741373 learned_addr learned my addr 172.21.15.185:0/1589741373 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201785700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc144a60 0x7f11fc153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 -- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 msgr2=0x7f11fc0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:07.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 0x7f11fc0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 -- 172.21.15.185:0/1589741373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc144a60 msgr2=0x7f11fc153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc144a60 0x7f11fc153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 -- 172.21.15.185:0/1589741373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc147cd0 con 0x7f11fc0b7610 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11fbfff700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 0x7f11fc0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1200f84700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f11f000db90 tx=0x7f11f0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0009790 con 0x7f11fc0b7610 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00098f0 con 0x7f11fc0b7610 2020-11-16T22:21:07.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0009c90 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc153490 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0aefd0 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc153620 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f11f0012630 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 0x7f11d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f11f006d8c0 con 0x7f11fc0b7610 2020-11-16T22:21:07.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11fbfff700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 0x7f11d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:07.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.266+0000 7f11fbfff700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 0x7f11d4057520 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f11e8006e30 tx=0x7f11e8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:07.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.270+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f11f0043bd0 con 0x7f11fc0b7610 2020-11-16T22:21:07.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:07.422+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11fc05d200 con 0x7f11fc0b7610 2020-11-16T22:21:07.599 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:07.605 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:07.607 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:08.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.130+0000 7f11f8ff9700 1 -- 172.21.15.185:0/1589741373 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f11f00471f0 con 0x7f11fc0b7610 2020-11-16T22:21:08.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 msgr2=0x7f11d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 0x7f11d4057520 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f11e8006e30 tx=0x7f11e8015000).stop 2020-11-16T22:21:08.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 msgr2=0x7f11fc1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f11f000db90 tx=0x7f11f0008040).stop 2020-11-16T22:21:08.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 shutdown_connections 2020-11-16T22:21:08.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f11d4055060 0x7f11d4057520 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11fc144a60 0x7f11fc153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11fc0b7610 0x7f11fc1572e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 --2- 172.21.15.185:0/1589741373 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11fc1440f0 0x7f11fc0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 >> 172.21.15.185:0/1589741373 conn(0x7f11fc063780 msgr2=0x7f11fc145340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 shutdown_connections 2020-11-16T22:21:08.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.134+0000 7f1201f86700 1 -- 172.21.15.185:0/1589741373 wait complete. 2020-11-16T22:21:08.128 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:21:08.142 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:21:08.142 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:21:08.142 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:21:08.143 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:21:08.143 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:21:08.143 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:21:08.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:21:08.143 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:21:08.144 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:21:08.144 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:21:08.144 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:21:08.144 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:21:08.155 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:21:08.156 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:21:08.157 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:21:08.158 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:21:08.158 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:21:08.158 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:21:08.202 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:21:08.202 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:21:08.203 DEBUG:teuthology.parallel:result is None 2020-11-16T22:21:08.203 DEBUG:teuthology.parallel:result is None 2020-11-16T22:21:08.204 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:21:08.204 DEBUG:teuthology.parallel:result is None 2020-11-16T22:21:08.205 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:21:08.278 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:21:08.279 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:21:08.295 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:21:08.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.342+0000 7fd4f740b700 1 Processor -- start 2020-11-16T22:21:08.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.342+0000 7fd4f740b700 1 -- start start 2020-11-16T22:21:08.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd4f00abef0 0x7fd4f00ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd4f00accc0 0x7fd4f01454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00b4bb0 con 0x7fd4f00b53b0 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f0145a40 con 0x7fd4f00abef0 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f0145c40 con 0x7fd4f00accc0 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd4f00accc0 0x7fd4f01454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f59a8700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd4f00b53b0 0x7fd4f01422b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55510/0 (socket says 172.21.15.185:55510) 2020-11-16T22:21:08.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f59a8700 1 -- 172.21.15.185:0/2693448993 learned_addr learned my addr 172.21.15.185:0/2693448993 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:08.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f51a7700 1 --2- 172.21.15.185:0/2693448993 >> v2:172.21.15.96:3300/0 conn(0x7fd4f00abef0 0x7fd4f00ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 -- 172.21.15.185:0/2693448993 >> v2:172.21.15.96:3300/0 conn(0x7fd4f00abef0 msgr2=0x7fd4f00ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 --2- 172.21.15.185:0/2693448993 >> v2:172.21.15.96:3300/0 conn(0x7fd4f00abef0 0x7fd4f00ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 -- 172.21.15.185:0/2693448993 >> v1:172.21.15.185:6789/0 conn(0x7fd4f00b53b0 legacy=0x7fd4f01422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 -- 172.21.15.185:0/2693448993 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0145ee0 con 0x7fd4f00accc0 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4effff700 1 -- 172.21.15.185:0/2693448993 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687833922 0 0) 0x7fd4f00b4bb0 con 0x7fd4f00b53b0 2020-11-16T22:21:08.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f49a6700 1 --2- 172.21.15.185:0/2693448993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00accc0 0x7fd4f01454c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd4e400b9a0 tx=0x7fd4e4006df0).ready entity=mon.1 client_cookie=d01d807641472bb8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4effff700 1 -- 172.21.15.185:0/2693448993 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009770 con 0x7fd4f00accc0 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4effff700 1 -- 172.21.15.185:0/2693448993 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e40098d0 con 0x7fd4f00accc0 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4effff700 1 -- 172.21.15.185:0/2693448993 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009c50 con 0x7fd4f00accc0 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- 172.21.15.185:0/2693448993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00accc0 msgr2=0x7fd4f01454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 --2- 172.21.15.185:0/2693448993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00accc0 0x7fd4f01454c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd4e400b9a0 tx=0x7fd4e4006df0).stop 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- 172.21.15.185:0/2693448993 shutdown_connections 2020-11-16T22:21:08.341 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 --2- 172.21.15.185:0/2693448993 >> v2:172.21.15.96:3300/0 conn(0x7fd4f00abef0 0x7fd4f00ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 --2- 172.21.15.185:0/2693448993 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00accc0 0x7fd4f01454c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- 172.21.15.185:0/2693448993 >> 172.21.15.185:0/2693448993 conn(0x7fd4f0060130 msgr2=0x7fd4f0060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:08.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- 172.21.15.185:0/2693448993 shutdown_connections 2020-11-16T22:21:08.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.346+0000 7fd4f740b700 1 -- 172.21.15.185:0/2693448993 wait complete. 2020-11-16T22:21:08.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 Processor -- start 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- start start 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4f00abef0 0x7fd4f013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00b53b0 0x7fd4f013e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0155090 con 0x7fd4f00abef0 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0155330 con 0x7fd4f00b53b0 2020-11-16T22:21:08.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0141550 con 0x7fd4f00accc0 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55520/0 (socket says 172.21.15.185:55520) 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 -- 172.21.15.185:0/224173523 learned_addr learned my addr 172.21.15.185:0/224173523 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f51a7700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4f00abef0 0x7fd4f013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f59a8700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00b53b0 0x7fd4f013e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 -- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00b53b0 msgr2=0x7fd4f013e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00b53b0 0x7fd4f013e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 -- 172.21.15.185:0/224173523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4f00abef0 msgr2=0x7fd4f013dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:08.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4f00abef0 0x7fd4f013dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:08.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 -- 172.21.15.185:0/224173523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f013ea00 con 0x7fd4f00accc0 2020-11-16T22:21:08.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f49a6700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fd4f0145d80 tx=0x7fd4e4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:08.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4011d50 con 0x7fd4f00accc0 2020-11-16T22:21:08.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e401e460 con 0x7fd4f00accc0 2020-11-16T22:21:08.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0111a30 con 0x7fd4f00accc0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e401e7e0 con 0x7fd4f00accc0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.350+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f00b20d0 con 0x7fd4f00accc0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd4e4025030 con 0x7fd4f00accc0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f013ca10 con 0x7fd4f00accc0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4ed7fa700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 0x7fd4c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4f51a7700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 0x7fd4c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:08.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5511+0+0 (secure 0 0 0) 0x7fd4e406f1d0 con 0x7fd4f00accc0 2020-11-16T22:21:08.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.354+0000 7fd4f51a7700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 0x7fd4c8057520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd4e00099f0 tx=0x7fd4e00091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:08.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.362+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd4f013ca10 con 0x7fd4f00accc0 2020-11-16T22:21:08.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:08.534+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd4f013ca10 con 0x7fd4f00accc0 2020-11-16T22:21:09.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.338+0000 7fd4ed7fa700 1 -- 172.21.15.185:0/224173523 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd4f013ca10 con 0x7fd4f00accc0 2020-11-16T22:21:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 msgr2=0x7fd4c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 0x7fd4c8057520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd4e00099f0 tx=0x7fd4e00091d0).stop 2020-11-16T22:21:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 msgr2=0x7fd4f013c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fd4f0145d80 tx=0x7fd4e4015000).stop 2020-11-16T22:21:09.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 shutdown_connections 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd4c8055060 0x7fd4c8057520 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd4f00b53b0 0x7fd4f013e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4f00abef0 0x7fd4f013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 --2- 172.21.15.185:0/224173523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd4f00accc0 0x7fd4f013c520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 >> 172.21.15.185:0/224173523 conn(0x7fd4f0060130 msgr2=0x7fd4f00ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 shutdown_connections 2020-11-16T22:21:09.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.342+0000 7fd4f740b700 1 -- 172.21.15.185:0/224173523 wait complete. 2020-11-16T22:21:09.338 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:21:09.351 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:21:09.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 Processor -- start 2020-11-16T22:21:09.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- start start 2020-11-16T22:21:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa4dc0b3090 0x7fa4dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4dc0ad110 0x7fa4dc1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc061fe0 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc0b2b60 con 0x7fa4dc0ad110 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc14c060 con 0x7fa4dc0b3090 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa4dc0b3090 0x7fa4dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e1af8700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa4dc0b1d30 0x7fa4dc14b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55526/0 (socket says 172.21.15.185:55526) 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa4dc0b3090 0x7fa4dc0b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33852/0 (socket says 172.21.15.185:33852) 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e1af8700 1 -- 172.21.15.185:0/3031079552 learned_addr learned my addr 172.21.15.185:0/3031079552 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e2afa700 1 --2- 172.21.15.185:0/3031079552 >> v2:172.21.15.96:3300/0 conn(0x7fa4dc0ad110 0x7fa4dc1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 -- 172.21.15.185:0/3031079552 >> v2:172.21.15.96:3300/0 conn(0x7fa4dc0ad110 msgr2=0x7fa4dc1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:09.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 --2- 172.21.15.185:0/3031079552 >> v2:172.21.15.96:3300/0 conn(0x7fa4dc0ad110 0x7fa4dc1477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 -- 172.21.15.185:0/3031079552 >> v1:172.21.15.185:6789/0 conn(0x7fa4dc0b1d30 legacy=0x7fa4dc14b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 -- 172.21.15.185:0/3031079552 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc0b0fe0 con 0x7fa4dc0b3090 2020-11-16T22:21:09.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e22f9700 1 --2- 172.21.15.185:0/3031079552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc0b1720 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fa4dc14c1a0 tx=0x7fa4d40091d0).ready entity=mon.1 client_cookie=74a92e154b7e12a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:09.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e12f7700 1 -- 172.21.15.185:0/3031079552 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400ba70 con 0x7fa4dc0b3090 2020-11-16T22:21:09.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e12f7700 1 -- 172.21.15.185:0/3031079552 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d400bbd0 con 0x7fa4dc0b3090 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e12f7700 1 -- 172.21.15.185:0/3031079552 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011400 con 0x7fa4dc0b3090 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- 172.21.15.185:0/3031079552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0b3090 msgr2=0x7fa4dc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/3031079552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc0b1720 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fa4dc14c1a0 tx=0x7fa4d40091d0).stop 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- 172.21.15.185:0/3031079552 shutdown_connections 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/3031079552 >> v2:172.21.15.96:3300/0 conn(0x7fa4dc0ad110 0x7fa4dc1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/3031079552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc0b1720 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- 172.21.15.185:0/3031079552 >> 172.21.15.185:0/3031079552 conn(0x7fa4dc060130 msgr2=0x7fa4dc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- 172.21.15.185:0/3031079552 shutdown_connections 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.486+0000 7fa4e32fb700 1 -- 172.21.15.185:0/3031079552 wait complete. 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 Processor -- start 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- start start 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc1116a0 con 0x7fa4dc0b3090 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc14e6a0 con 0x7fa4dc0ad110 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc14e940 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e22f9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55534/0 (socket says 172.21.15.185:55534) 2020-11-16T22:21:09.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e22f9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33860/0 (socket says 172.21.15.185:33860) 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 -- 172.21.15.185:0/2477459845 learned_addr learned my addr 172.21.15.185:0/2477459845 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e2afa700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 -- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 msgr2=0x7fa4dc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 -- 172.21.15.185:0/2477459845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc0b3090 msgr2=0x7fa4dc1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 -- 172.21.15.185:0/2477459845 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc14ebe0 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e22f9700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e1af8700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fa4d000c710 tx=0x7fa4d0007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00072e0 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d0007440 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc157c50 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0005800 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.490+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc157940 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc05d200 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa4d0010070 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4ce7fc700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 0x7fa4b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:09.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa4d006d160 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4e22f9700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 0x7fa4b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:09.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.494+0000 7fa4e22f9700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 0x7fa4b4057520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fa4d4009210 tx=0x7fa4d4016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:09.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.502+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa4d00435e0 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.702+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa4dc157de0 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.702+0000 7fa4ce7fc700 1 -- 172.21.15.185:0/2477459845 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4274 (secure 0 0 0) 0x7fa4d0046c00 con 0x7fa4dc0b1d30 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:21:09.697 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.698 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.699 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.700 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.701 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.702 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.703 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.704 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:21:09.705 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.706 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.707 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.708 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.709 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.710 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:21:09.711 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow r", 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.712 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:21:09.713 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:21:09.713 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:21:09.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:21:09.914 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:21:09.915 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.916 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.917 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:21:09.918 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:21:09.919 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.919 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:09.919 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:21:09.919 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:09.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 msgr2=0x7fa4b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 0x7fa4b4057520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fa4d4009210 tx=0x7fa4d4016000).stop 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 msgr2=0x7fa4dc14aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fa4d000c710 tx=0x7fa4d0007b80).stop 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 shutdown_connections 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4b4055060 0x7fa4b4057520 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4dc0ad110 0x7fa4dc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc0b3090 0x7fa4dc1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 --2- 172.21.15.185:0/2477459845 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4dc0b1d30 0x7fa4dc14aa60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:09.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 >> 172.21.15.185:0/2477459845 conn(0x7fa4dc060130 msgr2=0x7fa4dc1493c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:09.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 shutdown_connections 2020-11-16T22:21:09.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:09.706+0000 7fa4e32fb700 1 -- 172.21.15.185:0/2477459845 wait complete. 2020-11-16T22:21:09.923 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:10.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 Processor -- start 2020-11-16T22:21:10.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- start start 2020-11-16T22:21:10.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa1a4067910 0x7fa1a4067f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40b3020 con 0x7fa1a40b7610 2020-11-16T22:21:10.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40b3280 con 0x7fa1a41426e0 2020-11-16T22:21:10.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a40b3520 con 0x7fa1a4067910 2020-11-16T22:21:10.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1aa99e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa1a41426e0 0x7fa1a4140c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55540/0 (socket says 172.21.15.185:55540) 2020-11-16T22:21:10.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1aa19d700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa1a40b7610 0x7fa1a4141fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:41600/0 (socket says 172.21.15.185:41600) 2020-11-16T22:21:10.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1aa99e700 1 -- 172.21.15.185:0/2772807814 learned_addr learned my addr 172.21.15.185:0/2772807814 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:10.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a999c700 1 --2- 172.21.15.185:0/2772807814 >> v2:172.21.15.96:3300/0 conn(0x7fa1a4067910 0x7fa1a4067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894066761 0 0) 0x7fa1a40b3280 con 0x7fa1a41426e0 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa190003240 con 0x7fa1a41426e0 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438920694 0 0) 0x7fa1a40b3020 con 0x7fa1a40b7610 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a40b3280 con 0x7fa1a40b7610 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a999c700 1 -- 172.21.15.185:0/2772807814 >> v1:172.21.15.185:6789/0 conn(0x7fa1a41426e0 legacy=0x7fa1a4140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1509958096 0 0) 0x7fa190003240 con 0x7fa1a41426e0 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a999c700 1 -- 172.21.15.185:0/2772807814 >> v1:172.21.15.96:6789/0 conn(0x7fa1a40b7610 legacy=0x7fa1a4141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a999c700 1 -- 172.21.15.185:0/2772807814 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4061750 con 0x7fa1a4067910 2020-11-16T22:21:10.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a999c700 1 --2- 172.21.15.185:0/2772807814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a4067910 0x7fa1a4067f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa19400b920 tx=0x7fa194006df0).ready entity=mon.0 client_cookie=bfc87455837c74c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0014e0 con 0x7fa1a4067910 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194012460 con 0x7fa1a4067910 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1a919b700 1 -- 172.21.15.185:0/2772807814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1940127e0 con 0x7fa1a4067910 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2772807814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a4067910 msgr2=0x7fa1a4067f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2772807814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a4067910 0x7fa1a4067f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa19400b920 tx=0x7fa194006df0).stop 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2772807814 shutdown_connections 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2772807814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a4067910 0x7fa1a4067f40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2772807814 >> 172.21.15.185:0/2772807814 conn(0x7fa1a40637b0 msgr2=0x7fa1a40b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2772807814 shutdown_connections 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.058+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2772807814 wait complete. 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 Processor -- start 2020-11-16T22:21:10.261 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- start start 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a40b7610 0x7fa1a4156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1a41426e0 0x7fa1a4156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4140480 con 0x7fa1a40b7610 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a413d880 con 0x7fa1a4067910 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a413ffe0 con 0x7fa1a41426e0 2020-11-16T22:21:10.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33872/0 (socket says 172.21.15.185:33872) 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 -- 172.21.15.185:0/2515057951 learned_addr learned my addr 172.21.15.185:0/2515057951 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa99e700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1a41426e0 0x7fa1a4156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1a999c700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a40b7610 0x7fa1a4156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 -- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1a41426e0 msgr2=0x7fa1a4156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1a41426e0 0x7fa1a4156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 -- 172.21.15.185:0/2515057951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a40b7610 msgr2=0x7fa1a4156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a40b7610 0x7fa1a4156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 -- 172.21.15.185:0/2515057951 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a413ce80 con 0x7fa1a4067910 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1aa19d700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fa1a000ce90 tx=0x7fa1a0007d20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004a90 con 0x7fa1a4067910 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0004bf0 con 0x7fa1a4067910 2020-11-16T22:21:10.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a4157160 con 0x7fa1a4067910 2020-11-16T22:21:10.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0007170 con 0x7fa1a4067910 2020-11-16T22:21:10.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.062+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a413e650 con 0x7fa1a4067910 2020-11-16T22:21:10.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a41401a0 con 0x7fa1a4067910 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa1a0010070 con 0x7fa1a4067910 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa19a7fc700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 0x7fa17c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa1a006d8b0 con 0x7fa1a4067910 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa1a999c700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 0x7fa17c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.066+0000 7fa1a999c700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 0x7fa17c057520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fa19400b150 tx=0x7fa194014000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.070+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa1a0043ec0 con 0x7fa1a4067910 2020-11-16T22:21:10.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.230+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1a405d200 con 0x7fa1a4067910 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.230+0000 7fa19a7fc700 1 -- 172.21.15.185:0/2515057951 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+13676 (secure 0 0 0) 0x7fa1a00474e0 con 0x7fa1a4067910 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 msgr2=0x7fa17c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 0x7fa17c057520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fa19400b150 tx=0x7fa194014000).stop 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 msgr2=0x7fa1a413c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fa1a000ce90 tx=0x7fa1a0007d20).stop 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 shutdown_connections 2020-11-16T22:21:10.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa17c055060 0x7fa17c057520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1a4067910 0x7fa1a413c990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1a41426e0 0x7fa1a4156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 --2- 172.21.15.185:0/2515057951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a40b7610 0x7fa1a4156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 >> 172.21.15.185:0/2515057951 conn(0x7fa1a40637b0 msgr2=0x7fa1a40b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 shutdown_connections 2020-11-16T22:21:10.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.234+0000 7fa1ab19f700 1 -- 172.21.15.185:0/2515057951 wait complete. 2020-11-16T22:21:10.470 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:10.470 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":69,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:09.335823+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:10.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:21:10.471 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:21:10.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.618+0000 7fc166e98700 1 Processor -- start 2020-11-16T22:21:10.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.618+0000 7fc166e98700 1 -- start start 2020-11-16T22:21:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc166e98700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc1600ae580 0x7fc1600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc166e98700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1600ad490 con 0x7fc1600afc60 2020-11-16T22:21:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc166e98700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1600ad730 con 0x7fc1600b7440 2020-11-16T22:21:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc166e98700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1600ad9d0 con 0x7fc1600ae580 2020-11-16T22:21:10.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc164c34700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc1600b7440 0x7fc16013dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55552/0 (socket says 172.21.15.185:55552) 2020-11-16T22:21:10.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc164c34700 1 -- 172.21.15.185:0/1450059808 learned_addr learned my addr 172.21.15.185:0/1450059808 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:10.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc165435700 1 --2- 172.21.15.185:0/1450059808 >> v2:172.21.15.96:3300/0 conn(0x7fc1600ae580 0x7fc1600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750981142 0 0) 0x7fc1600ad730 con 0x7fc1600b7440 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc140003240 con 0x7fc1600b7440 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3450292191 0 0) 0x7fc140003240 con 0x7fc1600b7440 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 >> v2:172.21.15.96:3300/0 conn(0x7fc1600ae580 msgr2=0x7fc1600b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 --2- 172.21.15.185:0/1450059808 >> v2:172.21.15.96:3300/0 conn(0x7fc1600ae580 0x7fc1600b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 >> v1:172.21.15.96:6789/0 conn(0x7fc1600afc60 legacy=0x7fc1600ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16013e5b0 con 0x7fc1600b7440 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fc1500014e0 con 0x7fc1600b7440 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 >> v1:172.21.15.185:6789/0 conn(0x7fc1600b7440 legacy=0x7fc16013dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc140003730 0x7fc140003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 0x7fc140004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc140004e50 0x7fc14000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600ad9d0 con 0x7fc140003730 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140004260 con 0x7fc140004e50 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14000bc20 con 0x7fc1400044e0 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc165435700 1 --2- 172.21.15.185:0/1450059808 >> v2:172.21.15.96:3300/0 conn(0x7fc1600ae580 0x7fc1600b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 0x7fc140004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc165435700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc140004e50 0x7fc14000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc164c34700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc140003730 0x7fc140003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 -- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc140004e50 msgr2=0x7fc14000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc140004e50 0x7fc14000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 -- 172.21.15.185:0/1450059808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc140003730 msgr2=0x7fc140003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc140003730 0x7fc140003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 -- 172.21.15.185:0/1450059808 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140003240 con 0x7fc1400044e0 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc165435700 1 -- 172.21.15.185:0/1450059808 reap_dead start 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15ffff700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 0x7fc140004960 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fc1480017a0 tx=0x7fc148007b00).ready entity=mon.2 client_cookie=2ffd84c51a39e478 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148007020 con 0x7fc1400044e0 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148007180 con 0x7fc1400044e0 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.622+0000 7fc15f7fe700 1 -- 172.21.15.185:0/1450059808 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800c640 con 0x7fc1400044e0 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 msgr2=0x7fc140004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 0x7fc140004960 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fc1480017a0 tx=0x7fc148007b00).stop 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/1450059808 shutdown_connections 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 --2- 172.21.15.185:0/1450059808 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1400044e0 0x7fc140004960 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/1450059808 >> 172.21.15.185:0/1450059808 conn(0x7fc160067860 msgr2=0x7fc160067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:10.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/1450059808 shutdown_connections 2020-11-16T22:21:10.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/1450059808 wait complete. 2020-11-16T22:21:10.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 Processor -- start 2020-11-16T22:21:10.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- start start 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 0x7fc1600b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1600b3350 0x7fc160153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1601534a0 con 0x7fc1400044e0 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16014b040 con 0x7fc16014a900 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16014ada0 con 0x7fc1600b3350 2020-11-16T22:21:10.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33894/0 (socket says 172.21.15.185:33894) 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc165435700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1600b3350 0x7fc160153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 -- 172.21.15.185:0/197199532 learned_addr learned my addr 172.21.15.185:0/197199532 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc164c34700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 0x7fc1600b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 -- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1600b3350 msgr2=0x7fc160153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1600b3350 0x7fc160153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 -- 172.21.15.185:0/197199532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 msgr2=0x7fc1600b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 0x7fc1600b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 -- 172.21.15.185:0/197199532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160153700 con 0x7fc16014a900 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc164c34700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 0x7fc1600b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15ffff700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fc14000bd60 tx=0x7fc148015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148018030 con 0x7fc16014a900 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148006700 con 0x7fc16014a900 2020-11-16T22:21:10.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1600b2ef0 con 0x7fc16014a900 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148006aa0 con 0x7fc16014a900 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.626+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc160154090 con 0x7fc16014a900 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc148026070 con 0x7fc16014a900 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc15cff9700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 0x7fc13c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc14806e350 con 0x7fc16014a900 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc164c34700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 0x7fc13c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16005d4e0 con 0x7fc16014a900 2020-11-16T22:21:10.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.630+0000 7fc164c34700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 0x7fc13c057520 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fc16005fa60 tx=0x7fc150009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.634+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc14801a030 con 0x7fc16014a900 2020-11-16T22:21:10.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc160144fe0 con 0x7fc16014a900 2020-11-16T22:21:10.778 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc15cff9700 1 -- 172.21.15.185:0/197199532 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v69) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc1480447d0 con 0x7fc16014a900 2020-11-16T22:21:10.778 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:21:10.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 msgr2=0x7fc13c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 0x7fc13c057520 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fc16005fa60 tx=0x7fc150009040).stop 2020-11-16T22:21:10.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 msgr2=0x7fc1600b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fc14000bd60 tx=0x7fc148015000).stop 2020-11-16T22:21:10.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 shutdown_connections 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc13c055060 0x7fc13c057520 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc1600b3350 0x7fc160153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1400044e0 0x7fc1600b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 --2- 172.21.15.185:0/197199532 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc16014a900 0x7fc1600b2a00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 >> 172.21.15.185:0/197199532 conn(0x7fc160067860 msgr2=0x7fc1600b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 shutdown_connections 2020-11-16T22:21:10.781 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.786+0000 7fc166e98700 1 -- 172.21.15.185:0/197199532 wait complete. 2020-11-16T22:21:10.792 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:21:10.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 Processor -- start 2020-11-16T22:21:10.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 -- start start 2020-11-16T22:21:10.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3400144860 0x7f34000ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34000ad490 con 0x7f34000b7440 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000ad730 con 0x7f3400144860 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f3406132700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000ad9d0 con 0x7f34000afc60 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f33feffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.926+0000 7f33feffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55580/0 (socket says 172.21.15.185:55580) 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 -- 172.21.15.185:0/2307243957 learned_addr learned my addr 172.21.15.185:0/2307243957 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:10.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33fffff700 1 --2- 172.21.15.185:0/2307243957 >> v2:172.21.15.96:3300/0 conn(0x7f3400144860 0x7f34000ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 -- 172.21.15.185:0/2307243957 >> v2:172.21.15.96:3300/0 conn(0x7f3400144860 msgr2=0x7f34000ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 --2- 172.21.15.185:0/2307243957 >> v2:172.21.15.96:3300/0 conn(0x7f3400144860 0x7f34000ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33fe7fc700 1 -- 172.21.15.185:0/2307243957 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611900824 0 0) 0x7f34000ad490 con 0x7f34000b7440 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 -- 172.21.15.185:0/2307243957 >> v1:172.21.15.96:6789/0 conn(0x7f34000b7440 legacy=0x7f340013dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 -- 172.21.15.185:0/2307243957 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340013e5b0 con 0x7f34000afc60 2020-11-16T22:21:10.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33feffd700 1 --2- 172.21.15.185:0/2307243957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f33f4006fd0 tx=0x7f33f4006eb0).ready entity=mon.2 client_cookie=f11237b701ca8950 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33fe7fc700 1 -- 172.21.15.185:0/2307243957 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4009810 con 0x7f34000afc60 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33fe7fc700 1 -- 172.21.15.185:0/2307243957 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4009970 con 0x7f34000afc60 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f33fe7fc700 1 -- 172.21.15.185:0/2307243957 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4009cf0 con 0x7f34000afc60 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- 172.21.15.185:0/2307243957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 msgr2=0x7f34000b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 --2- 172.21.15.185:0/2307243957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f33f4006fd0 tx=0x7f33f4006eb0).stop 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- 172.21.15.185:0/2307243957 shutdown_connections 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 --2- 172.21.15.185:0/2307243957 >> v2:172.21.15.96:3300/0 conn(0x7f3400144860 0x7f34000ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 --2- 172.21.15.185:0/2307243957 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000afc60 0x7f34000b7820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- 172.21.15.185:0/2307243957 >> 172.21.15.185:0/2307243957 conn(0x7f3400067860 msgr2=0x7f3400067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:10.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- 172.21.15.185:0/2307243957 shutdown_connections 2020-11-16T22:21:10.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- 172.21.15.185:0/2307243957 wait complete. 2020-11-16T22:21:10.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 Processor -- start 2020-11-16T22:21:10.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.930+0000 7f3406132700 1 -- start start 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 0x7f34001530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3400144860 0x7f3400156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b1f30 con 0x7f34000afc60 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400156690 con 0x7f3400144860 2020-11-16T22:21:10.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b2160 con 0x7f34000b7440 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 0x7f34001530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33fffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3400144860 0x7f3400156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55586/0 (socket says 172.21.15.185:55586) 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 0x7f34001530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.185:38202/0 (socket says 172.21.15.185:38202) 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 -- 172.21.15.185:0/381067877 learned_addr learned my addr 172.21.15.185:0/381067877 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:10.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 -- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3400144860 msgr2=0x7f3400156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3400144860 0x7f3400156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 -- 172.21.15.185:0/381067877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 msgr2=0x7f34001530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 0x7f34001530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 -- 172.21.15.185:0/381067877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000b2400 con 0x7f34000b7440 2020-11-16T22:21:10.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33feffd700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f34000adb10 tx=0x7f33f4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4011c80 con 0x7f34000b7440 2020-11-16T22:21:10.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4011de0 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34001568c0 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f401e670 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.934+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34000b0960 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340005d4e0 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f33f4025030 con 0x7f34000b7440 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f33df7fe700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 0x7f33d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:10.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f33f406f160 con 0x7f34000b7440 2020-11-16T22:21:10.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f33ff7fe700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 0x7f33d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:10.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.938+0000 7f33ff7fe700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 0x7f33d8057520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f33f00011a0 tx=0x7f33f0007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:10.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:10.942+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f33f4045690 con 0x7f34000b7440 2020-11-16T22:21:11.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3400144cf0 con 0x7f34000b7440 2020-11-16T22:21:11.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f33df7fe700 1 -- 172.21.15.185:0/381067877 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33f4023020 con 0x7f34000b7440 2020-11-16T22:21:11.091 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:11.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 msgr2=0x7f33d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 0x7f33d8057520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f33f00011a0 tx=0x7f33f0007a00).stop 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 msgr2=0x7f34000b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.098+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f34000adb10 tx=0x7f33f4015000).stop 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 shutdown_connections 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f33d8055060 0x7f33d8057520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3400144860 0x7f3400156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34000afc60 0x7f34001530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 --2- 172.21.15.185:0/381067877 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f34000b7440 0x7f34000b5b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 >> 172.21.15.185:0/381067877 conn(0x7f3400067860 msgr2=0x7f34000b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:11.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 shutdown_connections 2020-11-16T22:21:11.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.102+0000 7f3406132700 1 -- 172.21.15.185:0/381067877 wait complete. 2020-11-16T22:21:11.106 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:21:11.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 Processor -- start 2020-11-16T22:21:11.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 -- start start 2020-11-16T22:21:11.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bb0141190 0x7f2bb0141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00b3af0 con 0x7f2bb00b2eb0 2020-11-16T22:21:11.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb0061e60 con 0x7f2bb0141190 2020-11-16T22:21:11.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb74eb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00b3d20 con 0x7f2bb00662c0 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55592/0 (socket says 172.21.15.185:55592) 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb64e9700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f2bb00b2eb0 0x7f2bb013a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55586/0 (socket says 172.21.15.185:55586) 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/2488827117 learned_addr learned my addr 172.21.15.185:0/2488827117 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb6cea700 1 --2- 172.21.15.185:0/2488827117 >> v2:172.21.15.96:3300/0 conn(0x7f2bb0141190 0x7f2bb0141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/2488827117 >> v2:172.21.15.96:3300/0 conn(0x7f2bb0141190 msgr2=0x7f2bb0141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:11.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 --2- 172.21.15.185:0/2488827117 >> v2:172.21.15.96:3300/0 conn(0x7f2bb0141190 0x7f2bb0141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/2488827117 >> v1:172.21.15.185:6789/0 conn(0x7f2bb00b2eb0 legacy=0x7f2bb013a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb54e7700 1 -- 172.21.15.185:0/2488827117 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204055068 0 0) 0x7f2bb00b3af0 con 0x7f2bb00b2eb0 2020-11-16T22:21:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/2488827117 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0140cc0 con 0x7f2bb00662c0 2020-11-16T22:21:11.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb5ce8700 1 --2- 172.21.15.185:0/2488827117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f2ba0006fd0 tx=0x7f2ba0006f40).ready entity=mon.2 client_cookie=bfb72011205b0b67 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.230+0000 7f2bb54e7700 1 -- 172.21.15.185:0/2488827117 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000dc60 con 0x7f2bb00662c0 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb54e7700 1 -- 172.21.15.185:0/2488827117 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000ddc0 con 0x7f2bb00662c0 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb54e7700 1 -- 172.21.15.185:0/2488827117 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0010680 con 0x7f2bb00662c0 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/2488827117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 msgr2=0x7f2bb00b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/2488827117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f2ba0006fd0 tx=0x7f2ba0006f40).stop 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/2488827117 shutdown_connections 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/2488827117 >> v2:172.21.15.96:3300/0 conn(0x7f2bb0141190 0x7f2bb0141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/2488827117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00662c0 0x7f2bb00b3290 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0x7f2ba0006fd0 tx=0x7f2ba0006f40).stop 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/2488827117 >> 172.21.15.185:0/2488827117 conn(0x7f2bb0060130 msgr2=0x7f2bb0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/2488827117 shutdown_connections 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/2488827117 wait complete. 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 Processor -- start 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- start start 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb00b2eb0 0x7f2bb00afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00ae410 0x7f2bb0156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0156b20 con 0x7f2bb00b2eb0 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00ad970 con 0x7f2bb0141190 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00adc10 con 0x7f2bb00ae410 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33924/0 (socket says 172.21.15.185:33924) 2020-11-16T22:21:11.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/1157140443 learned_addr learned my addr 172.21.15.185:0/1157140443 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb6cea700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00ae410 0x7f2bb0156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb64e9700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb00b2eb0 0x7f2bb00afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00ae410 msgr2=0x7f2bb0156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00ae410 0x7f2bb0156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/1157140443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb00b2eb0 msgr2=0x7f2bb00afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb00b2eb0 0x7f2bb00afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 -- 172.21.15.185:0/1157140443 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00adeb0 con 0x7f2bb0141190 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb5ce8700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2bb00b2e80 tx=0x7f2ba0006f40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba001d070 con 0x7f2bb0141190 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0018610 con 0x7f2bb0141190 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb0143ae0 con 0x7f2bb0141190 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00189b0 con 0x7f2bb0141190 2020-11-16T22:21:11.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.234+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb0153500 con 0x7f2bb0141190 2020-11-16T22:21:11.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb005d200 con 0x7f2bb0141190 2020-11-16T22:21:11.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2ba0029070 con 0x7f2bb0141190 2020-11-16T22:21:11.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2b9e7fc700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 0x7f2b88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2ba006ed90 con 0x7f2bb0141190 2020-11-16T22:21:11.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2bb64e9700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 0x7f2b88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.238+0000 7f2bb64e9700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 0x7f2b88057520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f2bac0011a0 tx=0x7f2bac007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.246+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2ba0045320 con 0x7f2bb0141190 2020-11-16T22:21:11.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.394+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2bb0141b10 con 0x7f2bb0141190 2020-11-16T22:21:11.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2b9e7fc700 1 -- 172.21.15.185:0/1157140443 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2ba0048940 con 0x7f2bb0141190 2020-11-16T22:21:11.389 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:11.389 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:21:11.389 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:11.390 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:21:11.390 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:21:11.390 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:21:11.390 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:21:11.390 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:21:11.391 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:21:11.391 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:11.391 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:21:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 msgr2=0x7f2b88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 0x7f2b88057520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f2bac0011a0 tx=0x7f2bac007a20).stop 2020-11-16T22:21:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 msgr2=0x7f2bb00ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2bb00b2e80 tx=0x7f2ba0006f40).stop 2020-11-16T22:21:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 shutdown_connections 2020-11-16T22:21:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2b88055060 0x7f2b88057520 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2bb00ae410 0x7f2bb0156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bb00b2eb0 0x7f2bb00afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 --2- 172.21.15.185:0/1157140443 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2bb0141190 0x7f2bb00ac220 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 >> 172.21.15.185:0/1157140443 conn(0x7f2bb0060130 msgr2=0x7f2bb0137bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:11.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 shutdown_connections 2020-11-16T22:21:11.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.398+0000 7f2bb74eb700 1 -- 172.21.15.185:0/1157140443 wait complete. 2020-11-16T22:21:11.411 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:21:11.411 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:21:11.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 Processor -- start 2020-11-16T22:21:11.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 -- start start 2020-11-16T22:21:11.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9de013e1f0 0x7f9de0143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 0x7f9de00666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00b3a00 con 0x7f9de013de10 2020-11-16T22:21:11.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de00b3c80 con 0x7f9de013e1f0 2020-11-16T22:21:11.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9de499d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00b3f20 con 0x7f9de00b3570 2020-11-16T22:21:11.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 0x7f9de00666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddf7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f9de013de10 0x7f9de00b4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55600/0 (socket says 172.21.15.185:55600) 2020-11-16T22:21:11.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddf7fe700 1 -- 172.21.15.185:0/3011622830 learned_addr learned my addr 172.21.15.185:0/3011622830 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:11.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddeffd700 1 --2- 172.21.15.185:0/3011622830 >> v2:172.21.15.96:3300/0 conn(0x7f9de013e1f0 0x7f9de0143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9dde7fc700 1 -- 172.21.15.185:0/3011622830 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553960895 0 0) 0x7f9de00b3a00 con 0x7f9de013de10 2020-11-16T22:21:11.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9dde7fc700 1 -- 172.21.15.185:0/3011622830 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dcc003240 con 0x7f9de013de10 2020-11-16T22:21:11.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 -- 172.21.15.185:0/3011622830 >> v2:172.21.15.96:3300/0 conn(0x7f9de013e1f0 msgr2=0x7f9de0143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:11.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 --2- 172.21.15.185:0/3011622830 >> v2:172.21.15.96:3300/0 conn(0x7f9de013e1f0 0x7f9de0143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 -- 172.21.15.185:0/3011622830 >> v1:172.21.15.185:6789/0 conn(0x7f9de013de10 legacy=0x7f9de00b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 -- 172.21.15.185:0/3011622830 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00b41c0 con 0x7f9de00b3570 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9ddffff700 1 --2- 172.21.15.185:0/3011622830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 0x7f9de00666b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f9dd8009c90 tx=0x7f9dd80091d0).ready entity=mon.2 client_cookie=7963ad4c9880025f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9dde7fc700 1 -- 172.21.15.185:0/3011622830 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800b9c0 con 0x7f9de00b3570 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9dde7fc700 1 -- 172.21.15.185:0/3011622830 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd800bb20 con 0x7f9de00b3570 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.554+0000 7f9dde7fc700 1 -- 172.21.15.185:0/3011622830 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800bea0 con 0x7f9de00b3570 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- 172.21.15.185:0/3011622830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 msgr2=0x7f9de00666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- 172.21.15.185:0/3011622830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 0x7f9de00666b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f9dd8009c90 tx=0x7f9dd80091d0).stop 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- 172.21.15.185:0/3011622830 shutdown_connections 2020-11-16T22:21:11.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- 172.21.15.185:0/3011622830 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de00b3570 0x7f9de00666b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- 172.21.15.185:0/3011622830 >> v2:172.21.15.96:3300/0 conn(0x7f9de013e1f0 0x7f9de0143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- 172.21.15.185:0/3011622830 >> 172.21.15.185:0/3011622830 conn(0x7f9de0064b90 msgr2=0x7f9de0063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:11.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- 172.21.15.185:0/3011622830 shutdown_connections 2020-11-16T22:21:11.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- 172.21.15.185:0/3011622830 wait complete. 2020-11-16T22:21:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 Processor -- start 2020-11-16T22:21:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- start start 2020-11-16T22:21:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9de00b3570 0x7f9de0157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de013e1f0 0x7f9de013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de013ebd0 con 0x7f9de013e1f0 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de013ee70 con 0x7f9de00b3570 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.558+0000 7f9de499d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de013f110 con 0x7f9de013de10 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddf7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9de00b3570 0x7f9de0157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55614/0 (socket says 172.21.15.185:55614) 2020-11-16T22:21:11.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 -- 172.21.15.185:0/184402425 learned_addr learned my addr 172.21.15.185:0/184402425 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddffff700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de013e1f0 0x7f9de013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 -- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9de00b3570 msgr2=0x7f9de0157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9de00b3570 0x7f9de0157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 -- 172.21.15.185:0/184402425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de013e1f0 msgr2=0x7f9de013d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de013e1f0 0x7f9de013d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 -- 172.21.15.185:0/184402425 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de013f3b0 con 0x7f9de013de10 2020-11-16T22:21:11.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddeffd700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f9dd400eba0 tx=0x7f9dd4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400b3a0 con 0x7f9de013de10 2020-11-16T22:21:11.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd400c8e0 con 0x7f9de013de10 2020-11-16T22:21:11.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de0147d50 con 0x7f9de013de10 2020-11-16T22:21:11.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400cca0 con 0x7f9de013de10 2020-11-16T22:21:11.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de0148360 con 0x7f9de013de10 2020-11-16T22:21:11.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de00b43a0 con 0x7f9de013de10 2020-11-16T22:21:11.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9dd4012600 con 0x7f9de013de10 2020-11-16T22:21:11.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 0x7f9dbc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:11.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9dd406dd60 con 0x7f9de013de10 2020-11-16T22:21:11.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.562+0000 7f9ddf7fe700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 0x7f9dbc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:11.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.566+0000 7f9ddf7fe700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 0x7f9dbc057520 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f9dd00055a0 tx=0x7f9dd0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:11.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.566+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9dd40442f0 con 0x7f9de013de10 2020-11-16T22:21:11.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:11.714+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9de005d200 con 0x7f9de013de10 2020-11-16T22:21:12.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.162+0000 7f9dc77fe700 1 -- 172.21.15.185:0/184402425 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v70) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9dd400ce00 con 0x7f9de013de10 2020-11-16T22:21:12.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 msgr2=0x7f9dbc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 0x7f9dbc057520 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f9dd00055a0 tx=0x7f9dd0009040).stop 2020-11-16T22:21:12.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 msgr2=0x7f9de0147860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f9dd400eba0 tx=0x7f9dd4006df0).stop 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 shutdown_connections 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9dbc055060 0x7f9dbc057520 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9de00b3570 0x7f9de0157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de013e1f0 0x7f9de013d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 --2- 172.21.15.185:0/184402425 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9de013de10 0x7f9de0147860 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0x7f9dd400eba0 tx=0x7f9dd4006df0).stop 2020-11-16T22:21:12.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 >> 172.21.15.185:0/184402425 conn(0x7f9de0064b90 msgr2=0x7f9de014a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:12.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 shutdown_connections 2020-11-16T22:21:12.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.166+0000 7f9de499d700 1 -- 172.21.15.185:0/184402425 wait complete. 2020-11-16T22:21:12.159 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:21:12.171 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:21:12.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 Processor -- start 2020-11-16T22:21:12.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- start start 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbce80af7f0 0x7fbce8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce80b47b0 con 0x7fbce8063fd0 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce80b4a50 con 0x7fbce80af7f0 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80b4cf0 con 0x7fbce80b3ec0 2020-11-16T22:21:12.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55622/0 (socket says 172.21.15.185:55622) 2020-11-16T22:21:12.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 -- 172.21.15.185:0/2237670522 learned_addr learned my addr 172.21.15.185:0/2237670522 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:12.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbced361700 1 --2- 172.21.15.185:0/2237670522 >> v2:172.21.15.96:3300/0 conn(0x7fbce80af7f0 0x7fbce8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 -- 172.21.15.185:0/2237670522 >> v2:172.21.15.96:3300/0 conn(0x7fbce80af7f0 msgr2=0x7fbce8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:12.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 --2- 172.21.15.185:0/2237670522 >> v2:172.21.15.96:3300/0 conn(0x7fbce80af7f0 0x7fbce8141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 -- 172.21.15.185:0/2237670522 >> v1:172.21.15.96:6789/0 conn(0x7fbce8063fd0 legacy=0x7fbce80b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 -- 172.21.15.185:0/2237670522 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80b4430 con 0x7fbce80b3ec0 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcee363700 1 --2- 172.21.15.185:0/2237670522 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbcd8009d50 tx=0x7fbcd8009290).ready entity=mon.2 client_cookie=b06dc2f32a7ff8c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcecb60700 1 -- 172.21.15.185:0/2237670522 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800bc80 con 0x7fbce80b3ec0 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcecb60700 1 -- 172.21.15.185:0/2237670522 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd800bde0 con 0x7fbce80b3ec0 2020-11-16T22:21:12.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbcecb60700 1 -- 172.21.15.185:0/2237670522 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8011610 con 0x7fbce80b3ec0 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- 172.21.15.185:0/2237670522 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 msgr2=0x7fbce8141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 --2- 172.21.15.185:0/2237670522 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fbcd8009d50 tx=0x7fbcd8009290).stop 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- 172.21.15.185:0/2237670522 shutdown_connections 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 --2- 172.21.15.185:0/2237670522 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce8141740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 --2- 172.21.15.185:0/2237670522 >> v2:172.21.15.96:3300/0 conn(0x7fbce80af7f0 0x7fbce8141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- 172.21.15.185:0/2237670522 >> 172.21.15.185:0/2237670522 conn(0x7fbce8067940 msgr2=0x7fbce8067d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:12.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- 172.21.15.185:0/2237670522 shutdown_connections 2020-11-16T22:21:12.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.306+0000 7fbceeb64700 1 -- 172.21.15.185:0/2237670522 wait complete. 2020-11-16T22:21:12.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 Processor -- start 2020-11-16T22:21:12.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- start start 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8063fd0 0x7fbce81445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 0x7fbce812aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce812b9d0 con 0x7fbce8063fd0 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8144f90 con 0x7fbce80af7f0 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8144000 con 0x7fbce80b3ec0 2020-11-16T22:21:12.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbced361700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 0x7fbce812aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55628/0 (socket says 172.21.15.185:55628) 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 -- 172.21.15.185:0/3071931648 learned_addr learned my addr 172.21.15.185:0/3071931648 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbced361700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 0x7fbce812aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33954/0 (socket says 172.21.15.185:33954) 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcedb62700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8063fd0 0x7fbce81445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 -- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 msgr2=0x7fbce812aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 0x7fbce812aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 -- 172.21.15.185:0/3071931648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8063fd0 msgr2=0x7fbce81445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8063fd0 0x7fbce81445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 -- 172.21.15.185:0/3071931648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce805d4e0 con 0x7fbce80b3ec0 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbcee363700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fbce8067470 tx=0x7fbcd8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800b750 con 0x7fbce80b3ec0 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8011e70 con 0x7fbce80b3ec0 2020-11-16T22:21:12.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce80b7790 con 0x7fbce80b3ec0 2020-11-16T22:21:12.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8010cb0 con 0x7fbce80b3ec0 2020-11-16T22:21:12.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.310+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80b6d00 con 0x7fbce80b3ec0 2020-11-16T22:21:12.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce80b7380 con 0x7fbce80b3ec0 2020-11-16T22:21:12.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbcd8017070 con 0x7fbce80b3ec0 2020-11-16T22:21:12.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbce5ffb700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 0x7fbcc4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:12.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fbcd806fdd0 con 0x7fbce80b3ec0 2020-11-16T22:21:12.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbcedb62700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 0x7fbcc4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:12.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.314+0000 7fbcedb62700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 0x7fbcc4057520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fbce80b5790 tx=0x7fbce0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:12.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.322+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbcd8046200 con 0x7fbce80b3ec0 2020-11-16T22:21:12.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:12.478+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbce80b71a0 con 0x7fbce80b3ec0 2020-11-16T22:21:13.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.178+0000 7fbce5ffb700 1 -- 172.21.15.185:0/3071931648 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v71) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbce80b71a0 con 0x7fbce80b3ec0 2020-11-16T22:21:13.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 msgr2=0x7fbcc4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 0x7fbcc4057520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fbce80b5790 tx=0x7fbce0009040).stop 2020-11-16T22:21:13.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 msgr2=0x7fbce812b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fbce8067470 tx=0x7fbcd8016000).stop 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 shutdown_connections 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbcc4055060 0x7fbcc4057520 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbce80b3ec0 0x7fbce812b4e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8063fd0 0x7fbce81445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 --2- 172.21.15.185:0/3071931648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbce80af7f0 0x7fbce812aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 >> 172.21.15.185:0/3071931648 conn(0x7fbce8067940 msgr2=0x7fbce8138760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:13.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 shutdown_connections 2020-11-16T22:21:13.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.182+0000 7fbceeb64700 1 -- 172.21.15.185:0/3071931648 wait complete. 2020-11-16T22:21:13.177 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:21:13.189 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:21:13.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 Processor -- start 2020-11-16T22:21:13.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- start start 2020-11-16T22:21:13.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc0b68d0 con 0x7fddfc0ac090 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc0b6b70 con 0x7fddfc14bc70 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc1488b0 con 0x7fddfc0b1a60 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfbfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fddfc14bc70 0x7fddfc1479a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55630/0 (socket says 172.21.15.185:55630) 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfbfff700 1 -- 172.21.15.185:0/926993068 learned_addr learned my addr 172.21.15.185:0/926993068 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:13.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55636/0 (socket says 172.21.15.185:55636) 2020-11-16T22:21:13.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfb7fe700 1 -- 172.21.15.185:0/926993068 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795367204 0 0) 0x7fddfc0b6b70 con 0x7fddfc14bc70 2020-11-16T22:21:13.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 -- 172.21.15.185:0/926993068 >> v1:172.21.15.185:6789/0 conn(0x7fddfc14bc70 legacy=0x7fddfc1479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 -- 172.21.15.185:0/926993068 >> v1:172.21.15.96:6789/0 conn(0x7fddfc0ac090 legacy=0x7fddfc148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 -- 172.21.15.185:0/926993068 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0b65c0 con 0x7fddfc0b1a60 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde0100b700 1 --2- 172.21.15.185:0/926993068 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fddf4009cc0 tx=0x7fddf40091d0).ready entity=mon.2 client_cookie=d5c3e0d106ad5b2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfb7fe700 1 -- 172.21.15.185:0/926993068 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf400bbc0 con 0x7fddfc0b1a60 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfb7fe700 1 -- 172.21.15.185:0/926993068 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf400bd20 con 0x7fddfc0b1a60 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fddfb7fe700 1 -- 172.21.15.185:0/926993068 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4011610 con 0x7fddfc0b1a60 2020-11-16T22:21:13.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- 172.21.15.185:0/926993068 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 msgr2=0x7fddfc1480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 --2- 172.21.15.185:0/926993068 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fddf4009cc0 tx=0x7fddf40091d0).stop 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- 172.21.15.185:0/926993068 shutdown_connections 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 --2- 172.21.15.185:0/926993068 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc0b1a60 0x7fddfc1480b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- 172.21.15.185:0/926993068 >> 172.21.15.185:0/926993068 conn(0x7fddfc065970 msgr2=0x7fddfc065d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.318+0000 7fde02a6e700 1 -- 172.21.15.185:0/926993068 shutdown_connections 2020-11-16T22:21:13.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 -- 172.21.15.185:0/926993068 wait complete. 2020-11-16T22:21:13.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 Processor -- start 2020-11-16T22:21:13.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 -- start start 2020-11-16T22:21:13.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 0x7fddfc14a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddfc0b1a60 0x7fddfc15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc1133a0 con 0x7fddfc0b1a60 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc112de0 con 0x7fddfc0ac090 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde02a6e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc068180 con 0x7fddfc15b460 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0080a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 0x7fddfc14a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0100b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0100b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55644/0 (socket says 172.21.15.185:55644) 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0100b700 1 -- 172.21.15.185:0/2711518356 learned_addr learned my addr 172.21.15.185:0/2711518356 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fddfbfff700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddfc0b1a60 0x7fddfc15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0080a700 1 -- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 msgr2=0x7fddfc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0080a700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0080a700 1 -- 172.21.15.185:0/2711518356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddfc0b1a60 msgr2=0x7fddfc15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.322+0000 7fde0080a700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddfc0b1a60 0x7fddfc15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde0080a700 1 -- 172.21.15.185:0/2711518356 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0ad210 con 0x7fddfc0ac090 2020-11-16T22:21:13.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde0100b700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:13.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde0080a700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 0x7fddfc14a330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fddf000c740 tx=0x7fddf0007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:13.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0007020 con 0x7fddfc0ac090 2020-11-16T22:21:13.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0007180 con 0x7fddfc0ac090 2020-11-16T22:21:13.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc11d020 con 0x7fddfc0ac090 2020-11-16T22:21:13.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00064f0 con 0x7fddfc0ac090 2020-11-16T22:21:13.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc11d560 con 0x7fddfc0ac090 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc05d4e0 con 0x7fddfc0ac090 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fddf0010030 con 0x7fddfc0ac090 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 0x7fddd4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fddf006d470 con 0x7fddfc0ac090 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddfbfff700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 0x7fddd4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.326+0000 7fddfbfff700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 0x7fddd4057520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fddfc1123d0 tx=0x7fddec0079a0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:13.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.334+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fddf0043740 con 0x7fddfc0ac090 2020-11-16T22:21:13.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.534+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fddfc0b5210 con 0x7fddfc0ac090 2020-11-16T22:21:13.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.546+0000 7fddf8ff9700 1 -- 172.21.15.185:0/2711518356 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v80) v1 ==== 189+0+0 (secure 0 0 0) 0x7fddf0046d60 con 0x7fddfc0ac090 2020-11-16T22:21:13.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.558+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 msgr2=0x7fddd4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.558+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 0x7fddd4057520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fddfc1123d0 tx=0x7fddec0079a0).stop 2020-11-16T22:21:13.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.558+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 msgr2=0x7fddfc14a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.558+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 0x7fddfc14a330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fddf000c740 tx=0x7fddf0007b00).stop 2020-11-16T22:21:13.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 shutdown_connections 2020-11-16T22:21:13.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fddd4055060 0x7fddd4057520 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fddfc0ac090 0x7fddfc14a330 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fddfc15b460 0x7fddfc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 --2- 172.21.15.185:0/2711518356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddfc0b1a60 0x7fddfc15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 >> 172.21.15.185:0/2711518356 conn(0x7fddfc065970 msgr2=0x7fddfc147c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:13.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 shutdown_connections 2020-11-16T22:21:13.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.562+0000 7fde02a6e700 1 -- 172.21.15.185:0/2711518356 wait complete. 2020-11-16T22:21:13.556 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 13 and data pool 14 2020-11-16T22:21:13.568 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:13.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 Processor -- start 2020-11-16T22:21:13.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 -- start start 2020-11-16T22:21:13.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f0061fe0 con 0x7fd7f00ad110 2020-11-16T22:21:13.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00b0fe0 con 0x7fd7f00b3090 2020-11-16T22:21:13.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7f5745700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00b2b60 con 0x7fd7f00b1d30 2020-11-16T22:21:13.684 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7effff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd7f00b3090 0x7fd7f014b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55644/0 (socket says 172.21.15.185:55644) 2020-11-16T22:21:13.685 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.690+0000 7fd7effff700 1 -- 172.21.15.185:0/564594535 learned_addr learned my addr 172.21.15.185:0/564594535 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7ef7fe700 1 -- 172.21.15.185:0/564594535 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962473469 0 0) 0x7fd7f00b0fe0 con 0x7fd7f00b3090 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7ef7fe700 1 -- 172.21.15.185:0/564594535 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d0003240 con 0x7fd7f00b3090 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7e77fe700 1 -- 172.21.15.185:0/564594535 >> v1:172.21.15.185:6789/0 conn(0x7fd7f00b3090 legacy=0x7fd7f014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7e77fe700 1 -- 172.21.15.185:0/564594535 >> v1:172.21.15.96:6789/0 conn(0x7fd7f00ad110 legacy=0x7fd7f01477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7e77fe700 1 -- 172.21.15.185:0/564594535 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00b1280 con 0x7fd7f00b1d30 2020-11-16T22:21:13.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7e77fe700 1 --2- 172.21.15.185:0/564594535 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f00b1720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fd7d800b9d0 tx=0x7fd7d8006df0).ready entity=mon.2 client_cookie=99a846000678a528 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:13.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7ef7fe700 1 -- 172.21.15.185:0/564594535 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80097b0 con 0x7fd7f00b1d30 2020-11-16T22:21:13.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7ef7fe700 1 -- 172.21.15.185:0/564594535 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8009910 con 0x7fd7f00b1d30 2020-11-16T22:21:13.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7ef7fe700 1 -- 172.21.15.185:0/564594535 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8009c90 con 0x7fd7f00b1d30 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 -- 172.21.15.185:0/564594535 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 msgr2=0x7fd7f00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 --2- 172.21.15.185:0/564594535 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f00b1720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fd7d800b9d0 tx=0x7fd7d8006df0).stop 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 -- 172.21.15.185:0/564594535 shutdown_connections 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 --2- 172.21.15.185:0/564594535 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f00b1720 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 -- 172.21.15.185:0/564594535 >> 172.21.15.185:0/564594535 conn(0x7fd7f0060130 msgr2=0x7fd7f0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 -- 172.21.15.185:0/564594535 shutdown_connections 2020-11-16T22:21:13.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.694+0000 7fd7f5745700 1 -- 172.21.15.185:0/564594535 wait complete. 2020-11-16T22:21:13.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 Processor -- start 2020-11-16T22:21:13.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- start start 2020-11-16T22:21:13.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7f00b3090 0x7fd7f014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f014d870 con 0x7fd7f00b3090 2020-11-16T22:21:13.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f014f1a0 con 0x7fd7f00ad110 2020-11-16T22:21:13.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00b0000 con 0x7fd7f00b1d30 2020-11-16T22:21:13.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7effff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7effff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:33978/0 (socket says 172.21.15.185:33978) 2020-11-16T22:21:13.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7effff700 1 -- 172.21.15.185:0/2087733658 learned_addr learned my addr 172.21.15.185:0/2087733658 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:13.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f4f44700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7f00b3090 0x7fd7f014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 -- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 msgr2=0x7fd7f014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 -- 172.21.15.185:0/2087733658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7f00b3090 msgr2=0x7fd7f014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:13.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7f00b3090 0x7fd7f014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:13.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 -- 172.21.15.185:0/2087733658 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00b02a0 con 0x7fd7f00b1d30 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7effff700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7e77fe700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f014ebb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fd7f0062a80 tx=0x7fd7d8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8011d50 con 0x7fd7f00b1d30 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d801e460 con 0x7fd7f00b1d30 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00b0540 con 0x7fd7f00b1d30 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d801e7e0 con 0x7fd7f00b1d30 2020-11-16T22:21:13.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.698+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f013a910 con 0x7fd7f00b1d30 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00b5a50 con 0x7fd7f00b1d30 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd7d8025030 con 0x7fd7f00b1d30 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7ecff9700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 0x7fd7cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7effff700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 0x7fd7cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd7d806f410 con 0x7fd7f00b1d30 2020-11-16T22:21:13.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.702+0000 7fd7effff700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 0x7fd7cc057520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fd7e00010e0 tx=0x7fd7e0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:14.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.710+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd7f00b5a50 con 0x7fd7f00b1d30 2020-11-16T22:21:14.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.862+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f00b5a50 con 0x7fd7f00b1d30 2020-11-16T22:21:14.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.862+0000 7fd7ecff9700 1 -- 172.21.15.185:0/2087733658 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16855 (secure 0 0 0) 0x7fd7d80311f0 con 0x7fd7f00b1d30 2020-11-16T22:21:14.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 msgr2=0x7fd7cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 0x7fd7cc057520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fd7e00010e0 tx=0x7fd7e0009040).stop 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 msgr2=0x7fd7f014ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f014ebb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fd7f0062a80 tx=0x7fd7d8015000).stop 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 shutdown_connections 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd7cc055060 0x7fd7cc057520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd7f00ad110 0x7fd7f014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7f00b3090 0x7fd7f014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 --2- 172.21.15.185:0/2087733658 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd7f00b1d30 0x7fd7f014ebb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 >> 172.21.15.185:0/2087733658 conn(0x7fd7f0060130 msgr2=0x7fd7f00b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:14.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 shutdown_connections 2020-11-16T22:21:14.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:13.866+0000 7fd7f5745700 1 -- 172.21.15.185:0/2087733658 wait complete. 2020-11-16T22:21:14.306 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:14.306 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":72,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:13.541269+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:14.714 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:14.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 Processor -- start 2020-11-16T22:21:14.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 -- start start 2020-11-16T22:21:14.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e900b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e90061fe0 con 0x7f6e900ad110 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900b0fe0 con 0x7f6e900b1d30 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e958a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b2b60 con 0x7f6e900b3090 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8ffff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6e900b1d30 0x7f6e9014b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55656/0 (socket says 172.21.15.185:55656) 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8ffff700 1 -- 172.21.15.185:0/4083769875 learned_addr learned my addr 172.21.15.185:0/4083769875 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e948a5700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e900b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197539930 0 0) 0x7f6e900b0fe0 con 0x7f6e900b1d30 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e74003240 con 0x7f6e900b1d30 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1588057796 0 0) 0x7f6e74003240 con 0x7f6e900b1d30 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 msgr2=0x7f6e900b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e900b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 >> v1:172.21.15.96:6789/0 conn(0x7f6e900ad110 legacy=0x7f6e901477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900b1280 con 0x7f6e900b1d30 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f6e780014e0 con 0x7f6e900b1d30 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 >> v1:172.21.15.185:6789/0 conn(0x7f6e900b1d30 legacy=0x7f6e9014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e74003730 0x7f6e74003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e740049d0 0x7f6e74004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b2b60 con 0x7f6e740049d0 2020-11-16T22:21:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740041d0 con 0x7f6e74003730 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7400b730 con 0x7f6e900b3090 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e948a5700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e74003730 0x7f6e74003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e8ffff700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e950a6700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e740049d0 0x7f6e74004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.854+0000 7f6e948a5700 1 -- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 msgr2=0x7f6e740044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e948a5700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e740044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e948a5700 1 -- 172.21.15.185:0/4083769875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e740049d0 msgr2=0x7f6e74004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:14.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e948a5700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e740049d0 0x7f6e74004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:14.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e948a5700 1 -- 172.21.15.185:0/4083769875 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74003240 con 0x7f6e74003730 2020-11-16T22:21:14.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e950a6700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e740049d0 0x7f6e74004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:14.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e8ffff700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900b3090 0x7f6e740044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:14.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e948a5700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e74003730 0x7f6e74003b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6e8000bc20 tx=0x7f6e80006df0).ready entity=mon.1 client_cookie=fe7f156198b12b84 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:14.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000b300 con 0x7f6e74003730 2020-11-16T22:21:14.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8000ea90 con 0x7f6e74003730 2020-11-16T22:21:14.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e8f7fe700 1 -- 172.21.15.185:0/4083769875 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000ee10 con 0x7f6e74003730 2020-11-16T22:21:14.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e74003730 msgr2=0x7f6e74003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:14.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 --2- 172.21.15.185:0/4083769875 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e74003730 0x7f6e74003b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6e8000bc20 tx=0x7f6e80006df0).stop 2020-11-16T22:21:14.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e950a6700 1 -- 172.21.15.185:0/4083769875 reap_dead start 2020-11-16T22:21:15.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- 172.21.15.185:0/4083769875 shutdown_connections 2020-11-16T22:21:15.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- 172.21.15.185:0/4083769875 >> 172.21.15.185:0/4083769875 conn(0x7f6e90060130 msgr2=0x7f6e90060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:15.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- 172.21.15.185:0/4083769875 shutdown_connections 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- 172.21.15.185:0/4083769875 wait complete. 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 Processor -- start 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- start start 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900aff30 0x7f6e900b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e9014f8f0 0x7f6e90112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9010f830 con 0x7f6e9014f8f0 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9014f450 con 0x7f6e9014e8c0 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.858+0000 7f6e958a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9010fad0 con 0x7f6e900aff30 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34000/0 (socket says 172.21.15.185:34000) 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 -- 172.21.15.185:0/1890877208 learned_addr learned my addr 172.21.15.185:0/1890877208 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e948a5700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900aff30 0x7f6e900b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e950a6700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e9014f8f0 0x7f6e90112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 -- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900aff30 msgr2=0x7f6e900b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900aff30 0x7f6e900b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 -- 172.21.15.185:0/1890877208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e9014f8f0 msgr2=0x7f6e90112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e9014f8f0 0x7f6e90112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 -- 172.21.15.185:0/1890877208 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e901118d0 con 0x7f6e9014e8c0 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8ffff700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6e780079f0 tx=0x7f6e900b1170).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:15.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800fca0 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7800fe00 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9014e180 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78012560 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9014fef0 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6e7801b030 con 0x7f6e9014e8c0 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 0x7f6e60057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e7806d8a0 con 0x7f6e9014e8c0 2020-11-16T22:21:15.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e948a5700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 0x7f6e60057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e948a5700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 0x7f6e60057520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f6e90045400 tx=0x7f6e80019000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:15.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.862+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9014e4d0 con 0x7f6e9014e8c0 2020-11-16T22:21:15.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:14.870+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6e78019070 con 0x7f6e9014e8c0 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.022+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e9005d200 con 0x7f6e9014e8c0 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e8cff9700 1 -- 172.21.15.185:0/1890877208 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16842 (secure 0 0 0) 0x7f6e78043b00 con 0x7f6e9014e8c0 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 msgr2=0x7f6e60057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 0x7f6e60057520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f6e90045400 tx=0x7f6e80019000).stop 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 msgr2=0x7f6e90157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6e780079f0 tx=0x7f6e900b1170).stop 2020-11-16T22:21:15.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 shutdown_connections 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e60055060 0x7f6e60057520 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e9014f8f0 0x7f6e90112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e900aff30 0x7f6e900b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 --2- 172.21.15.185:0/1890877208 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e9014e8c0 0x7f6e90157ae0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.026+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 >> 172.21.15.185:0/1890877208 conn(0x7f6e90060130 msgr2=0x7f6e90147810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:15.270 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.030+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 shutdown_connections 2020-11-16T22:21:15.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.030+0000 7f6e958a7700 1 -- 172.21.15.185:0/1890877208 wait complete. 2020-11-16T22:21:15.271 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:15.271 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":73,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:14.544614+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:15.682 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:21:15.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.822+0000 7ffb5114a700 1 Processor -- start 2020-11-16T22:21:15.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.822+0000 7ffb5114a700 1 -- start start 2020-11-16T22:21:15.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ffb4c0b7690 0x7ffb4c1447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 0x7ffb4c1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c0666b0 con 0x7ffb4c0b7690 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c066950 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0603c0 con 0x7ffb4c0b3500 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb50949700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 0x7ffb4c1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34006/0 (socket says 172.21.15.185:34006) 2020-11-16T22:21:15.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb50949700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 0x7ffb4c1433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55686/0 (socket says 172.21.15.185:55686) 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 -- 172.21.15.185:0/3056168690 learned_addr learned my addr 172.21.15.185:0/3056168690 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4bfff700 1 --2- 172.21.15.185:0/3056168690 >> v2:172.21.15.96:3300/0 conn(0x7ffb4c0b7690 0x7ffb4c1447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 -- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 msgr2=0x7ffb4c1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 0x7ffb4c1433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 -- 172.21.15.185:0/3056168690 >> v2:172.21.15.96:3300/0 conn(0x7ffb4c0b7690 msgr2=0x7ffb4c1447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/3056168690 >> v2:172.21.15.96:3300/0 conn(0x7ffb4c0b7690 0x7ffb4c1447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 -- 172.21.15.185:0/3056168690 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0609a0 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7ffb4c0ae650 tx=0x7ffb40006eb0).ready entity=mon.1 client_cookie=ba45a2657a3dae19 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4affd700 1 -- 172.21.15.185:0/3056168690 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40009810 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4affd700 1 -- 172.21.15.185:0/3056168690 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb40009970 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb4affd700 1 -- 172.21.15.185:0/3056168690 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40009cf0 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 msgr2=0x7ffb4c144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7ffb4c0ae650 tx=0x7ffb40006eb0).stop 2020-11-16T22:21:15.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- 172.21.15.185:0/3056168690 shutdown_connections 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b3500 0x7ffb4c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- 172.21.15.185:0/3056168690 >> v2:172.21.15.96:3300/0 conn(0x7ffb4c0b7690 0x7ffb4c1447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 --2- 172.21.15.185:0/3056168690 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c144ca0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- 172.21.15.185:0/3056168690 >> 172.21.15.185:0/3056168690 conn(0x7ffb4c064b90 msgr2=0x7ffb4c063790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- 172.21.15.185:0/3056168690 shutdown_connections 2020-11-16T22:21:15.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.826+0000 7ffb5114a700 1 -- 172.21.15.185:0/3056168690 wait complete. 2020-11-16T22:21:15.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.830+0000 7ffb5114a700 1 Processor -- start 2020-11-16T22:21:15.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.830+0000 7ffb5114a700 1 -- start start 2020-11-16T22:21:15.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb4c0b3500 0x7ffb4c1453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b7690 0x7ffb4c0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c13f650 con 0x7ffb4c0b3500 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c140160 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c148a10 con 0x7ffb4c0b7690 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34012/0 (socket says 172.21.15.185:34012) 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 -- 172.21.15.185:0/383566312 learned_addr learned my addr 172.21.15.185:0/383566312 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:15.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb50949700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b7690 0x7ffb4c0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb4c0b3500 0x7ffb4c1453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 -- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b7690 msgr2=0x7ffb4c0ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b7690 0x7ffb4c0ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 -- 172.21.15.185:0/383566312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb4c0b3500 msgr2=0x7ffb4c1453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb4c0b3500 0x7ffb4c1453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:15.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 -- 172.21.15.185:0/383566312 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c148cb0 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb4bfff700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7ffb3c00db90 tx=0x7ffb3c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:15.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c009790 con 0x7ffb4c0b1f20 2020-11-16T22:21:15.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0098f0 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c1458b0 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c009c90 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c140a00 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c148eb0 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.834+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ffb3c013480 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.838+0000 7ffb2bfff700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 0x7ffb24057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.838+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffb3c06e080 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.838+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 0x7ffb24057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.838+0000 7ffb4b7fe700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 0x7ffb24057550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7ffb4c0ae650 tx=0x7ffb40015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:16.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:15.846+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ffb3c097ef0 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.042+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb4c140b90 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.566 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.574+0000 7ffb2bfff700 1 -- 172.21.15.185:0/383566312 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb4c148eb0 con 0x7ffb4c0b1f20 2020-11-16T22:21:16.569 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 msgr2=0x7ffb24057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 0x7ffb24057550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7ffb4c0ae650 tx=0x7ffb40015000).stop 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 msgr2=0x7ffb4c13fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7ffb3c00db90 tx=0x7ffb3c008040).stop 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 shutdown_connections 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffb24055090 0x7ffb24057550 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.570 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb4c0b1f20 0x7ffb4c13fb30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb4c0b7690 0x7ffb4c0ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 --2- 172.21.15.185:0/383566312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb4c0b3500 0x7ffb4c1453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 >> 172.21.15.185:0/383566312 conn(0x7ffb4c064b90 msgr2=0x7ffb4c142510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:16.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 shutdown_connections 2020-11-16T22:21:16.571 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.578+0000 7ffb5114a700 1 -- 172.21.15.185:0/383566312 wait complete. 2020-11-16T22:21:16.586 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:16.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 Processor -- start 2020-11-16T22:21:16.714 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 -- start start 2020-11-16T22:21:16.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e681446c0 0x7f0e68062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e680b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e68067930 con 0x7f0e68062900 2020-11-16T22:21:16.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e680632e0 con 0x7f0e681446c0 2020-11-16T22:21:16.716 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6f195700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68066ae0 con 0x7f0e68067de0 2020-11-16T22:21:16.717 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.722+0000 7f0e6d732700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e680b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e681446c0 0x7f0e68062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6cf31700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0e68062900 0x7f0e68067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:41754/0 (socket says 172.21.15.185:41754) 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e681446c0 0x7f0e68062ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.185:38316/0 (socket says 172.21.15.185:38316) 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6cf31700 1 -- 172.21.15.185:0/274660657 learned_addr learned my addr 172.21.15.185:0/274660657 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 -- 172.21.15.185:0/274660657 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 msgr2=0x7f0e680b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:16.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e680b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 -- 172.21.15.185:0/274660657 >> v1:172.21.15.96:6789/0 conn(0x7f0e68062900 legacy=0x7f0e68067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 -- 172.21.15.185:0/274660657 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68066d80 con 0x7f0e681446c0 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e677fe700 1 -- 172.21.15.185:0/274660657 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338506148 0 0) 0x7f0e68067930 con 0x7f0e68062900 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6d732700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e680b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e67fff700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e68062ce0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f0e5000cb30 tx=0x7f0e50006e80).ready entity=mon.0 client_cookie=e361d61dcadcb866 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e677fe700 1 -- 172.21.15.185:0/274660657 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000aa50 con 0x7f0e681446c0 2020-11-16T22:21:16.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e677fe700 1 -- 172.21.15.185:0/274660657 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5000abb0 con 0x7f0e681446c0 2020-11-16T22:21:16.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e677fe700 1 -- 172.21.15.185:0/274660657 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000ad20 con 0x7f0e681446c0 2020-11-16T22:21:16.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 -- 172.21.15.185:0/274660657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 msgr2=0x7f0e68062ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e68062ce0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f0e5000cb30 tx=0x7f0e50006e80).stop 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 -- 172.21.15.185:0/274660657 shutdown_connections 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e680b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 --2- 172.21.15.185:0/274660657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e68062ce0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 -- 172.21.15.185:0/274660657 >> 172.21.15.185:0/274660657 conn(0x7f0e680b4410 msgr2=0x7f0e68060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 -- 172.21.15.185:0/274660657 shutdown_connections 2020-11-16T22:21:16.721 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.726+0000 7f0e6f195700 1 -- 172.21.15.185:0/274660657 wait complete. 2020-11-16T22:21:16.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 Processor -- start 2020-11-16T22:21:16.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 -- start start 2020-11-16T22:21:16.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 0x7f0e6814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e6814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e680af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6814ef70 con 0x7f0e681446c0 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680acfb0 con 0x7f0e68062900 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6f195700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680b0ec0 con 0x7f0e68067de0 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6cf31700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 0x7f0e6814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6cf31700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 0x7f0e6814f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34026/0 (socket says 172.21.15.185:34026) 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e6cf31700 1 -- 172.21.15.185:0/3587868123 learned_addr learned my addr 172.21.15.185:0/3587868123 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.730+0000 7f0e67fff700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e6814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6d732700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e680af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 -- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 msgr2=0x7f0e6814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 0x7f0e6814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 -- 172.21.15.185:0/3587868123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 msgr2=0x7f0e680af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e680af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 -- 172.21.15.185:0/3587868123 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680b6790 con 0x7f0e68067de0 2020-11-16T22:21:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6d732700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e680af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e67fff700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e6814fa90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f0e50013000 tx=0x7f0e50006e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50018510 con 0x7f0e68067de0 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50018670 con 0x7f0e68067de0 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e680ad1e0 con 0x7f0e68067de0 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500189f0 con 0x7f0e68067de0 2020-11-16T22:21:16.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6813edd0 con 0x7f0e68067de0 2020-11-16T22:21:16.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6805d4e0 con 0x7f0e68067de0 2020-11-16T22:21:16.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0e50021070 con 0x7f0e68067de0 2020-11-16T22:21:16.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 0x7f0e44057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:16.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e50070980 con 0x7f0e68067de0 2020-11-16T22:21:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.734+0000 7f0e6cf31700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 0x7f0e44057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.738+0000 7f0e6cf31700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 0x7f0e44057520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f0e68141810 tx=0x7f0e58007ac0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.738+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0e50046bc0 con 0x7f0e68067de0 2020-11-16T22:21:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.942+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e68061e90 con 0x7f0e68067de0 2020-11-16T22:21:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.942+0000 7f0e64ff9700 1 -- 172.21.15.185:0/3587868123 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f0e5004a1e0 con 0x7f0e68067de0 2020-11-16T22:21:16.935 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:16.936 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":78},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:16.571738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 msgr2=0x7f0e44057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 0x7f0e44057520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f0e68141810 tx=0x7f0e58007ac0).stop 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 msgr2=0x7f0e6814fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e6814fa90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f0e50013000 tx=0x7f0e50006e80).stop 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 shutdown_connections 2020-11-16T22:21:16.938 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e44055060 0x7f0e44057520 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e68062900 0x7f0e6814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e681446c0 0x7f0e680af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 --2- 172.21.15.185:0/3587868123 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e68067de0 0x7f0e6814fa90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 >> 172.21.15.185:0/3587868123 conn(0x7f0e680b4410 msgr2=0x7f0e68140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 shutdown_connections 2020-11-16T22:21:16.939 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:16.946+0000 7f0e6f195700 1 -- 172.21.15.185:0/3587868123 wait complete. 2020-11-16T22:21:16.940 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 83 2020-11-16T22:21:16.950 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:17.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 Processor -- start 2020-11-16T22:21:17.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 -- start start 2020-11-16T22:21:17.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 0x7f066c066a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f066c148bf0 0x7f066c0b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c149350 con 0x7f066c060c50 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0685c0 con 0x7f066c148bf0 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f067335e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c1490b0 con 0x7f066c066680 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f0671b5b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f066c060c50 0x7f066c1484e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55704/0 (socket says 172.21.15.185:55704) 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.082+0000 7f0671b5b700 1 -- 172.21.15.185:0/4102516186 learned_addr learned my addr 172.21.15.185:0/4102516186 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:17.077 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 --2- 172.21.15.185:0/4102516186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 0x7f066c066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f0672b5d700 1 --2- 172.21.15.185:0/4102516186 >> v2:172.21.15.96:3300/0 conn(0x7f066c148bf0 0x7f066c0b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067135a700 1 -- 172.21.15.185:0/4102516186 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305523237 0 0) 0x7f066c149350 con 0x7f066c060c50 2020-11-16T22:21:17.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067135a700 1 -- 172.21.15.185:0/4102516186 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0658003240 con 0x7f066c060c50 2020-11-16T22:21:17.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 -- 172.21.15.185:0/4102516186 >> v2:172.21.15.96:3300/0 conn(0x7f066c148bf0 msgr2=0x7f066c0b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:17.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 --2- 172.21.15.185:0/4102516186 >> v2:172.21.15.96:3300/0 conn(0x7f066c148bf0 0x7f066c0b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 -- 172.21.15.185:0/4102516186 >> v1:172.21.15.185:6789/0 conn(0x7f066c060c50 legacy=0x7f066c1484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 -- 172.21.15.185:0/4102516186 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0682b0 con 0x7f066c066680 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067235c700 1 --2- 172.21.15.185:0/4102516186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 0x7f066c066a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f065c00b9a0 tx=0x7f065c006df0).ready entity=mon.2 client_cookie=f55c4f2829342760 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067135a700 1 -- 172.21.15.185:0/4102516186 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c009710 con 0x7f066c066680 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067135a700 1 -- 172.21.15.185:0/4102516186 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c009870 con 0x7f066c066680 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067135a700 1 -- 172.21.15.185:0/4102516186 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c009bf0 con 0x7f066c066680 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- 172.21.15.185:0/4102516186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 msgr2=0x7f066c066a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 --2- 172.21.15.185:0/4102516186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 0x7f066c066a60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f065c00b9a0 tx=0x7f065c006df0).stop 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- 172.21.15.185:0/4102516186 shutdown_connections 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 --2- 172.21.15.185:0/4102516186 >> v2:172.21.15.96:3300/0 conn(0x7f066c148bf0 0x7f066c0b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 --2- 172.21.15.185:0/4102516186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c066680 0x7f066c066a60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- 172.21.15.185:0/4102516186 >> 172.21.15.185:0/4102516186 conn(0x7f066c060130 msgr2=0x7f066c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- 172.21.15.185:0/4102516186 shutdown_connections 2020-11-16T22:21:17.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- 172.21.15.185:0/4102516186 wait complete. 2020-11-16T22:21:17.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 Processor -- start 2020-11-16T22:21:17.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.086+0000 7f067335e700 1 -- start start 2020-11-16T22:21:17.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f066c066680 0x7f066c143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 0x7f066c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c1450e0 con 0x7f066c066680 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c14b040 con 0x7f066c148bf0 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c149c70 con 0x7f066c060c50 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55716/0 (socket says 172.21.15.185:55716) 2020-11-16T22:21:17.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f0672b5d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 0x7f066c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 -- 172.21.15.185:0/3243661470 learned_addr learned my addr 172.21.15.185:0/3243661470 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f0671b5b700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f066c066680 0x7f066c143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 -- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 msgr2=0x7f066c143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 0x7f066c143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 -- 172.21.15.185:0/3243661470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f066c066680 msgr2=0x7f066c143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f066c066680 0x7f066c143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 -- 172.21.15.185:0/3243661470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0acbc0 con 0x7f066c060c50 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f0672b5d700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 0x7f066c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067235c700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f065c009410 tx=0x7f065c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c011c80 con 0x7f066c060c50 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c011de0 con 0x7f066c060c50 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c149f10 con 0x7f066c060c50 2020-11-16T22:21:17.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c01e670 con 0x7f066c060c50 2020-11-16T22:21:17.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c144440 con 0x7f066c060c50 2020-11-16T22:21:17.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.090+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c0b08c0 con 0x7f066c060c50 2020-11-16T22:21:17.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.094+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f065c025030 con 0x7f066c060c50 2020-11-16T22:21:17.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.094+0000 7f06627fc700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 0x7f0644057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.094+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f065c06f4d0 con 0x7f066c060c50 2020-11-16T22:21:17.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.094+0000 7f0671b5b700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 0x7f0644057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.094+0000 7f0671b5b700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 0x7f0644057520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f066c0b3410 tx=0x7f0668009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.098+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f065c045730 con 0x7f066c060c50 2020-11-16T22:21:17.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.246+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066c05d200 con 0x7f066c060c50 2020-11-16T22:21:17.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.246+0000 7f06627fc700 1 -- 172.21.15.185:0/3243661470 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f065c023090 con 0x7f066c060c50 2020-11-16T22:21:17.239 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:17.239 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:15.557108+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:17.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 msgr2=0x7f0644057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 0x7f0644057520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f066c0b3410 tx=0x7f0668009040).stop 2020-11-16T22:21:17.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 msgr2=0x7f066c0ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f065c009410 tx=0x7f065c015000).stop 2020-11-16T22:21:17.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 shutdown_connections 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0644055060 0x7f0644057520 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f066c148bf0 0x7f066c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f066c060c50 0x7f066c0ac6d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 --2- 172.21.15.185:0/3243661470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f066c066680 0x7f066c143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 >> 172.21.15.185:0/3243661470 conn(0x7f066c060130 msgr2=0x7f066c141840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 shutdown_connections 2020-11-16T22:21:17.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.250+0000 7f067335e700 1 -- 172.21.15.185:0/3243661470 wait complete. 2020-11-16T22:21:17.445 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:21:17.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.586+0000 7f96d2b38700 1 Processor -- start 2020-11-16T22:21:17.580 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.586+0000 7f96d2b38700 1 -- start start 2020-11-16T22:21:17.581 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f96cc0b7eb0 0x7f96cc1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc140ff0 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc141290 con 0x7f96cc067590 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc141530 con 0x7f96cc144e10 2020-11-16T22:21:17.582 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2337700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55726/0 (socket says 172.21.15.185:55726) 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2337700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34052/0 (socket says 172.21.15.185:34052) 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 -- 172.21.15.185:0/3669517969 learned_addr learned my addr 172.21.15.185:0/3669517969 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1b36700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.96:3300/0 conn(0x7f96cc0b7eb0 0x7f96cc1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.583 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 -- 172.21.15.185:0/3669517969 >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 msgr2=0x7f96cc063390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 -- 172.21.15.185:0/3669517969 >> v2:172.21.15.96:3300/0 conn(0x7f96cc0b7eb0 msgr2=0x7f96cc1419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.96:3300/0 conn(0x7f96cc0b7eb0 0x7f96cc1419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 -- 172.21.15.185:0/3669517969 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0664a0 con 0x7f96cc144e10 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2337700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:17.584 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d1335700 1 --2- 172.21.15.185:0/3669517969 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f96cc13b510 tx=0x7f96c0006eb0).ready entity=mon.2 client_cookie=23c7136a367b82a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d0b34700 1 -- 172.21.15.185:0/3669517969 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0009720 con 0x7f96cc144e10 2020-11-16T22:21:17.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d0b34700 1 -- 172.21.15.185:0/3669517969 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0009880 con 0x7f96cc144e10 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d0b34700 1 -- 172.21.15.185:0/3669517969 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0009c00 con 0x7f96cc144e10 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- 172.21.15.185:0/3669517969 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 msgr2=0x7f96cc062ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- 172.21.15.185:0/3669517969 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f96cc13b510 tx=0x7f96c0006eb0).stop 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- 172.21.15.185:0/3669517969 shutdown_connections 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.185:3300/0 conn(0x7f96cc067590 0x7f96cc063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- 172.21.15.185:0/3669517969 >> v2:172.21.15.96:3300/0 conn(0x7f96cc0b7eb0 0x7f96cc1419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 --2- 172.21.15.185:0/3669517969 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc144e10 0x7f96cc062ea0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- 172.21.15.185:0/3669517969 >> 172.21.15.185:0/3669517969 conn(0x7f96cc061d10 msgr2=0x7f96cc0b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- 172.21.15.185:0/3669517969 shutdown_connections 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 -- 172.21.15.185:0/3669517969 wait complete. 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.590+0000 7f96d2b38700 1 Processor -- start 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- start start 2020-11-16T22:21:17.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 0x7f96cc140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc144e10 0x7f96cc0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc13dbf0 con 0x7f96cc144e10 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc13b560 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc13d490 con 0x7f96cc067590 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1b36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.589 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 0x7f96cc140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1b36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55730/0 (socket says 172.21.15.185:55730) 2020-11-16T22:21:17.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1b36700 1 -- 172.21.15.185:0/4150289427 learned_addr learned my addr 172.21.15.185:0/4150289427 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:17.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2337700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc144e10 0x7f96cc0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 -- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 msgr2=0x7f96cc13af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.590 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 -- 172.21.15.185:0/4150289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc144e10 msgr2=0x7f96cc0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc144e10 0x7f96cc0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 -- 172.21.15.185:0/4150289427 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc13c4d0 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1b36700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d1335700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 0x7f96cc140740 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f96c0012000 tx=0x7f96c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0011bc0 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.591 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0011d20 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc13c770 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c001e690 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.592 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc0b10f0 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.594+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc13b990 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.598+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f96c0025030 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.598+0000 7f96bdffb700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 0x7f96a4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.598+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96c006efb0 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.598+0000 7f96d1b36700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 0x7f96a4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.598+0000 7f96d1b36700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 0x7f96a4057550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f96c4006af0 tx=0x7f96c40091a0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.602+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f96cc13b990 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.810+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f96cc13b990 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.818+0000 7f96bdffb700 1 -- 172.21.15.185:0/4150289427 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f96cc13b990 con 0x7f96cc0b7eb0 2020-11-16T22:21:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 msgr2=0x7f96a4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 0x7f96a4057550 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f96c4006af0 tx=0x7f96c40091a0).stop 2020-11-16T22:21:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 msgr2=0x7f96cc140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 0x7f96cc140740 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f96c0012000 tx=0x7f96c0015000).stop 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 shutdown_connections 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96a4055090 0x7f96a4057550 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc144e10 0x7f96cc0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96cc067590 0x7f96cc13af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 --2- 172.21.15.185:0/4150289427 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96cc0b7eb0 0x7f96cc140740 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 >> 172.21.15.185:0/4150289427 conn(0x7f96cc061d10 msgr2=0x7f96cc0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:17.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 shutdown_connections 2020-11-16T22:21:17.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.822+0000 7f96d2b38700 1 -- 172.21.15.185:0/4150289427 wait complete. 2020-11-16T22:21:17.815 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:21:17.830 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:17.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 Processor -- start 2020-11-16T22:21:17.954 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 -- start start 2020-11-16T22:21:17.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1eb00669a0 0x7f1eb0066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00abf90 con 0x7f1eb00acc10 2020-11-16T22:21:17.955 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00ac230 con 0x7f1eb0061e60 2020-11-16T22:21:17.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb59bc700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb00ac4d0 con 0x7f1eb00669a0 2020-11-16T22:21:17.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb51bb700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f1eb0061e60 0x7f1eb01442f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55736/0 (socket says 172.21.15.185:55736) 2020-11-16T22:21:17.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb51bb700 1 -- 172.21.15.185:0/2442023229 learned_addr learned my addr 172.21.15.185:0/2442023229 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:17.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2442023229 >> v2:172.21.15.96:3300/0 conn(0x7f1eb00669a0 0x7f1eb0066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121925883 0 0) 0x7f1eb00ac230 con 0x7f1eb0061e60 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e94003240 con 0x7f1eb0061e60 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704161956 0 0) 0x7f1eb00abf90 con 0x7f1eb00acc10 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb00ac230 con 0x7f1eb00acc10 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb49ba700 1 -- 172.21.15.185:0/2442023229 >> v1:172.21.15.185:6789/0 conn(0x7f1eb0061e60 legacy=0x7f1eb01442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb49ba700 1 -- 172.21.15.185:0/2442023229 >> v1:172.21.15.96:6789/0 conn(0x7f1eb00acc10 legacy=0x7f1eb0147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.962+0000 7f1eb49ba700 1 -- 172.21.15.185:0/2442023229 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00ac770 con 0x7f1eb00669a0 2020-11-16T22:21:17.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2442023229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb00669a0 0x7f1eb0066d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1ea000b9a0 tx=0x7f1ea0006df0).ready entity=mon.0 client_cookie=6f59d34a2717abda server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0009680 con 0x7f1eb00669a0 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00097e0 con 0x7f1eb00669a0 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaf7fe700 1 -- 172.21.15.185:0/2442023229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0009b60 con 0x7f1eb00669a0 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2442023229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb00669a0 msgr2=0x7f1eb0066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2442023229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb00669a0 0x7f1eb0066d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1ea000b9a0 tx=0x7f1ea0006df0).stop 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2442023229 shutdown_connections 2020-11-16T22:21:17.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2442023229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb00669a0 0x7f1eb0066d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2442023229 >> 172.21.15.185:0/2442023229 conn(0x7f1eb0060130 msgr2=0x7f1eb0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2442023229 shutdown_connections 2020-11-16T22:21:17.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2442023229 wait complete. 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 Processor -- start 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- start start 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 0x7f1eb00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1eb0140220 0x7f1eb0140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00af4f0 con 0x7f1eb0061e60 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00add30 con 0x7f1eb00669a0 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb59bc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00adfd0 con 0x7f1eb0140220 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34070/0 (socket says 172.21.15.185:34070) 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eaffff700 1 -- 172.21.15.185:0/2607199045 learned_addr learned my addr 172.21.15.185:0/2607199045 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb51bb700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1eb0140220 0x7f1eb0140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.966+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 0x7f1eb00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 -- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1eb0140220 msgr2=0x7f1eb0140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1eb0140220 0x7f1eb0140680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 -- 172.21.15.185:0/2607199045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 msgr2=0x7f1eb00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 0x7f1eb00afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 -- 172.21.15.185:0/2607199045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00af6d0 con 0x7f1eb00669a0 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 0x7f1eb00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eaffff700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1e9800c710 tx=0x7f1e98007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800e560 con 0x7f1eb00669a0 2020-11-16T22:21:17.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9800e6c0 con 0x7f1eb00669a0 2020-11-16T22:21:17.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb0140b70 con 0x7f1eb00669a0 2020-11-16T22:21:17.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800ea40 con 0x7f1eb00669a0 2020-11-16T22:21:17.965 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb01480e0 con 0x7f1eb00669a0 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb00af860 con 0x7f1eb00669a0 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1e98010070 con 0x7f1eb00669a0 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 0x7f1e80057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e9806d900 con 0x7f1eb00669a0 2020-11-16T22:21:17.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 0x7f1e80057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:17.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.970+0000 7f1eb49ba700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 0x7f1e80057520 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1ea000c010 tx=0x7f1ea0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:17.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:17.982+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1e98043b60 con 0x7f1eb00669a0 2020-11-16T22:21:18.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.182+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eb005d200 con 0x7f1eb00669a0 2020-11-16T22:21:18.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.182+0000 7f1eacff9700 1 -- 172.21.15.185:0/2607199045 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f1e98047180 con 0x7f1eb00669a0 2020-11-16T22:21:18.175 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:18.175 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":78},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:16.571738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:18.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 msgr2=0x7f1e80057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:18.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 0x7f1e80057520 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1ea000c010 tx=0x7f1ea0015000).stop 2020-11-16T22:21:18.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 msgr2=0x7f1eb01464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:18.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f1e9800c710 tx=0x7f1e98007a50).stop 2020-11-16T22:21:18.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 shutdown_connections 2020-11-16T22:21:18.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1e80055060 0x7f1e80057520 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1eb0140220 0x7f1eb0140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb0061e60 0x7f1eb00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 --2- 172.21.15.185:0/2607199045 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1eb00669a0 0x7f1eb01464e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 >> 172.21.15.185:0/2607199045 conn(0x7f1eb0060130 msgr2=0x7f1eb00b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 shutdown_connections 2020-11-16T22:21:18.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:18.186+0000 7f1eb59bc700 1 -- 172.21.15.185:0/2607199045 wait complete. 2020-11-16T22:21:18.179 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 83 2020-11-16T22:21:18.492 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:21:13.541645+0000', 'modified': '2020-11-16T22:21:16.571738+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3098481638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3098481638}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3098481638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:21:18.493 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:21:18.493 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:21:18.643 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7083,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.1006430509999996,"mdsmap_epoch":83,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":10.351372355000001} 2020-11-16T22:21:18.643 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7083, "mdsmap_epoch": 83, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 5.100643051, "state": "up:active", "uptime": 10.351372355, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:21:18.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:21:18.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:21:18.644 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:21:18.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:21:18.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:21:18.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:21:18.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:21:18.646 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:21:18.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.647 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:21:18.665 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.666 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:21:18.667 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:21:18.668 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.669 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:21:18.670 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:21:18.670 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:21:18.670 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:21:18.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.671 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:21:18.705 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:21:18.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.709 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:21:18.734 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:21:18.734 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:21:18.735 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:21:18.735 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:21:18.735 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:21:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.736 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:21:18.736 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:21:18.736 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:21:18.737 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:21:18.737 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:21:18.737 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:21:18.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.848 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:21:18.864 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:21:18.864 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:21:18.865 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:21:18.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:18.866 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:21:18.895 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:21:19.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 Processor -- start 2020-11-16T22:21:19.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 -- start start 2020-11-16T22:21:19.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc9700669a0 0x7fc970066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc970061e60 0x7fc9701442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9700abf90 con 0x7fc9700acc10 2020-11-16T22:21:19.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9700ac230 con 0x7fc9700669a0 2020-11-16T22:21:19.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc975eec700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9700ac4d0 con 0x7fc970061e60 2020-11-16T22:21:19.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc970061e60 0x7fc9701442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc970061e60 0x7fc9701442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34080/0 (socket says 172.21.15.185:34080) 2020-11-16T22:21:19.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 -- 172.21.15.185:0/3127639716 learned_addr learned my addr 172.21.15.185:0/3127639716 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:19.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc974eea700 1 --2- 172.21.15.185:0/3127639716 >> v2:172.21.15.96:3300/0 conn(0x7fc9700669a0 0x7fc970066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 -- 172.21.15.185:0/3127639716 >> v2:172.21.15.96:3300/0 conn(0x7fc9700669a0 msgr2=0x7fc970066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:19.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 --2- 172.21.15.185:0/3127639716 >> v2:172.21.15.96:3300/0 conn(0x7fc9700669a0 0x7fc970066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 -- 172.21.15.185:0/3127639716 >> v1:172.21.15.96:6789/0 conn(0x7fc9700acc10 legacy=0x7fc970147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 -- 172.21.15.185:0/3127639716 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700ac770 con 0x7fc970061e60 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.034+0000 7fc9756eb700 1 --2- 172.21.15.185:0/3127639716 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc9701442f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fc9700b2c50 tx=0x7fc968009290).ready entity=mon.1 client_cookie=fa23f05ab265de06 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc9677fe700 1 -- 172.21.15.185:0/3127639716 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96800bbc0 con 0x7fc970061e60 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc9677fe700 1 -- 172.21.15.185:0/3127639716 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96800bd20 con 0x7fc970061e60 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc9677fe700 1 -- 172.21.15.185:0/3127639716 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011610 con 0x7fc970061e60 2020-11-16T22:21:19.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- 172.21.15.185:0/3127639716 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 msgr2=0x7fc9701442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- 172.21.15.185:0/3127639716 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc9701442f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fc9700b2c50 tx=0x7fc968009290).stop 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- 172.21.15.185:0/3127639716 shutdown_connections 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- 172.21.15.185:0/3127639716 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc9701442f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- 172.21.15.185:0/3127639716 >> v2:172.21.15.96:3300/0 conn(0x7fc9700669a0 0x7fc970066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- 172.21.15.185:0/3127639716 >> 172.21.15.185:0/3127639716 conn(0x7fc970060130 msgr2=0x7fc970060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- 172.21.15.185:0/3127639716 shutdown_connections 2020-11-16T22:21:19.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- 172.21.15.185:0/3127639716 wait complete. 2020-11-16T22:21:19.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 Processor -- start 2020-11-16T22:21:19.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- start start 2020-11-16T22:21:19.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc970147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 0x7fc970153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970147490 0x7fc970153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970144030 con 0x7fc970147490 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b08f0 con 0x7fc970061e60 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc975eec700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc970143850 con 0x7fc9700669a0 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc967fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 0x7fc970153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc970147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc967fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 0x7fc970153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55758/0 (socket says 172.21.15.185:55758) 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc967fff700 1 -- 172.21.15.185:0/544305309 learned_addr learned my addr 172.21.15.185:0/544305309 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc9756eb700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970147490 0x7fc970153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 -- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 msgr2=0x7fc970153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 0x7fc970153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 -- 172.21.15.185:0/544305309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970147490 msgr2=0x7fc970153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970147490 0x7fc970153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 -- 172.21.15.185:0/544305309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970143af0 con 0x7fc970061e60 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.038+0000 7fc974eea700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc970147e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fc96c00eba0 tx=0x7fc96c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00c820 con 0x7fc970061e60 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00c980 con 0x7fc970061e60 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc970143d90 con 0x7fc970061e60 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00cd00 con 0x7fc970061e60 2020-11-16T22:21:19.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9701482f0 con 0x7fc970061e60 2020-11-16T22:21:19.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97005d200 con 0x7fc970061e60 2020-11-16T22:21:19.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc96c012600 con 0x7fc970061e60 2020-11-16T22:21:19.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 0x7fc948057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc96c06dfc0 con 0x7fc970061e60 2020-11-16T22:21:19.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc967fff700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 0x7fc948057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.042+0000 7fc967fff700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 0x7fc948057520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fc970147a20 tx=0x7fc960009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.050+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc96c044220 con 0x7fc970061e60 2020-11-16T22:21:19.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.202+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc970067270 con 0x7fc970061e60 2020-11-16T22:21:19.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc964ff9700 1 -- 172.21.15.185:0/544305309 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16829 (secure 0 0 0) 0x7fc96c00ce60 con 0x7fc970061e60 2020-11-16T22:21:19.197 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:19.197 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:15.557108+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:21:19.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 msgr2=0x7fc948057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 0x7fc948057520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fc970147a20 tx=0x7fc960009040).stop 2020-11-16T22:21:19.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 msgr2=0x7fc970147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc970147e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fc96c00eba0 tx=0x7fc96c006df0).stop 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 shutdown_connections 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc948055060 0x7fc948057520 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc970061e60 0x7fc970147e00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc970147490 0x7fc970153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 --2- 172.21.15.185:0/544305309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc9700669a0 0x7fc970153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 >> 172.21.15.185:0/544305309 conn(0x7fc970060130 msgr2=0x7fc970142320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:19.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 shutdown_connections 2020-11-16T22:21:19.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.206+0000 7fc975eec700 1 -- 172.21.15.185:0/544305309 wait complete. 2020-11-16T22:21:19.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:21:11.777337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:21:12.541132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:21:19.213 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:21:19.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.350+0000 7fcc68b62700 1 Processor -- start 2020-11-16T22:21:19.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.350+0000 7fcc68b62700 1 -- start start 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcc640b7440 0x7fcc640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 0x7fcc64144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc64144d50 con 0x7fcc6413dea0 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640ad490 con 0x7fcc640afc60 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640ad730 con 0x7fcc640b7440 2020-11-16T22:21:19.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcc640b7440 0x7fcc640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcc640b7440 0x7fcc640b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34092/0 (socket says 172.21.15.185:34092) 2020-11-16T22:21:19.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 -- 172.21.15.185:0/4043206723 learned_addr learned my addr 172.21.15.185:0/4043206723 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:19.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc61d9b700 1 --2- 172.21.15.185:0/4043206723 >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 0x7fcc64144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 -- 172.21.15.185:0/4043206723 >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 msgr2=0x7fcc64144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 --2- 172.21.15.185:0/4043206723 >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 0x7fcc64144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 -- 172.21.15.185:0/4043206723 >> v1:172.21.15.96:6789/0 conn(0x7fcc6413dea0 legacy=0x7fcc640ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 -- 172.21.15.185:0/4043206723 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640ad9d0 con 0x7fcc640b7440 2020-11-16T22:21:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc61d9b700 1 --2- 172.21.15.185:0/4043206723 >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 0x7fcc64144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6259c700 1 --2- 172.21.15.185:0/4043206723 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b7820 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fcc64062650 tx=0x7fcc58006eb0).ready entity=mon.1 client_cookie=8ef44378512695c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6159a700 1 -- 172.21.15.185:0/4043206723 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580098d0 con 0x7fcc640b7440 2020-11-16T22:21:19.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6159a700 1 -- 172.21.15.185:0/4043206723 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58009a30 con 0x7fcc640b7440 2020-11-16T22:21:19.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc6159a700 1 -- 172.21.15.185:0/4043206723 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58009db0 con 0x7fcc640b7440 2020-11-16T22:21:19.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- 172.21.15.185:0/4043206723 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 msgr2=0x7fcc640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 --2- 172.21.15.185:0/4043206723 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b7820 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fcc64062650 tx=0x7fcc58006eb0).stop 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- 172.21.15.185:0/4043206723 shutdown_connections 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 --2- 172.21.15.185:0/4043206723 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b7820 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 --2- 172.21.15.185:0/4043206723 >> v2:172.21.15.96:3300/0 conn(0x7fcc640afc60 0x7fcc64144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- 172.21.15.185:0/4043206723 >> 172.21.15.185:0/4043206723 conn(0x7fcc64067860 msgr2=0x7fcc64067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- 172.21.15.185:0/4043206723 shutdown_connections 2020-11-16T22:21:19.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.354+0000 7fcc68b62700 1 -- 172.21.15.185:0/4043206723 wait complete. 2020-11-16T22:21:19.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 Processor -- start 2020-11-16T22:21:19.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 -- start start 2020-11-16T22:21:19.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc640afc60 0x7fcc6413e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc6413dea0 0x7fcc640b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153940 con 0x7fcc6413dea0 2020-11-16T22:21:19.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153be0 con 0x7fcc640b7440 2020-11-16T22:21:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc68b62700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153e80 con 0x7fcc640afc60 2020-11-16T22:21:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc61d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc6259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc640afc60 0x7fcc6413e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc61d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34098/0 (socket says 172.21.15.185:34098) 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.358+0000 7fcc61d9b700 1 -- 172.21.15.185:0/739383359 learned_addr learned my addr 172.21.15.185:0/739383359 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc62d9d700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc6413dea0 0x7fcc640b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 -- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc640afc60 msgr2=0x7fcc6413e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc640afc60 0x7fcc6413e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 -- 172.21.15.185:0/739383359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc6413dea0 msgr2=0x7fcc640b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc6413dea0 0x7fcc640b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:19.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 -- 172.21.15.185:0/739383359 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64154120 con 0x7fcc640b7440 2020-11-16T22:21:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc61d9b700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fcc5400db60 tx=0x7fcc54008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54009730 con 0x7fcc640b7440 2020-11-16T22:21:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc54009890 con 0x7fcc640b7440 2020-11-16T22:21:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6413e9a0 con 0x7fcc640b7440 2020-11-16T22:21:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54009c50 con 0x7fcc640b7440 2020-11-16T22:21:19.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc640b2490 con 0x7fcc640b7440 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcc54013480 con 0x7fcc640b7440 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc640adbb0 con 0x7fcc640b7440 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 0x7fcc38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcc5406e170 con 0x7fcc640b7440 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc6259c700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 0x7fcc38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.362+0000 7fcc6259c700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 0x7fcc38057520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fcc58006ef0 tx=0x7fcc58015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:19.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.370+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcc640adbb0 con 0x7fcc640b7440 2020-11-16T22:21:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.522+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcc64154300 con 0x7fcc640b7440 2020-11-16T22:21:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.522+0000 7fcc527fc700 1 -- 172.21.15.185:0/739383359 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcc64154300 con 0x7fcc640b7440 2020-11-16T22:21:19.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 msgr2=0x7fcc38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 0x7fcc38057520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fcc58006ef0 tx=0x7fcc58015000).stop 2020-11-16T22:21:20.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 msgr2=0x7fcc640b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fcc5400db60 tx=0x7fcc54008040).stop 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 shutdown_connections 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcc38055060 0x7fcc38057520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc6413dea0 0x7fcc640b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcc640afc60 0x7fcc6413e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 --2- 172.21.15.185:0/739383359 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcc640b7440 0x7fcc640b3e10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 >> 172.21.15.185:0/739383359 conn(0x7fcc64067860 msgr2=0x7fcc640ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:20.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 shutdown_connections 2020-11-16T22:21:20.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:19.526+0000 7fcc68b62700 1 -- 172.21.15.185:0/739383359 wait complete. 2020-11-16T22:21:20.168 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:21:20.168 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:21:20.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 Processor -- start 2020-11-16T22:21:20.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 -- start start 2020-11-16T22:21:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd78c0b5c40 0x7fd78c0b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 0x7fd78c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c144450 con 0x7fd78c0ba800 2020-11-16T22:21:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c1446e0 con 0x7fd78c144000 2020-11-16T22:21:20.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791d8a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c144980 con 0x7fd78c0b5c40 2020-11-16T22:21:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd790d88700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fd78c0b5c40 0x7fd78c0b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd78bfff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fd78c0ba800 0x7fd78c13dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55776/0 (socket says 172.21.15.185:55776) 2020-11-16T22:21:20.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd78bfff700 1 -- 172.21.15.185:0/1596872151 learned_addr learned my addr 172.21.15.185:0/1596872151 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:20.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.314+0000 7fd791589700 1 --2- 172.21.15.185:0/1596872151 >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 0x7fd78c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd790d88700 1 -- 172.21.15.185:0/1596872151 >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 msgr2=0x7fd78c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd790d88700 1 --2- 172.21.15.185:0/1596872151 >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 0x7fd78c0b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd790d88700 1 -- 172.21.15.185:0/1596872151 >> v1:172.21.15.185:6789/0 conn(0x7fd78c0ba800 legacy=0x7fd78c13dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd790d88700 1 -- 172.21.15.185:0/1596872151 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c144c20 con 0x7fd78c0b5c40 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd78b7fe700 1 -- 172.21.15.185:0/1596872151 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341373445 0 0) 0x7fd78c144450 con 0x7fd78c0ba800 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791589700 1 --2- 172.21.15.185:0/1596872151 >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 0x7fd78c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd790d88700 1 --2- 172.21.15.185:0/1596872151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0b5c40 0x7fd78c0b6020 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd78c0ba7d0 tx=0x7fd780006df0).ready entity=mon.1 client_cookie=6b7ba0b4d4fac9d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:20.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd78b7fe700 1 -- 172.21.15.185:0/1596872151 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7800097b0 con 0x7fd78c0b5c40 2020-11-16T22:21:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd78b7fe700 1 -- 172.21.15.185:0/1596872151 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780009910 con 0x7fd78c0b5c40 2020-11-16T22:21:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd78b7fe700 1 -- 172.21.15.185:0/1596872151 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780009c90 con 0x7fd78c0b5c40 2020-11-16T22:21:20.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 -- 172.21.15.185:0/1596872151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0b5c40 msgr2=0x7fd78c0b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1596872151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0b5c40 0x7fd78c0b6020 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fd78c0ba7d0 tx=0x7fd780006df0).stop 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 -- 172.21.15.185:0/1596872151 shutdown_connections 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1596872151 >> v2:172.21.15.96:3300/0 conn(0x7fd78c144000 0x7fd78c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1596872151 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0b5c40 0x7fd78c0b6020 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 -- 172.21.15.185:0/1596872151 >> 172.21.15.185:0/1596872151 conn(0x7fd78c05ba80 msgr2=0x7fd78c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 -- 172.21.15.185:0/1596872151 shutdown_connections 2020-11-16T22:21:20.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.318+0000 7fd791d8a700 1 -- 172.21.15.185:0/1596872151 wait complete. 2020-11-16T22:21:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 Processor -- start 2020-11-16T22:21:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 -- start start 2020-11-16T22:21:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 0x7fd78c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c144000 0x7fd78c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0ae140 con 0x7fd78c144000 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0af690 con 0x7fd78c0ba800 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0af3f0 con 0x7fd78c0b5c40 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd790d88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 0x7fd78c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34112/0 (socket says 172.21.15.185:34112) 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd790d88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 0x7fd78c0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55786/0 (socket says 172.21.15.185:55786) 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 -- 172.21.15.185:0/1919452635 learned_addr learned my addr 172.21.15.185:0/1919452635 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791589700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c144000 0x7fd78c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 -- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 msgr2=0x7fd78c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 0x7fd78c0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 -- 172.21.15.185:0/1919452635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c144000 msgr2=0x7fd78c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c144000 0x7fd78c0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 -- 172.21.15.185:0/1919452635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c153320 con 0x7fd78c0ba800 2020-11-16T22:21:20.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd78bfff700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fd77c00c710 tx=0x7fd77c007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:20.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0072e0 con 0x7fd78c0ba800 2020-11-16T22:21:20.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c007440 con 0x7fd78c0ba800 2020-11-16T22:21:20.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c005800 con 0x7fd78c0ba800 2020-11-16T22:21:20.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.322+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c1535c0 con 0x7fd78c0ba800 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c1540f0 con 0x7fd78c0ba800 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c153750 con 0x7fd78c0ba800 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fd77c010070 con 0x7fd78c0ba800 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd788ff9700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 0x7fd764057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd77c06d460 con 0x7fd78c0ba800 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd790d88700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 0x7fd764057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:20.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.326+0000 7fd790d88700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 0x7fd764057520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fd780006e30 tx=0x7fd780015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:20.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.334+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fd77c0436c0 con 0x7fd78c0ba800 2020-11-16T22:21:20.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.490+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd78c05d200 con 0x7fd78c0ba800 2020-11-16T22:21:20.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.490+0000 7fd788ff9700 1 -- 172.21.15.185:0/1919452635 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd77c007c00 con 0x7fd78c0ba800 2020-11-16T22:21:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 msgr2=0x7fd764057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 0x7fd764057520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fd780006e30 tx=0x7fd780015000).stop 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 msgr2=0x7fd78c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fd77c00c710 tx=0x7fd77c007ac0).stop 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 shutdown_connections 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fd764055060 0x7fd764057520 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c144000 0x7fd78c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fd78c0b5c40 0x7fd78c0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 --2- 172.21.15.185:0/1919452635 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fd78c0ba800 0x7fd78c153a60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 >> 172.21.15.185:0/1919452635 conn(0x7fd78c05ba80 msgr2=0x7fd78c13d320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 shutdown_connections 2020-11-16T22:21:20.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:20.494+0000 7fd791d8a700 1 -- 172.21.15.185:0/1919452635 wait complete. 2020-11-16T22:21:20.932 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:20.933 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:21:21.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.078+0000 7fbee27f4700 1 Processor -- start 2020-11-16T22:21:21.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.078+0000 7fbee27f4700 1 -- start start 2020-11-16T22:21:21.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.078+0000 7fbee27f4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbedc13e1f0 0x7fbedc0666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0b7c20 con 0x7fbedc0b3570 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0b3a80 con 0x7fbedc13de10 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0b3d20 con 0x7fbedc13e1f0 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee17f2700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbedc13de10 0x7fbedc0b7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55788/0 (socket says 172.21.15.185:55788) 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee17f2700 1 -- 172.21.15.185:0/3486408907 learned_addr learned my addr 172.21.15.185:0/3486408907 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:21.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee0ff1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbedc13e1f0 0x7fbedc0666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbed3fff700 1 -- 172.21.15.185:0/3486408907 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057995800 0 0) 0x7fbedc0b3a80 con 0x7fbedc13de10 2020-11-16T22:21:21.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbed3fff700 1 -- 172.21.15.185:0/3486408907 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec0003240 con 0x7fbedc13de10 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee0ff1700 1 -- 172.21.15.185:0/3486408907 >> v1:172.21.15.185:6789/0 conn(0x7fbedc13de10 legacy=0x7fbedc0b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee0ff1700 1 -- 172.21.15.185:0/3486408907 >> v1:172.21.15.96:6789/0 conn(0x7fbedc0b3570 legacy=0x7fbedc0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee0ff1700 1 -- 172.21.15.185:0/3486408907 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc0b3fc0 con 0x7fbedc13e1f0 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee0ff1700 1 --2- 172.21.15.185:0/3486408907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc13e1f0 0x7fbedc0666b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fbecc00b9a0 tx=0x7fbecc006df0).ready entity=mon.1 client_cookie=bc1f19324ae5b34 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbed3fff700 1 -- 172.21.15.185:0/3486408907 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0097b0 con 0x7fbedc13e1f0 2020-11-16T22:21:21.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbed3fff700 1 -- 172.21.15.185:0/3486408907 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc009910 con 0x7fbedc13e1f0 2020-11-16T22:21:21.076 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbed3fff700 1 -- 172.21.15.185:0/3486408907 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc009c90 con 0x7fbedc13e1f0 2020-11-16T22:21:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- 172.21.15.185:0/3486408907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc13e1f0 msgr2=0x7fbedc0666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 --2- 172.21.15.185:0/3486408907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc13e1f0 0x7fbedc0666b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7fbecc00b9a0 tx=0x7fbecc006df0).stop 2020-11-16T22:21:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- 172.21.15.185:0/3486408907 shutdown_connections 2020-11-16T22:21:21.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 --2- 172.21.15.185:0/3486408907 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc13e1f0 0x7fbedc0666b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- 172.21.15.185:0/3486408907 >> 172.21.15.185:0/3486408907 conn(0x7fbedc064b90 msgr2=0x7fbedc063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- 172.21.15.185:0/3486408907 shutdown_connections 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.082+0000 7fbee27f4700 1 -- 172.21.15.185:0/3486408907 wait complete. 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 Processor -- start 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- start start 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 0x7fbedc157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbedc157510 0x7fbedc0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0ae7a0 con 0x7fbedc13de10 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc156920 con 0x7fbedc0b3570 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0af620 con 0x7fbedc157510 2020-11-16T22:21:21.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34124/0 (socket says 172.21.15.185:34124) 2020-11-16T22:21:21.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 -- 172.21.15.185:0/832543933 learned_addr learned my addr 172.21.15.185:0/832543933 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:21.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee1ff3700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbedc157510 0x7fbedc0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee0ff1700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 0x7fbedc157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 -- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbedc157510 msgr2=0x7fbedc0ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbedc157510 0x7fbedc0ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 -- 172.21.15.185:0/832543933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 msgr2=0x7fbedc157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 0x7fbedc157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 -- 172.21.15.185:0/832543933 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc156b80 con 0x7fbedc0b3570 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee0ff1700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 0x7fbedc157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:21.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee17f2700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fbed800c6e0 tx=0x7fbed8007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80072e0 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8007440 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0af8c0 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8005800 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.086+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc13faa0 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc05d200 con 0x7fbedc0b3570 2020-11-16T22:21:21.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbed801a060 con 0x7fbedc0b3570 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbed17fa700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 0x7fbeb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbed806d580 con 0x7fbedc0b3570 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbee0ff1700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 0x7fbeb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.090+0000 7fbee0ff1700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 0x7fbeb4057520 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fbedc0684d0 tx=0x7fbecc015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.102+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbed80437e0 con 0x7fbedc0b3570 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.250+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbedc156d10 con 0x7fbedc0b3570 2020-11-16T22:21:21.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.250+0000 7fbed17fa700 1 -- 172.21.15.185:0/832543933 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbed8018020 con 0x7fbedc0b3570 2020-11-16T22:21:21.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 msgr2=0x7fbeb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 0x7fbeb4057520 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fbedc0684d0 tx=0x7fbecc015000).stop 2020-11-16T22:21:21.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 msgr2=0x7fbedc0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fbed800c6e0 tx=0x7fbed8007ad0).stop 2020-11-16T22:21:21.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 shutdown_connections 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbeb4055060 0x7fbeb4057520 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbedc0b3570 0x7fbedc0b6180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbedc157510 0x7fbedc0ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 --2- 172.21.15.185:0/832543933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbedc13de10 0x7fbedc157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 >> 172.21.15.185:0/832543933 conn(0x7fbedc064b90 msgr2=0x7fbedc13d540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:21.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 shutdown_connections 2020-11-16T22:21:21.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.254+0000 7fbee27f4700 1 -- 172.21.15.185:0/832543933 wait complete. 2020-11-16T22:21:21.696 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:21.696 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:21:21.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 Processor -- start 2020-11-16T22:21:21.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 -- start start 2020-11-16T22:21:21.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f54fc144c00 0x7f54fc0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54fc0ade00 0x7f54fc142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:21.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0b02e0 con 0x7f54fc0b74b0 2020-11-16T22:21:21.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc143fa0 con 0x7f54fc144c00 2020-11-16T22:21:21.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5502d48700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0b0580 con 0x7f54fc0ade00 2020-11-16T22:21:21.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54fc0ade00 0x7f54fc142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54fc0ade00 0x7f54fc142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34136/0 (socket says 172.21.15.185:34136) 2020-11-16T22:21:21.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f54fbfff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f54fc144c00 0x7f54fc0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:21.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 -- 172.21.15.185:0/1904741752 learned_addr learned my addr 172.21.15.185:0/1904741752 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:21.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f5500ae4700 1 --1- 172.21.15.185:0/1904741752 >> v1:172.21.15.96:6789/0 conn(0x7f54fc0b74b0 0x7f54fc0ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:41866/0 (socket says 172.21.15.185:41866) 2020-11-16T22:21:21.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 -- 172.21.15.185:0/1904741752 >> v2:172.21.15.96:3300/0 conn(0x7f54fc144c00 msgr2=0x7f54fc0afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 --2- 172.21.15.185:0/1904741752 >> v2:172.21.15.96:3300/0 conn(0x7f54fc144c00 0x7f54fc0afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 -- 172.21.15.185:0/1904741752 >> v1:172.21.15.96:6789/0 conn(0x7f54fc0b74b0 legacy=0x7f54fc0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 -- 172.21.15.185:0/1904741752 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0ad250 con 0x7f54fc0ade00 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f55012e5700 1 --2- 172.21.15.185:0/1904741752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc0ade00 0x7f54fc142ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f54fc0b7db0 tx=0x7f54f4009290).ready entity=mon.1 client_cookie=1a5cd9c6a2482538 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f54fb7fe700 1 -- 172.21.15.185:0/1904741752 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400bbc0 con 0x7f54fc0ade00 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f54fb7fe700 1 -- 172.21.15.185:0/1904741752 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f400bd20 con 0x7f54fc0ade00 2020-11-16T22:21:21.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.838+0000 7f54fb7fe700 1 -- 172.21.15.185:0/1904741752 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4011610 con 0x7f54fc0ade00 2020-11-16T22:21:21.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- 172.21.15.185:0/1904741752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc0ade00 msgr2=0x7f54fc142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- 172.21.15.185:0/1904741752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc0ade00 0x7f54fc142ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f54fc0b7db0 tx=0x7f54f4009290).stop 2020-11-16T22:21:22.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- 172.21.15.185:0/1904741752 shutdown_connections 2020-11-16T22:21:22.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- 172.21.15.185:0/1904741752 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc0ade00 0x7f54fc142ef0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- 172.21.15.185:0/1904741752 >> v2:172.21.15.96:3300/0 conn(0x7f54fc144c00 0x7f54fc0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- 172.21.15.185:0/1904741752 >> 172.21.15.185:0/1904741752 conn(0x7f54fc067860 msgr2=0x7f54fc067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:22.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- 172.21.15.185:0/1904741752 shutdown_connections 2020-11-16T22:21:22.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- 172.21.15.185:0/1904741752 wait complete. 2020-11-16T22:21:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 Processor -- start 2020-11-16T22:21:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- start start 2020-11-16T22:21:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54fc0ade00 0x7f54fc142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54fc0b74b0 0x7f54fc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1470d0 con 0x7f54fc0ade00 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc147330 con 0x7f54fc144c00 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5502d48700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc145440 con 0x7f54fc0b74b0 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34142/0 (socket says 172.21.15.185:34142) 2020-11-16T22:21:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 -- 172.21.15.185:0/3569630756 learned_addr learned my addr 172.21.15.185:0/3569630756 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f54fbfff700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54fc0b74b0 0x7f54fc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f5500ae4700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54fc0ade00 0x7f54fc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 -- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54fc0b74b0 msgr2=0x7f54fc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54fc0b74b0 0x7f54fc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 -- 172.21.15.185:0/3569630756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54fc0ade00 msgr2=0x7f54fc142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54fc0ade00 0x7f54fc142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 -- 172.21.15.185:0/3569630756 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc1456e0 con 0x7f54fc144c00 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f55012e5700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f54fc1440e0 tx=0x7f54f4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.842+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400bbc0 con 0x7f54fc144c00 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f400be90 con 0x7f54fc144c00 2020-11-16T22:21:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc0b5940 con 0x7f54fc144c00 2020-11-16T22:21:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4010cf0 con 0x7f54fc144c00 2020-11-16T22:21:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc145be0 con 0x7f54fc144c00 2020-11-16T22:21:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc145dd0 con 0x7f54fc144c00 2020-11-16T22:21:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f54f4017070 con 0x7f54fc144c00 2020-11-16T22:21:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f54f8ff9700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 0x7f54d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54f4070670 con 0x7f54fc144c00 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.846+0000 7f5500ae4700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 0x7f54d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.850+0000 7f5500ae4700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 0x7f54d4057520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f54ec007270 tx=0x7f54ec007b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:21.854+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f54f4046a10 con 0x7f54fc144c00 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.006+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54fc145dd0 con 0x7f54fc144c00 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f54f8ff9700 1 -- 172.21.15.185:0/3569630756 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f54fc145dd0 con 0x7f54fc144c00 2020-11-16T22:21:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 msgr2=0x7f54d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 0x7f54d4057520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f54ec007270 tx=0x7f54ec007b80).stop 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 msgr2=0x7f54fc13d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f54fc1440e0 tx=0x7f54f4016000).stop 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 shutdown_connections 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54d4055060 0x7f54d4057520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54fc144c00 0x7f54fc13d350 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54fc0ade00 0x7f54fc142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 --2- 172.21.15.185:0/3569630756 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54fc0b74b0 0x7f54fc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 >> 172.21.15.185:0/3569630756 conn(0x7f54fc067860 msgr2=0x7f54fc0b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:22.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 shutdown_connections 2020-11-16T22:21:22.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.010+0000 7f5502d48700 1 -- 172.21.15.185:0/3569630756 wait complete. 2020-11-16T22:21:22.453 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:21:22.454 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:21:22.454 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:22.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 Processor -- start 2020-11-16T22:21:22.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 -- start start 2020-11-16T22:21:22.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 0x7f661814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66180abf90 con 0x7f66180acc10 2020-11-16T22:21:22.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66180ac230 con 0x7f66180669a0 2020-11-16T22:21:22.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661f58b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180ac4d0 con 0x7f6618061e60 2020-11-16T22:21:22.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661ed8a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 0x7f661814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661e589700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f66180669a0 0x7f6618066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55818/0 (socket says 172.21.15.185:55818) 2020-11-16T22:21:22.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661e589700 1 -- 172.21.15.185:0/3919230871 learned_addr learned my addr 172.21.15.185:0/3919230871 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:22.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.594+0000 7f661d587700 1 -- 172.21.15.185:0/3919230871 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955114308 0 0) 0x7f66180ac230 con 0x7f66180669a0 2020-11-16T22:21:22.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661d587700 1 -- 172.21.15.185:0/3919230871 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6600003240 con 0x7f66180669a0 2020-11-16T22:21:22.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661ed8a700 1 -- 172.21.15.185:0/3919230871 >> v1:172.21.15.185:6789/0 conn(0x7f66180669a0 legacy=0x7f6618066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661ed8a700 1 -- 172.21.15.185:0/3919230871 >> v1:172.21.15.96:6789/0 conn(0x7f66180acc10 legacy=0x7f66181432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661ed8a700 1 -- 172.21.15.185:0/3919230871 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180ac770 con 0x7f6618061e60 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661ed8a700 1 --2- 172.21.15.185:0/3919230871 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 0x7f661814e8f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f6610009cc0 tx=0x7f66100091d0).ready entity=mon.2 client_cookie=c84394bc78a80d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661d587700 1 -- 172.21.15.185:0/3919230871 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661000b9d0 con 0x7f6618061e60 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661d587700 1 -- 172.21.15.185:0/3919230871 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661000bb30 con 0x7f6618061e60 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661d587700 1 -- 172.21.15.185:0/3919230871 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661000beb0 con 0x7f6618061e60 2020-11-16T22:21:22.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 -- 172.21.15.185:0/3919230871 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 msgr2=0x7f661814e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 --2- 172.21.15.185:0/3919230871 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 0x7f661814e8f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f6610009cc0 tx=0x7f66100091d0).stop 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 -- 172.21.15.185:0/3919230871 shutdown_connections 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 --2- 172.21.15.185:0/3919230871 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6618061e60 0x7f661814e8f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 -- 172.21.15.185:0/3919230871 >> 172.21.15.185:0/3919230871 conn(0x7f6618060130 msgr2=0x7f6618060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 -- 172.21.15.185:0/3919230871 shutdown_connections 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.598+0000 7f661f58b700 1 -- 172.21.15.185:0/3919230871 wait complete. 2020-11-16T22:21:22.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 Processor -- start 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 -- start start 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6618061e60 0x7f6618157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6618139470 0x7f6618142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661813b050 con 0x7f6618061e60 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618141720 con 0x7f6618139470 2020-11-16T22:21:22.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661f58b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66181419c0 con 0x7f66180669a0 2020-11-16T22:21:22.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55830/0 (socket says 172.21.15.185:55830) 2020-11-16T22:21:22.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 -- 172.21.15.185:0/2037941705 learned_addr learned my addr 172.21.15.185:0/2037941705 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:22.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661e589700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6618061e60 0x7f6618157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661ed8a700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6618139470 0x7f6618142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:22.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 -- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6618139470 msgr2=0x7f6618142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6618139470 0x7f6618142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 -- 172.21.15.185:0/2037941705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6618061e60 msgr2=0x7f6618157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6618061e60 0x7f6618157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 -- 172.21.15.185:0/2037941705 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618141c60 con 0x7f66180669a0 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.602+0000 7f661dd88700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f660800c760 tx=0x7f6608007a70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800e560 con 0x7f66180669a0 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800e6c0 con 0x7f66180669a0 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618141f00 con 0x7f66180669a0 2020-11-16T22:21:22.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800ea80 con 0x7f66180669a0 2020-11-16T22:21:23.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618138a00 con 0x7f66180669a0 2020-11-16T22:21:23.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618142090 con 0x7f66180669a0 2020-11-16T22:21:23.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6608019030 con 0x7f66180669a0 2020-11-16T22:21:23.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 0x7f65f0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:23.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f660806d2c0 con 0x7f66180669a0 2020-11-16T22:21:23.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.606+0000 7f661e589700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 0x7f65f0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:23.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.610+0000 7f661e589700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 0x7f65f0057520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f66140055a0 tx=0x7f6614009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:23.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.618+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6608043520 con 0x7f66180669a0 2020-11-16T22:21:23.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.814+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f661805d200 con 0x7f66180669a0 2020-11-16T22:21:23.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.814+0000 7f66067fc700 1 -- 172.21.15.185:0/2037941705 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f6608046b40 con 0x7f66180669a0 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 msgr2=0x7f65f0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 0x7f65f0057520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f66140055a0 tx=0x7f6614009040).stop 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 msgr2=0x7f66181423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f660800c760 tx=0x7f6608007a70).stop 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 shutdown_connections 2020-11-16T22:21:23.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f65f0055060 0x7f65f0057520 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:23.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6618139470 0x7f6618142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:23.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6618061e60 0x7f6618157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:23.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 --2- 172.21.15.185:0/2037941705 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f66180669a0 0x7f66181423a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:23.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 >> 172.21.15.185:0/2037941705 conn(0x7f6618060130 msgr2=0x7f6618066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:23.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 shutdown_connections 2020-11-16T22:21:23.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:22.818+0000 7f661f58b700 1 -- 172.21.15.185:0/2037941705 wait complete. 2020-11-16T22:21:23.411 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 83 2020-11-16T22:21:23.411 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:23.411 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":78},{"gid":7080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:16.571738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:23.713 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:21:13.541645+0000', 'modified': '2020-11-16T22:21:16.571738+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3098481638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3098481638}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3098481638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:21:23.714 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:21:23.715 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:21:23.849 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7083,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.307252161999999,"mdsmap_epoch":83,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":15.557987161} 2020-11-16T22:21:23.850 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7083, "mdsmap_epoch": 83, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 10.307252162, "state": "up:active", "uptime": 15.557987161, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:21:23.850 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:21:23.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:23.851 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:21:23.870 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:21:23.870 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:21:23.991 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:21:23.991 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:21:24.016 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:21:24.016 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:21:24.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:21:24.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:21:24.025 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:21:24.056 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:21:24.191 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.198+0000 7f2ab359e700 1 Processor -- start 2020-11-16T22:21:24.192 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.198+0000 7f2ab359e700 1 -- start start 2020-11-16T22:21:24.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ab40b3020 0x7f2ab40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2ab4141150 0x7f2ab41415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab40acdf0 con 0x7f2ab40abef0 2020-11-16T22:21:24.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab41462b0 con 0x7f2ab40b3020 2020-11-16T22:21:24.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab40ad2c0 con 0x7f2ab4141150 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2ab4141150 0x7f2ab41415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f2ab4141150 0x7f2ab41415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34160/0 (socket says 172.21.15.185:34160) 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 -- 172.21.15.185:0/1318235744 learned_addr learned my addr 172.21.15.185:0/1318235744 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab259c700 1 --2- 172.21.15.185:0/1318235744 >> v2:172.21.15.96:3300/0 conn(0x7f2ab40b3020 0x7f2ab40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 -- 172.21.15.185:0/1318235744 >> v2:172.21.15.96:3300/0 conn(0x7f2ab40b3020 msgr2=0x7f2ab40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:24.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 --2- 172.21.15.185:0/1318235744 >> v2:172.21.15.96:3300/0 conn(0x7f2ab40b3020 0x7f2ab40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 -- 172.21.15.185:0/1318235744 >> v1:172.21.15.96:6789/0 conn(0x7f2ab40abef0 legacy=0x7f2ab4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:24.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 -- 172.21.15.185:0/1318235744 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40ad020 con 0x7f2ab4141150 2020-11-16T22:21:24.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab2d9d700 1 --2- 172.21.15.185:0/1318235744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab4141150 0x7f2ab41415a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f2ab40b2cb0 tx=0x7f2aac006eb0).ready entity=mon.1 client_cookie=57033738cc318154 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab159a700 1 -- 172.21.15.185:0/1318235744 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0098d0 con 0x7f2ab4141150 2020-11-16T22:21:24.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab159a700 1 -- 172.21.15.185:0/1318235744 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac009a30 con 0x7f2ab4141150 2020-11-16T22:21:24.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab159a700 1 -- 172.21.15.185:0/1318235744 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac009db0 con 0x7f2ab4141150 2020-11-16T22:21:24.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- 172.21.15.185:0/1318235744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab4141150 msgr2=0x7f2ab41415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 --2- 172.21.15.185:0/1318235744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab4141150 0x7f2ab41415a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f2ab40b2cb0 tx=0x7f2aac006eb0).stop 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- 172.21.15.185:0/1318235744 shutdown_connections 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 --2- 172.21.15.185:0/1318235744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab4141150 0x7f2ab41415a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 --2- 172.21.15.185:0/1318235744 >> v2:172.21.15.96:3300/0 conn(0x7f2ab40b3020 0x7f2ab40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- 172.21.15.185:0/1318235744 >> 172.21.15.185:0/1318235744 conn(0x7f2ab4060130 msgr2=0x7f2ab4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- 172.21.15.185:0/1318235744 shutdown_connections 2020-11-16T22:21:24.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- 172.21.15.185:0/1318235744 wait complete. 2020-11-16T22:21:24.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 Processor -- start 2020-11-16T22:21:24.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.202+0000 7f2ab359e700 1 -- start start 2020-11-16T22:21:24.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ab40abef0 0x7f2ab40b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ab4141150 0x7f2ab413ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4144650 con 0x7f2ab4141150 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab410f640 con 0x7f2ab40b3020 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab410f8e0 con 0x7f2ab40abef0 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34166/0 (socket says 172.21.15.185:34166) 2020-11-16T22:21:24.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 -- 172.21.15.185:0/2687060718 learned_addr learned my addr 172.21.15.185:0/2687060718 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab259c700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ab40abef0 0x7f2ab40b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab2d9d700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ab4141150 0x7f2ab413ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 -- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ab40abef0 msgr2=0x7f2ab40b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ab40abef0 0x7f2ab40b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 -- 172.21.15.185:0/2687060718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ab4141150 msgr2=0x7f2ab413ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ab4141150 0x7f2ab413ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 -- 172.21.15.185:0/2687060718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40afcb0 con 0x7f2ab40b3020 2020-11-16T22:21:24.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab1d9b700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f2a9c00c6e0 tx=0x7f2a9c007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c007020 con 0x7f2ab40b3020 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c007180 con 0x7f2ab40b3020 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab40b5ee0 con 0x7f2ab40b3020 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0064f0 con 0x7f2ab40b3020 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab40b0150 con 0x7f2ab40b3020 2020-11-16T22:21:24.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab40afe40 con 0x7f2ab40b3020 2020-11-16T22:21:24.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.206+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f2a9c010070 con 0x7f2ab40b3020 2020-11-16T22:21:24.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.210+0000 7f2aaa7fc700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 0x7f2a84057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.210+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a9c06d050 con 0x7f2ab40b3020 2020-11-16T22:21:24.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.210+0000 7f2ab259c700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 0x7f2a84057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.210+0000 7f2ab259c700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 0x7f2a84057520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f2aa400c890 tx=0x7f2aa4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.214+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f2a9c0432b0 con 0x7f2ab40b3020 2020-11-16T22:21:24.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.414+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f2ab405d200 con 0x7f2ab40b3020 2020-11-16T22:21:24.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.630+0000 7f2aaa7fc700 1 -- 172.21.15.185:0/2687060718 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v84) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2a9c0468d0 con 0x7f2ab40b3020 2020-11-16T22:21:24.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.634+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 msgr2=0x7f2a84057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.634+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 0x7f2a84057520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f2aa400c890 tx=0x7f2aa4008040).stop 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.634+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 msgr2=0x7f2ab4144c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.634+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f2a9c00c6e0 tx=0x7f2a9c007c50).stop 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 shutdown_connections 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2a84055060 0x7f2a84057520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ab4141150 0x7f2ab413ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2ab40abef0 0x7f2ab40b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 --2- 172.21.15.185:0/2687060718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f2ab40b3020 0x7f2ab4144c00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 >> 172.21.15.185:0/2687060718 conn(0x7f2ab4060130 msgr2=0x7f2ab4067890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:24.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 shutdown_connections 2020-11-16T22:21:24.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.638+0000 7f2ab359e700 1 -- 172.21.15.185:0/2687060718 wait complete. 2020-11-16T22:21:24.650 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:24.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.766+0000 7f0e7aa69700 1 Processor -- start 2020-11-16T22:21:24.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.766+0000 7f0e7aa69700 1 -- start start 2020-11-16T22:21:24.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 0x7f0e6c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e6c0ca630 0x7f0e6c03de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6c0424e0 con 0x7f0e6c0ca630 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6c042780 con 0x7f0e6c040650 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c042a20 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79a67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 0x7f0e6c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79a67700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 0x7f0e6c040a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34174/0 (socket says 172.21.15.185:34174) 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55852/0 (socket says 172.21.15.185:55852) 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79a67700 1 -- 172.21.15.185:0/2360899994 learned_addr learned my addr 172.21.15.185:0/2360899994 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7a268700 1 --2- 172.21.15.185:0/2360899994 >> v2:172.21.15.96:3300/0 conn(0x7f0e6c0ca630 0x7f0e6c03de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 -- 172.21.15.185:0/2360899994 >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 msgr2=0x7f0e6c040a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 --2- 172.21.15.185:0/2360899994 >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 0x7f0e6c040a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 -- 172.21.15.185:0/2360899994 >> v2:172.21.15.96:3300/0 conn(0x7f0e6c0ca630 msgr2=0x7f0e6c03de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:24.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 --2- 172.21.15.185:0/2360899994 >> v2:172.21.15.96:3300/0 conn(0x7f0e6c0ca630 0x7f0e6c03de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 -- 172.21.15.185:0/2360899994 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c042cc0 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e79266700 1 --2- 172.21.15.185:0/2360899994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f0e6c00cb30 tx=0x7f0e64006df0).ready entity=mon.2 client_cookie=3f52394a61167349 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e78a65700 1 -- 172.21.15.185:0/2360899994 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640097b0 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e78a65700 1 -- 172.21.15.185:0/2360899994 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64009910 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e78a65700 1 -- 172.21.15.185:0/2360899994 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64009c90 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2360899994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 msgr2=0x7f0e6c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2360899994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f0e6c00cb30 tx=0x7f0e64006df0).stop 2020-11-16T22:21:24.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2360899994 shutdown_connections 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2360899994 >> v2:172.21.15.96:3300/0 conn(0x7f0e6c0ca630 0x7f0e6c03de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2360899994 >> v2:172.21.15.185:3300/0 conn(0x7f0e6c040650 0x7f0e6c040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2360899994 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0d0c30 0x7f0e6c0ca140 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2360899994 >> 172.21.15.185:0/2360899994 conn(0x7f0e6c0441c0 msgr2=0x7f0e6c0099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2360899994 shutdown_connections 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.770+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2360899994 wait complete. 2020-11-16T22:21:24.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 Processor -- start 2020-11-16T22:21:24.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 -- start start 2020-11-16T22:21:24.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6c040650 0x7f0e6c046300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 0x7f0e6c0c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 0x7f0e6c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0ca3a0 con 0x7f0e6c040650 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0d4f70 con 0x7f0e6c0d0c30 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7aa69700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0d5210 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e79266700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 0x7f0e6c0c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 0x7f0e6c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e79266700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 0x7f0e6c0c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34182/0 (socket says 172.21.15.185:34182) 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e79266700 1 -- 172.21.15.185:0/2893486743 learned_addr learned my addr 172.21.15.185:0/2893486743 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:24.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e79a67700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6c040650 0x7f0e6c046300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 -- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 msgr2=0x7f0e6c0c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 0x7f0e6c0c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 -- 172.21.15.185:0/2893486743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6c040650 msgr2=0x7f0e6c046300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6c040650 0x7f0e6c046300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 -- 172.21.15.185:0/2893486743 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0e4090 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.774+0000 7f0e7a268700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 0x7f0e6c0c9eb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f0e74059e90 tx=0x7f0e7405da60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74084070 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74044fe0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c0467f0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74056dc0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c0e3c60 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0e74062930 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 0x7f0e4c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:24.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e79a67700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 0x7f0e4c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:24.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e740dd9d0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e79a67700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 0x7f0e4c057520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f0e70006fd0 tx=0x7f0e70006f40).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:24.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.778+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0d4dd0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.777 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.786+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f0e6c0d4dd0 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.986+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e6c0e6770 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.986+0000 7f0e61ffb700 1 -- 172.21.15.185:0/2893486743 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+2505 (secure 0 0 0) 0x7f0e6c0e6770 con 0x7f0e6c0cfe00 2020-11-16T22:21:24.979 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:24.980 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:24.636543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7083,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:creating","state_seq":2,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:24.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 msgr2=0x7f0e4c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 0x7f0e4c057520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f0e70006fd0 tx=0x7f0e70006f40).stop 2020-11-16T22:21:24.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 msgr2=0x7f0e6c0c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 0x7f0e6c0c9eb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f0e74059e90 tx=0x7f0e7405da60).stop 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 shutdown_connections 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0e4c055060 0x7f0e4c057520 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0e6c0cfe00 0x7f0e6c0c9eb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6c040650 0x7f0e6c046300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 --2- 172.21.15.185:0/2893486743 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0e6c0d0c30 0x7f0e6c0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:24.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 >> 172.21.15.185:0/2893486743 conn(0x7f0e6c0441c0 msgr2=0x7f0e6c00a140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:24.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 shutdown_connections 2020-11-16T22:21:24.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:24.990+0000 7f0e7aa69700 1 -- 172.21.15.185:0/2893486743 wait complete. 2020-11-16T22:21:24.987 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 85 2020-11-16T22:21:25.296 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:21:13.541645+0000', 'modified': '2020-11-16T22:21:24.636543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7083, 'mds_1': 7080}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7080': {'gid': 7080, 'name': 'b', 'rank': 1, 'incarnation': 85, 'state': 'up:creating', 'state_seq': 2, 'addr': '172.21.15.96:6833/862887203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 862887203}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 862887203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7083': {'gid': 7083, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3098481638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3098481638}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3098481638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:21:25.296 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7080:up:creating 2020-11-16T22:21:26.298 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:26.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.430+0000 7f7b76c4a700 1 Processor -- start 2020-11-16T22:21:26.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.430+0000 7f7b76c4a700 1 -- start start 2020-11-16T22:21:26.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b76c4a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7b700b7610 0x7f7b70067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:26.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b76c4a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700b3020 con 0x7f7b70142f30 2020-11-16T22:21:26.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b76c4a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700b3280 con 0x7f7b701441f0 2020-11-16T22:21:26.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b76c4a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b700b3520 con 0x7f7b700b7610 2020-11-16T22:21:26.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75c48700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7b700b7610 0x7f7b70067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:26.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75447700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7b701441f0 0x7f7b701446a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55868/0 (socket says 172.21.15.185:55868) 2020-11-16T22:21:26.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75447700 1 -- 172.21.15.185:0/743977725 learned_addr learned my addr 172.21.15.185:0/743977725 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:26.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399250973 0 0) 0x7f7b700b3280 con 0x7f7b701441f0 2020-11-16T22:21:26.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b58003240 con 0x7f7b701441f0 2020-11-16T22:21:26.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736178558 0 0) 0x7f7b700b3020 con 0x7f7b70142f30 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75c48700 1 -- 172.21.15.185:0/743977725 >> v1:172.21.15.185:6789/0 conn(0x7f7b701441f0 legacy=0x7f7b701446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75c48700 1 -- 172.21.15.185:0/743977725 >> v1:172.21.15.96:6789/0 conn(0x7f7b70142f30 legacy=0x7f7b70143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75c48700 1 -- 172.21.15.185:0/743977725 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70067f40 con 0x7f7b700b7610 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b75c48700 1 --2- 172.21.15.185:0/743977725 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b700b7610 0x7f7b70067910 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f7b6c00b9a0 tx=0x7f7b6c006df0).ready entity=mon.1 client_cookie=a23a8375d631d40f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0097b0 con 0x7f7b700b7610 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c009910 con 0x7f7b700b7610 2020-11-16T22:21:26.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.434+0000 7f7b74c46700 1 -- 172.21.15.185:0/743977725 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009c90 con 0x7f7b700b7610 2020-11-16T22:21:26.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- 172.21.15.185:0/743977725 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b700b7610 msgr2=0x7f7b70067910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/743977725 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b700b7610 0x7f7b70067910 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f7b6c00b9a0 tx=0x7f7b6c006df0).stop 2020-11-16T22:21:26.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- 172.21.15.185:0/743977725 shutdown_connections 2020-11-16T22:21:26.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/743977725 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b700b7610 0x7f7b70067910 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- 172.21.15.185:0/743977725 >> 172.21.15.185:0/743977725 conn(0x7f7b700637b0 msgr2=0x7f7b700b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:26.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- 172.21.15.185:0/743977725 shutdown_connections 2020-11-16T22:21:26.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- 172.21.15.185:0/743977725 wait complete. 2020-11-16T22:21:26.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 Processor -- start 2020-11-16T22:21:26.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.438+0000 7f7b76c4a700 1 -- start start 2020-11-16T22:21:26.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:26.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:26.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 0x7f7b7013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70142870 con 0x7f7b701441f0 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700af5d0 con 0x7f7b70142f30 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700ac400 con 0x7f7b700b7610 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75447700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75447700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34204/0 (socket says 172.21.15.185:34204) 2020-11-16T22:21:26.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55878/0 (socket says 172.21.15.185:55878) 2020-11-16T22:21:26.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75447700 1 -- 172.21.15.185:0/2872768033 learned_addr learned my addr 172.21.15.185:0/2872768033 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:26.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76449700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 0x7f7b7013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:26.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 -- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 msgr2=0x7f7b700af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 -- 172.21.15.185:0/2872768033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 msgr2=0x7f7b7013ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 0x7f7b7013ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 -- 172.21.15.185:0/2872768033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b701422d0 con 0x7f7b700b7610 2020-11-16T22:21:26.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76449700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 0x7f7b7013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:26.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75447700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b75c48700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7b6c012000 tx=0x7f7b6c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c011d50 con 0x7f7b700b7610 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c01e460 con 0x7f7b700b7610 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b70157340 con 0x7f7b700b7610 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c01e800 con 0x7f7b700b7610 2020-11-16T22:21:26.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.442+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b701412c0 con 0x7f7b700b7610 2020-11-16T22:21:26.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7b6c025030 con 0x7f7b700b7610 2020-11-16T22:21:26.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b70142460 con 0x7f7b700b7610 2020-11-16T22:21:26.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b61ffb700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 0x7f7b48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:26.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b6c06f520 con 0x7f7b700b7610 2020-11-16T22:21:26.641 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b75447700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 0x7f7b48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:26.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.446+0000 7f7b75447700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 0x7f7b48057520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f7b70148570 tx=0x7f7b64009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:26.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.454+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7b6c045670 con 0x7f7b700b7610 2020-11-16T22:21:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.658+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b7005d200 con 0x7f7b700b7610 2020-11-16T22:21:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b61ffb700 1 -- 172.21.15.185:0/2872768033 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2518 (secure 0 0 0) 0x7f7b6c023090 con 0x7f7b700b7610 2020-11-16T22:21:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 msgr2=0x7f7b48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 0x7f7b48057520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f7b70148570 tx=0x7f7b64009040).stop 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 msgr2=0x7f7b700aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f7b6c012000 tx=0x7f7b6c015000).stop 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 shutdown_connections 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b48055060 0x7f7b48057520 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b701441f0 0x7f7b7013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7b700b7610 0x7f7b700aebf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 --2- 172.21.15.185:0/2872768033 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7b70142f30 0x7f7b700af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 >> 172.21.15.185:0/2872768033 conn(0x7f7b700637b0 msgr2=0x7f7b700b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 shutdown_connections 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:26.662+0000 7f7b76c4a700 1 -- 172.21.15.185:0/2872768033 wait complete. 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 88 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:26.846 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:26.658559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:27.149 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:21:13.541645+0000', 'modified': '2020-11-16T22:21:26.658559+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7083, 'mds_1': 7080, 'mds_2': 6545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6545': {'gid': 6545, 'name': 'a', 'rank': 2, 'incarnation': 87, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.185:6835/2224996985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 2224996985}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 2224996985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7080': {'gid': 7080, 'name': 'b', 'rank': 1, 'incarnation': 85, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.96:6833/862887203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 862887203}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 862887203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7083': {'gid': 7083, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6835/3098481638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 3098481638}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 3098481638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:21:27.149 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:21:27.150 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:21:27.289 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":6545,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":1.6331657559999999,"mdsmap_epoch":88,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":18.981467877} 2020-11-16T22:21:27.289 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 6545, "mdsmap_epoch": 88, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 1.633165756, "state": "up:active", "uptime": 18.981467877, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:21:27.290 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:21:27.427 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":7080,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.789881791,"mdsmap_epoch":88,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":19.137225044000001} 2020-11-16T22:21:27.428 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7080, "mdsmap_epoch": 88, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 2.789881791, "state": "up:active", "uptime": 19.137225044, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:21:27.428 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:21:27.560 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7083,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.017457287999999,"mdsmap_epoch":88,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":19.268180674} 2020-11-16T22:21:27.560 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7083, "mdsmap_epoch": 88, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 14.017457288, "state": "up:active", "uptime": 19.268180674, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:21:27.561 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:27.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 Processor -- start 2020-11-16T22:21:27.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 -- start start 2020-11-16T22:21:27.687 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5a10141190 0x7f5a10141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a100b3af0 con 0x7f5a100b2eb0 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a10061e60 con 0x7f5a10141190 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a14fc9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100b3d20 con 0x7f5a100662c0 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a0effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.688 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a0effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55890/0 (socket says 172.21.15.185:55890) 2020-11-16T22:21:27.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a0effd700 1 -- 172.21.15.185:0/936922272 learned_addr learned my addr 172.21.15.185:0/936922272 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:27.689 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.694+0000 7f5a0ffff700 1 --2- 172.21.15.185:0/936922272 >> v2:172.21.15.96:3300/0 conn(0x7f5a10141190 0x7f5a10141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0effd700 1 -- 172.21.15.185:0/936922272 >> v2:172.21.15.96:3300/0 conn(0x7f5a10141190 msgr2=0x7f5a10141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:27.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0effd700 1 --2- 172.21.15.185:0/936922272 >> v2:172.21.15.96:3300/0 conn(0x7f5a10141190 0x7f5a10141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0effd700 1 -- 172.21.15.185:0/936922272 >> v1:172.21.15.96:6789/0 conn(0x7f5a100b2eb0 legacy=0x7f5a1013a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0effd700 1 -- 172.21.15.185:0/936922272 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10140cc0 con 0x7f5a100662c0 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0effd700 1 --2- 172.21.15.185:0/936922272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f59f8006fd0 tx=0x7f59f8006eb0).ready entity=mon.2 client_cookie=a932e77df532330 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0e7fc700 1 -- 172.21.15.185:0/936922272 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8009810 con 0x7f5a100662c0 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0e7fc700 1 -- 172.21.15.185:0/936922272 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8009970 con 0x7f5a100662c0 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a0e7fc700 1 -- 172.21.15.185:0/936922272 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8009cf0 con 0x7f5a100662c0 2020-11-16T22:21:27.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- 172.21.15.185:0/936922272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 msgr2=0x7f5a100b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/936922272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f59f8006fd0 tx=0x7f59f8006eb0).stop 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- 172.21.15.185:0/936922272 shutdown_connections 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/936922272 >> v2:172.21.15.96:3300/0 conn(0x7f5a10141190 0x7f5a10141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/936922272 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a100662c0 0x7f5a100b3290 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- 172.21.15.185:0/936922272 >> 172.21.15.185:0/936922272 conn(0x7f5a10060130 msgr2=0x7f5a10060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- 172.21.15.185:0/936922272 shutdown_connections 2020-11-16T22:21:27.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- 172.21.15.185:0/936922272 wait complete. 2020-11-16T22:21:27.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 Processor -- start 2020-11-16T22:21:27.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.698+0000 7f5a14fc9700 1 -- start start 2020-11-16T22:21:27.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 0x7f5a100ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a10141190 0x7f5a1014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10144940 con 0x7f5a100b2eb0 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10144b50 con 0x7f5a100662c0 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10144df0 con 0x7f5a10141190 2020-11-16T22:21:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a10141190 0x7f5a1014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34218/0 (socket says 172.21.15.185:34218) 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 -- 172.21.15.185:0/1348164573 learned_addr learned my addr 172.21.15.185:0/1348164573 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0effd700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 0x7f5a100ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 -- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a10141190 msgr2=0x7f5a1014f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a10141190 0x7f5a1014f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 -- 172.21.15.185:0/1348164573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 msgr2=0x7f5a100ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 0x7f5a100ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 -- 172.21.15.185:0/1348164573 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10153950 con 0x7f5a100662c0 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0effd700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 0x7f5a100ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0f7fe700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f5a0000c710 tx=0x7f5a00007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000e560 con 0x7f5a100662c0 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000e6c0 con 0x7f5a100662c0 2020-11-16T22:21:27.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a101445e0 con 0x7f5a100662c0 2020-11-16T22:21:27.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000ea80 con 0x7f5a100662c0 2020-11-16T22:21:27.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1014f480 con 0x7f5a100662c0 2020-11-16T22:21:27.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1005d200 con 0x7f5a100662c0 2020-11-16T22:21:27.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5a00019030 con 0x7f5a100662c0 2020-11-16T22:21:27.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 0x7f59ec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:27.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a0006d1d0 con 0x7f5a100662c0 2020-11-16T22:21:27.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0effd700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 0x7f59ec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:27.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.702+0000 7f5a0effd700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 0x7f59ec057520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f59f8006ef0 tx=0x7f59f8017000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:27.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.710+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5a00043430 con 0x7f5a100662c0 2020-11-16T22:21:27.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.910+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a10141b10 con 0x7f5a100662c0 2020-11-16T22:21:27.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.910+0000 7f5a077fe700 1 -- 172.21.15.185:0/1348164573 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+2518 (secure 0 0 0) 0x7f5a00017020 con 0x7f5a100662c0 2020-11-16T22:21:27.903 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:27.903 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:26.658559+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:27.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 msgr2=0x7f59ec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 0x7f59ec057520 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f59f8006ef0 tx=0x7f59f8017000).stop 2020-11-16T22:21:27.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 msgr2=0x7f5a10153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f5a0000c710 tx=0x7f5a00007c10).stop 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 shutdown_connections 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f59ec055060 0x7f59ec057520 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5a100662c0 0x7f5a10153460 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5a10141190 0x7f5a1014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 --2- 172.21.15.185:0/1348164573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a100b2eb0 0x7f5a100ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:27.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 >> 172.21.15.185:0/1348164573 conn(0x7f5a10060130 msgr2=0x7f5a10067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 shutdown_connections 2020-11-16T22:21:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:27.914+0000 7f5a14fc9700 1 -- 172.21.15.185:0/1348164573 wait complete. 2020-11-16T22:21:27.907 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 88 2020-11-16T22:21:27.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:27.918 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:21:27.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:27.986 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:21:28.167 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:21:28.178+0000 7fd96fe0c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:21:28.168 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:21:28.178+0000 7fd96fe0c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:21:28.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:28.301 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:21:28.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:28.343 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:21:28.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:28.385 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:21:28.420 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:28.556 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:21:28.556 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:28.557 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:30.558 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:30.692 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:21:30.693 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:30.693 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:32.694 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:32.833 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:21:32.834 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:32.834 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:34.836 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:34.975 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:21:34.976 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:34.976 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:36.977 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:37.116 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:21:37.117 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:37.117 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:37.821 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:37.827 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:37.830 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:21:39.118 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:21:39.255 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:21:39.256 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:21:39.458 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:21:39.459 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:21:39.593 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:21:39.594 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:21:39.802 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:21:41.803 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:21:41.940 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:21:41.941 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:21:41.941 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:21:43.942 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:21:44.080 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2020-11-16T22:21:44.081 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/d1/d2/d3/d4", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:21:44.081 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:21:46.083 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:21:46.218 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:21:46.218 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:21:46.422 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:21:46.423 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:21:46.560 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:21:46.561 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:21:46.766 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:21:46.767 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:46.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.906+0000 7f03e5807700 1 Processor -- start 2020-11-16T22:21:46.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- start start 2020-11-16T22:21:46.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f03e00675b0 0x7f03e0068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f03e00b4d40 0x7f03e013de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00b7e10 con 0x7f03e01418d0 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00b7670 con 0x7f03e00b4d40 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00b78d0 con 0x7f03e00675b0 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f03e00675b0 0x7f03e0068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:46.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03dffff700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f03e01418d0 0x7f03e00b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:41962/0 (socket says 172.21.15.185:41962) 2020-11-16T22:21:46.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03dffff700 1 -- 172.21.15.185:0/738349072 learned_addr learned my addr 172.21.15.185:0/738349072 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:46.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5006700 1 --2- 172.21.15.185:0/738349072 >> v2:172.21.15.96:3300/0 conn(0x7f03e00b4d40 0x7f03e013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:46.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03df7fe700 1 -- 172.21.15.185:0/738349072 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579872267 0 0) 0x7f03e00b7e10 con 0x7f03e01418d0 2020-11-16T22:21:46.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 -- 172.21.15.185:0/738349072 >> v2:172.21.15.96:3300/0 conn(0x7f03e00b4d40 msgr2=0x7f03e013de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:46.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 --2- 172.21.15.185:0/738349072 >> v2:172.21.15.96:3300/0 conn(0x7f03e00b4d40 0x7f03e013de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:46.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 -- 172.21.15.185:0/738349072 >> v1:172.21.15.96:6789/0 conn(0x7f03e01418d0 legacy=0x7f03e00b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:46.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 -- 172.21.15.185:0/738349072 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00b7b70 con 0x7f03e00675b0 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e4805700 1 --2- 172.21.15.185:0/738349072 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00675b0 0x7f03e0068460 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f03e0066cd0 tx=0x7f03d00091d0).ready entity=mon.1 client_cookie=1a8fe65450762ed3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03df7fe700 1 -- 172.21.15.185:0/738349072 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ba70 con 0x7f03e00675b0 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03df7fe700 1 -- 172.21.15.185:0/738349072 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d000bbd0 con 0x7f03e00675b0 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03df7fe700 1 -- 172.21.15.185:0/738349072 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011400 con 0x7f03e00675b0 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- 172.21.15.185:0/738349072 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00675b0 msgr2=0x7f03e0068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:46.905 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 --2- 172.21.15.185:0/738349072 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00675b0 0x7f03e0068460 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f03e0066cd0 tx=0x7f03d00091d0).stop 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- 172.21.15.185:0/738349072 shutdown_connections 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 --2- 172.21.15.185:0/738349072 >> v2:172.21.15.96:3300/0 conn(0x7f03e00b4d40 0x7f03e013de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 --2- 172.21.15.185:0/738349072 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00675b0 0x7f03e0068460 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- 172.21.15.185:0/738349072 >> 172.21.15.185:0/738349072 conn(0x7f03e005ba80 msgr2=0x7f03e005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- 172.21.15.185:0/738349072 shutdown_connections 2020-11-16T22:21:46.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.910+0000 7f03e5807700 1 -- 172.21.15.185:0/738349072 wait complete. 2020-11-16T22:21:46.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 Processor -- start 2020-11-16T22:21:46.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- start start 2020-11-16T22:21:46.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:46.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00b4d40 0x7f03e0136af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 0x7f03e0136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00aff10 con 0x7f03e01418d0 2020-11-16T22:21:47.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00b01b0 con 0x7f03e00b4d40 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00b0450 con 0x7f03e00675b0 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55910/0 (socket says 172.21.15.185:55910) 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 -- 172.21.15.185:0/1441912093 learned_addr learned my addr 172.21.15.185:0/1441912093 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03dffff700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00b4d40 0x7f03e0136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5006700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 0x7f03e0136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 -- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00b4d40 msgr2=0x7f03e0136af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00b4d40 0x7f03e0136af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 -- 172.21.15.185:0/1441912093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 msgr2=0x7f03e0136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 0x7f03e0136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 -- 172.21.15.185:0/1441912093 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e013c6f0 con 0x7f03e00675b0 2020-11-16T22:21:47.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5006700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 0x7f03e0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:47.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e4805700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f03d0009fd0 tx=0x7f03d0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d001c5f0 con 0x7f03e00675b0 2020-11-16T22:21:47.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d001c750 con 0x7f03e00675b0 2020-11-16T22:21:47.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e0140940 con 0x7f03e00675b0 2020-11-16T22:21:47.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d001cad0 con 0x7f03e00675b0 2020-11-16T22:21:47.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.914+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e00b0e00 con 0x7f03e00675b0 2020-11-16T22:21:47.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e005d200 con 0x7f03e00675b0 2020-11-16T22:21:47.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f03d0024030 con 0x7f03e00675b0 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03dcff9700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 0x7f03bc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f03d006f6c0 con 0x7f03e00675b0 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03dffff700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 0x7f03bc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.918+0000 7f03dffff700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 0x7f03bc057520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f03e00afd20 tx=0x7f03c8007a00).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:46.926+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f03d0045a50 con 0x7f03e00675b0 2020-11-16T22:21:47.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03e014bb50 con 0x7f03e00675b0 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03dcff9700 1 -- 172.21.15.185:0/1441912093 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f03d0022030 con 0x7f03e00675b0 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 msgr2=0x7f03bc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 0x7f03bc057520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f03e00afd20 tx=0x7f03c8007a00).stop 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 msgr2=0x7f03e013c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f03d0009fd0 tx=0x7f03d0016000).stop 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 shutdown_connections 2020-11-16T22:21:47.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f03bc055060 0x7f03bc057520 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e01418d0 0x7f03e0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f03e00675b0 0x7f03e013c200 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 --2- 172.21.15.185:0/1441912093 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f03e00b4d40 0x7f03e0136af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.130+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 >> 172.21.15.185:0/1441912093 conn(0x7f03e005ba80 msgr2=0x7f03e00b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.134+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 shutdown_connections 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.134+0000 7f03e5807700 1 -- 172.21.15.185:0/1441912093 wait complete. 2020-11-16T22:21:47.320 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 90 2020-11-16T22:21:47.321 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:47.321 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:44.248693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:47.321 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:21:47.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.462+0000 7f0c3eda1700 1 Processor -- start 2020-11-16T22:21:47.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- start start 2020-11-16T22:21:47.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c3813de10 0x7f0c380666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c3813e1f0 0x7f0c38143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380b7c20 con 0x7f0c380b3570 2020-11-16T22:21:47.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380b3a80 con 0x7f0c3813e1f0 2020-11-16T22:21:47.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380b3d20 con 0x7f0c3813de10 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c3813e1f0 0x7f0c38143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f0c3813e1f0 0x7f0c38143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34242/0 (socket says 172.21.15.185:34242) 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 -- 172.21.15.185:0/70169288 learned_addr learned my addr 172.21.15.185:0/70169288 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c3813de10 0x7f0c380666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 -- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c3813de10 msgr2=0x7f0c380666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c3813de10 0x7f0c380666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 -- 172.21.15.185:0/70169288 >> v1:172.21.15.96:6789/0 conn(0x7f0c380b3570 legacy=0x7f0c380b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 -- 172.21.15.185:0/70169288 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380b3fc0 con 0x7f0c3813e1f0 2020-11-16T22:21:47.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c3813e1f0 0x7f0c38143f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f0c28006fd0 tx=0x7f0c28006eb0).ready entity=mon.1 client_cookie=dacc00f7237a5ef7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3cd9d700 1 -- 172.21.15.185:0/70169288 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280097d0 con 0x7f0c3813e1f0 2020-11-16T22:21:47.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3cd9d700 1 -- 172.21.15.185:0/70169288 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28009930 con 0x7f0c3813e1f0 2020-11-16T22:21:47.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3cd9d700 1 -- 172.21.15.185:0/70169288 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28009cb0 con 0x7f0c3813e1f0 2020-11-16T22:21:47.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c3813e1f0 msgr2=0x7f0c38143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c3813e1f0 0x7f0c38143f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f0c28006fd0 tx=0x7f0c28006eb0).stop 2020-11-16T22:21:47.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- 172.21.15.185:0/70169288 shutdown_connections 2020-11-16T22:21:47.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c3813de10 0x7f0c380666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/70169288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c3813e1f0 0x7f0c38143f50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- 172.21.15.185:0/70169288 >> 172.21.15.185:0/70169288 conn(0x7f0c38064b90 msgr2=0x7f0c38063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- 172.21.15.185:0/70169288 shutdown_connections 2020-11-16T22:21:47.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.466+0000 7f0c3eda1700 1 -- 172.21.15.185:0/70169288 wait complete. 2020-11-16T22:21:47.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 Processor -- start 2020-11-16T22:21:47.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- start start 2020-11-16T22:21:47.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c3813de10 0x7f0c38147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 0x7f0c381508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380b0f20 con 0x7f0c3813de10 2020-11-16T22:21:47.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3813e960 con 0x7f0c38147f40 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3813ec00 con 0x7f0c380b3570 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55924/0 (socket says 172.21.15.185:55924) 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/1957179787 learned_addr learned my addr 172.21.15.185:0/1957179787 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 0x7f0c381508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c3813de10 0x7f0c38147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 msgr2=0x7f0c381508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 0x7f0c381508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/1957179787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c3813de10 msgr2=0x7f0c38147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c3813de10 0x7f0c38147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/1957179787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3813eea0 con 0x7f0c380b3570 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 0x7f0c381508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:47.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f0c3400ebd0 tx=0x7f0c34006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c25ffb700 1 -- 172.21.15.185:0/1957179787 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400b3a0 con 0x7f0c380b3570 2020-11-16T22:21:47.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c25ffb700 1 -- 172.21.15.185:0/1957179787 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400c8e0 con 0x7f0c380b3570 2020-11-16T22:21:47.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3813f140 con 0x7f0c380b3570 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c25ffb700 1 -- 172.21.15.185:0/1957179787 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400cc80 con 0x7f0c380b3570 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.470+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c38146ea0 con 0x7f0c380b3570 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c25ffb700 1 -- 172.21.15.185:0/1957179787 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0c3401c460 con 0x7f0c380b3570 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c25ffb700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 0x7f0c10057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c25ffb700 1 -- 172.21.15.185:0/1957179787 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c3406dfe0 con 0x7f0c380b3570 2020-11-16T22:21:47.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 0x7f0c10057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 0x7f0c10057520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0c38061210 tx=0x7f0c28015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 Processor -- start 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 -- start start 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b2d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29b94b0 con 0x290f7b0 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a6d930 con 0x28a1bb0 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.474+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x292d6f0 con 0x2b2dda0 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3d59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55936/0 (socket says 172.21.15.185:55936) 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/569045084 learned_addr learned my addr 172.21.15.185:0/569045084 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3d59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b2be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34258/0 (socket says 172.21.15.185:34258) 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b2d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 msgr2=0x2b2be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b2be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/569045084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 msgr2=0x2b2d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b2d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/569045084 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29285a0 con 0x2b2dda0 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b2d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f0c3000f5b0 tx=0x7f0c30007e60).ready entity=mon.2 client_cookie=3ac7d2e4beaee4ca server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c16ffd700 1 -- 172.21.15.185:0/569045084 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30002b20 con 0x2b2dda0 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c16ffd700 1 -- 172.21.15.185:0/569045084 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000fe90 con 0x2b2dda0 2020-11-16T22:21:47.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c16ffd700 1 -- 172.21.15.185:0/569045084 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30006870 con 0x2b2dda0 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 msgr2=0x2b2c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f0c3000f5b0 tx=0x7f0c30007e60).stop 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- 172.21.15.185:0/569045084 shutdown_connections 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b2dda0 0x2b2c380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b2d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/569045084 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- 172.21.15.185:0/569045084 >> 172.21.15.185:0/569045084 conn(0x2adde40 msgr2=0x2b2add0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- 172.21.15.185:0/569045084 shutdown_connections 2020-11-16T22:21:47.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- 172.21.15.185:0/569045084 wait complete. 2020-11-16T22:21:47.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- start start 2020-11-16T22:21:47.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 Processor -- start 2020-11-16T22:21:47.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b71b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 0x2b5bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a5a6e0 con 0x290f7b0 2020-11-16T22:21:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a5a820 con 0x2b2dda0 2020-11-16T22:21:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.478+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a5ba90 con 0x28a1bb0 2020-11-16T22:21:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 10 client.0 ll_register_callbacks cb 0x2a50ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:47.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 0x2b5bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55938/0 (socket says 172.21.15.185:55938) 2020-11-16T22:21:47.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/2510520811 learned_addr learned my addr 172.21.15.185:0/2510520811 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 0x2b5bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34266/0 (socket says 172.21.15.185:34266) 2020-11-16T22:21:47.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b71b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 msgr2=0x2b5bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 0x2b5bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 msgr2=0x2b71b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b71b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/2510520811 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a4b280 con 0x28a1bb0 2020-11-16T22:21:47.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b71b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:47.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f0c34047950 tx=0x7f0c34014000).ready entity=mon.2 client_cookie=6761cb476d4c0b75 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:21:47.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34043c90 con 0x28a1bb0 2020-11-16T22:21:47.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c34043df0 con 0x28a1bb0 2020-11-16T22:21:47.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2b0e6c0 con 0x28a1bb0 2020-11-16T22:21:47.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c34043fb0 con 0x28a1bb0 2020-11-16T22:21:47.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 10 client.6680 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:47.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x2880fd0 con 0x28a1bb0 2020-11-16T22:21:47.492 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2b0e6c0 con 0x28a1bb0 2020-11-16T22:21:47.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f0c34046e00 con 0x28a1bb0 2020-11-16T22:21:47.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 10 client.6680 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:47.493 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 10 client.6680 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:47.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 20 client.6680 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:47.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2ed00 0x2b75350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 4 client.6680 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:47.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x294bc50 con 0x2b2ed00 2020-11-16T22:21:47.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2ed00 0x2b75350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c0bfff700 10 client.6680 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:47.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.482+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2ed00 0x2b75350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ee0de52dec50de8 server_cookie=127b20a7594d7b87 in_seq=0 out_seq=0 2020-11-16T22:21:47.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c0bfff700 1 -- 172.21.15.185:0/2510520811 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x294bc50 con 0x2b2ed00 2020-11-16T22:21:47.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c0bfff700 10 client.6680 handle_command_reply: tid=0 2020-11-16T22:21:47.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 client.6680 shutdown 2020-11-16T22:21:47.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 msgr2=0x2b71670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f0c34047950 tx=0x7f0c34014000).stop 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 shutdown_connections 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2ed00 0x2b75350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b2dda0 0x2b5bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b71670 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c0bfff700 0 client.6680 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:47.498 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/2510520811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b71b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 >> 172.21.15.185:0/2510520811 conn(0x2adde40 msgr2=0x2b5ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 shutdown_connections 2020-11-16T22:21:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 1 -- 172.21.15.185:0/2510520811 wait complete. 2020-11-16T22:21:47.499 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.486+0000 7f0c4ea94740 20 client.6680 trim_cache size 0 max 16384 2020-11-16T22:21:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 Processor -- start 2020-11-16T22:21:47.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- start start 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b22bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a32e40 con 0x290f7b0 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b2baf0 con 0x28a1bb0 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b5c7d0 con 0x2adde40 2020-11-16T22:21:47.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3d59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55946/0 (socket says 172.21.15.185:55946) 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3d59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34270/0 (socket says 172.21.15.185:34270) 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/3783394759 learned_addr learned my addr 172.21.15.185:0/3783394759 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b22bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 msgr2=0x2b226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/3783394759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 msgr2=0x2b22bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b22bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/3783394759 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04008160 con 0x2adde40 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f0c3406e310 tx=0x7f0c34047750).ready entity=mon.2 client_cookie=fde2c38a6da03c18 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c177fe700 1 -- 172.21.15.185:0/3783394759 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400eee0 con 0x2adde40 2020-11-16T22:21:47.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c177fe700 1 -- 172.21.15.185:0/3783394759 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34043c90 con 0x2adde40 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c177fe700 1 -- 172.21.15.185:0/3783394759 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3406c7b0 con 0x2adde40 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 msgr2=0x2b30140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f0c3406e310 tx=0x7f0c34047750).stop 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- 172.21.15.185:0/3783394759 shutdown_connections 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2b22bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2adde40 0x2b30140 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/3783394759 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a1bb0 0x2b226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- 172.21.15.185:0/3783394759 >> 172.21.15.185:0/3783394759 conn(0x7f0c04002320 msgr2=0x2b27b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- 172.21.15.185:0/3783394759 shutdown_connections 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- 172.21.15.185:0/3783394759 wait complete. 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.506+0000 7f0c4ea94740 1 -- start start 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 Processor -- start 2020-11-16T22:21:47.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2bbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 0x2bbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bbdf60 con 0x290f7b0 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b69190 con 0x2adde40 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b2c320 con 0x28a1bb0 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 10 client.0 ll_register_callbacks cb 0x2b279a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:47.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 0x2bbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55950/0 (socket says 172.21.15.185:55950) 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 -- 172.21.15.185:0/1285009383 learned_addr learned my addr 172.21.15.185:0/1285009383 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3d59e700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2bbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 msgr2=0x2b302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 msgr2=0x2bbd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2bbd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 -- 172.21.15.185:0/1285009383 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x28bf720 con 0x2adde40 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3e5a0700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 0x2bbda70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f0c3000f4d0 tx=0x7f0c30018000).ready entity=mon.1 client_cookie=2bb70464ca2762b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30024070 con 0x2adde40 2020-11-16T22:21:47.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c30006870 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c3000fe90 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c30012e90 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 10 client.7462 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x28cae00 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000fe90 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7f0c3001fa30 con 0x2adde40 2020-11-16T22:21:47.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 10 client.7462 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 10 client.7462 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 20 client.7462 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2dda0 0x2bc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 4 client.7462 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x2952f50 con 0x2b2dda0 2020-11-16T22:21:47.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2dda0 0x2bc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c0bfff700 10 client.7462 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:47.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.510+0000 7f0c3dd9f700 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2dda0 0x2bc3df0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=dd137338db99f36 server_cookie=8a5e6bdf5080a4ef in_seq=0 out_seq=0 2020-11-16T22:21:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c0bfff700 1 -- 172.21.15.185:0/1285009383 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x2952f50 con 0x2b2dda0 2020-11-16T22:21:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c0bfff700 10 client.7462 handle_command_reply: tid=0 2020-11-16T22:21:47.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 client.7462 shutdown 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 msgr2=0x2bbda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 0x2bbda70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f0c3000f4d0 tx=0x7f0c30018000).stop 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 shutdown_connections 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b2dda0 0x2bc3df0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2adde40 0x2bbda70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c0bfff700 0 client.7462 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a1bb0 0x2b302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 --2- 172.21.15.185:0/1285009383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x290f7b0 0x2bbd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 >> 172.21.15.185:0/1285009383 conn(0x7f0c04002320 msgr2=0x2b72160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 shutdown_connections 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 1 -- 172.21.15.185:0/1285009383 wait complete. 2020-11-16T22:21:47.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c4ea94740 20 client.7462 trim_cache size 0 max 16384 2020-11-16T22:21:47.532 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:47.532 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:21:47.532 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:21:47.532 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:47.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 msgr2=0x7f0c10057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 0x7f0c10057520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0c38061210 tx=0x7f0c28015000).stop 2020-11-16T22:21:47.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 msgr2=0x7f0c381540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f0c3400ebd0 tx=0x7f0c34006df0).stop 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 shutdown_connections 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0c10055060 0x7f0c10057520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f0c38147f40 0x7f0c381508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f0c380b3570 0x7f0c381540b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 --2- 172.21.15.185:0/1957179787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c3813de10 0x7f0c38147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 >> 172.21.15.185:0/1957179787 conn(0x7f0c38064b90 msgr2=0x7f0c3813d310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 shutdown_connections 2020-11-16T22:21:47.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.538+0000 7f0c3eda1700 1 -- 172.21.15.185:0/1957179787 wait complete. 2020-11-16T22:21:47.544 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:47.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.674+0000 7f4493fff700 1 Processor -- start 2020-11-16T22:21:47.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 -- start start 2020-11-16T22:21:47.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f44940b7470 0x7f44940b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940682b0 0x7f4494143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4494067510 con 0x7f4494067070 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940677b0 con 0x7f44940b7470 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4493fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940b17a0 con 0x7f44940682b0 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f44927fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4494067070 0x7f44940b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:55954/0 (socket says 172.21.15.185:55954) 2020-11-16T22:21:47.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f44940b7470 0x7f44940b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f44927fc700 1 -- 172.21.15.185:0/2977872082 learned_addr learned my addr 172.21.15.185:0/2977872082 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:47.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f44937fe700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940682b0 0x7f4494143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 -- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940682b0 msgr2=0x7f4494143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:47.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940682b0 0x7f4494143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 -- 172.21.15.185:0/2977872082 >> v1:172.21.15.185:6789/0 conn(0x7f4494067070 legacy=0x7f44940b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4491ffb700 1 -- 172.21.15.185:0/2977872082 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399889048 0 0) 0x7f4494067510 con 0x7f4494067070 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 -- 172.21.15.185:0/2977872082 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940b1a40 con 0x7f44940b7470 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4492ffd700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940b7470 0x7f44940b7850 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f448400b9a0 tx=0x7f4484006df0).ready entity=mon.1 client_cookie=640d91a75d763e25 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4491ffb700 1 -- 172.21.15.185:0/2977872082 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840097b0 con 0x7f44940b7470 2020-11-16T22:21:47.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4491ffb700 1 -- 172.21.15.185:0/2977872082 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484009910 con 0x7f44940b7470 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.678+0000 7f4491ffb700 1 -- 172.21.15.185:0/2977872082 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484009c90 con 0x7f44940b7470 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940b7470 msgr2=0x7f44940b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940b7470 0x7f44940b7850 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f448400b9a0 tx=0x7f4484006df0).stop 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- 172.21.15.185:0/2977872082 shutdown_connections 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940682b0 0x7f4494143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- 172.21.15.185:0/2977872082 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940b7470 0x7f44940b7850 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- 172.21.15.185:0/2977872082 >> 172.21.15.185:0/2977872082 conn(0x7f44940637b0 msgr2=0x7f44940b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- 172.21.15.185:0/2977872082 shutdown_connections 2020-11-16T22:21:47.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- 172.21.15.185:0/2977872082 wait complete. 2020-11-16T22:21:47.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 Processor -- start 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- start start 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4494067070 0x7f4494140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494157870 con 0x7f4494067070 2020-11-16T22:21:47.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449413dce0 con 0x7f44940682b0 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4493fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449413ca50 con 0x7f44940b7470 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f44937fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f44927fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f44927fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34290/0 (socket says 172.21.15.185:34290) 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f44927fc700 1 -- 172.21.15.185:0/97376143 learned_addr learned my addr 172.21.15.185:0/97376143 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:47.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f44937fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55968/0 (socket says 172.21.15.185:55968) 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.682+0000 7f4492ffd700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4494067070 0x7f4494140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 -- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 msgr2=0x7f4494111800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 -- 172.21.15.185:0/97376143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4494067070 msgr2=0x7f4494140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4494067070 0x7f4494140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 -- 172.21.15.185:0/97376143 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449413c510 con 0x7f44940b7470 2020-11-16T22:21:47.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44927fc700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f44937fe700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f448c00ed50 tx=0x7f448c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00cb00 con 0x7f44940b7470 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c00cc80 con 0x7f44940b7470 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449413c7b0 con 0x7f44940b7470 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0123f0 con 0x7f44940b7470 2020-11-16T22:21:47.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449410ff50 con 0x7f44940b7470 2020-11-16T22:21:47.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449405d200 con 0x7f44940b7470 2020-11-16T22:21:47.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f448c022070 con 0x7f44940b7470 2020-11-16T22:21:47.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 0x7f4470057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:47.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f448c06e540 con 0x7f44940b7470 2020-11-16T22:21:47.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.686+0000 7f4492ffd700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 0x7f4470057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:47.683 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.690+0000 7f4492ffd700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 0x7f4470057520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f4484006e30 tx=0x7f4484015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:47.686 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.694+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f448c044690 con 0x7f44940b7470 2020-11-16T22:21:47.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.894+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4494061390 con 0x7f44940b7470 2020-11-16T22:21:47.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f447affd700 1 -- 172.21.15.185:0/97376143 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f448c016020 con 0x7f44940b7470 2020-11-16T22:21:47.888 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:47.889 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:44.248693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 msgr2=0x7f4470057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 0x7f4470057520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f4484006e30 tx=0x7f4484015000).stop 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 msgr2=0x7f4494111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f448c00ed50 tx=0x7f448c0091d0).stop 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 shutdown_connections 2020-11-16T22:21:47.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4470055060 0x7f4470057520 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f44940b7470 0x7f4494111cf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4494067070 0x7f4494140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 --2- 172.21.15.185:0/97376143 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f44940682b0 0x7f4494111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 >> 172.21.15.185:0/97376143 conn(0x7f44940637b0 msgr2=0x7f44940b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 shutdown_connections 2020-11-16T22:21:47.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:47.898+0000 7f4493fff700 1 -- 172.21.15.185:0/97376143 wait complete. 2020-11-16T22:21:47.893 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 90 2020-11-16T22:21:47.903 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:21:48.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 Processor -- start 2020-11-16T22:21:48.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 -- start start 2020-11-16T22:21:48.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1080b2e80 0x7fa1080b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1080b3920 0x7fa108143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1080ac070 con 0x7fa10813dcf0 2020-11-16T22:21:48.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080ac310 con 0x7fa1080b2e80 2020-11-16T22:21:48.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10f2c8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080ac5b0 con 0x7fa1080b3920 2020-11-16T22:21:48.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10e2c6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1080b2e80 0x7fa1080b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.038+0000 7fa10dac5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1080b3920 0x7fa108143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1080b2e80 0x7fa1080b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34296/0 (socket says 172.21.15.185:34296) 2020-11-16T22:21:48.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2929350218 learned_addr learned my addr 172.21.15.185:0/2929350218 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1080b3920 msgr2=0x7fa108143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1080b3920 0x7fa108143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2929350218 >> v1:172.21.15.96:6789/0 conn(0x7fa10813dcf0 legacy=0x7fa1080b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2929350218 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080ac850 con 0x7fa1080b2e80 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10d2c4700 1 -- 172.21.15.185:0/2929350218 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541745491 0 0) 0x7fa1080ac070 con 0x7fa10813dcf0 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1080b3260 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fa100009d50 tx=0x7fa100009290).ready entity=mon.1 client_cookie=6074b07518088c01 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10d2c4700 1 -- 172.21.15.185:0/2929350218 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000bbc0 con 0x7fa1080b2e80 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10d2c4700 1 -- 172.21.15.185:0/2929350218 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000bd20 con 0x7fa1080b2e80 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10d2c4700 1 -- 172.21.15.185:0/2929350218 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100011610 con 0x7fa1080b2e80 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 msgr2=0x7fa1080b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1080b3260 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fa100009d50 tx=0x7fa100009290).stop 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2929350218 shutdown_connections 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1080b3260 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2929350218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1080b3920 0x7fa108143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2929350218 >> 172.21.15.185:0/2929350218 conn(0x7fa108060130 msgr2=0x7fa108060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2929350218 shutdown_connections 2020-11-16T22:21:48.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2929350218 wait complete. 2020-11-16T22:21:48.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 Processor -- start 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.042+0000 7fa10f2c8700 1 -- start start 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1080b3920 0x7fa10813e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa10813dcf0 0x7fa10813f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108147d70 con 0x7fa1080b3920 2020-11-16T22:21:48.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081575b0 con 0x7fa1080b2e80 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080ad330 con 0x7fa10813dcf0 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34304/0 (socket says 172.21.15.185:34304) 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2061371681 learned_addr learned my addr 172.21.15.185:0/2061371681 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10eac7700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa10813dcf0 0x7fa10813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10dac5700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1080b3920 0x7fa10813e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.040 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa10813dcf0 msgr2=0x7fa10813f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa10813dcf0 0x7fa10813f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2061371681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1080b3920 msgr2=0x7fa10813e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1080b3920 0x7fa10813e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 -- 172.21.15.185:0/2061371681 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080ad5d0 con 0x7fa1080b2e80 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fa108063040 tx=0x7fa100016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa1067fc700 1 -- 172.21.15.185:0/2061371681 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10001b5e0 con 0x7fa1080b2e80 2020-11-16T22:21:48.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa1067fc700 1 -- 172.21.15.185:0/2061371681 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10001b740 con 0x7fa1080b2e80 2020-11-16T22:21:48.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1080ad870 con 0x7fa1080b2e80 2020-11-16T22:21:48.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa1067fc700 1 -- 172.21.15.185:0/2061371681 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10001bac0 con 0x7fa1080b2e80 2020-11-16T22:21:48.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.046+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa108147010 con 0x7fa1080b2e80 2020-11-16T22:21:48.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.050+0000 7fa1067fc700 1 -- 172.21.15.185:0/2061371681 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa100017070 con 0x7fa1080b2e80 2020-11-16T22:21:48.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.050+0000 7fa1067fc700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 0x7fa0e4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.050+0000 7fa10dac5700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 0x7fa0e4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.050+0000 7fa1067fc700 1 -- 172.21.15.185:0/2061371681 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa10006fdc0 con 0x7fa1080b2e80 2020-11-16T22:21:48.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.050+0000 7fa10dac5700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 0x7fa0e4057520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fa108147780 tx=0x7fa0f8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 Processor -- start 2020-11-16T22:21:48.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- start start 2020-11-16T22:21:48.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x140abb0 0x1694e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.046 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x1695380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15224b0 con 0x1696da0 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d6930 con 0x140abb0 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x14966f0 con 0x14787b0 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55986/0 (socket says 172.21.15.185:55986) 2020-11-16T22:21:48.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 -- 172.21.15.185:0/884109313 learned_addr learned my addr 172.21.15.185:0/884109313 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10dac5700 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x140abb0 0x1694e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10eac7700 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x1695380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 -- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x140abb0 msgr2=0x1694e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x140abb0 0x1694e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 -- 172.21.15.185:0/884109313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 msgr2=0x1695380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x1695380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 -- 172.21.15.185:0/884109313 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14915a0 con 0x14787b0 2020-11-16T22:21:48.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa100049b10 tx=0x7fa100012000).ready entity=mon.2 client_cookie=52e44778c690ef9a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa0ff7fe700 1 -- 172.21.15.185:0/884109313 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100049140 con 0x14787b0 2020-11-16T22:21:48.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa0ff7fe700 1 -- 172.21.15.185:0/884109313 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1000492a0 con 0x14787b0 2020-11-16T22:21:48.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa0ff7fe700 1 -- 172.21.15.185:0/884109313 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100049410 con 0x14787b0 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 msgr2=0x16968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fa100049b10 tx=0x7fa100012000).stop 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- 172.21.15.185:0/884109313 shutdown_connections 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x1695380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14787b0 0x16968b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 --2- 172.21.15.185:0/884109313 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x140abb0 0x1694e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- 172.21.15.185:0/884109313 >> 172.21.15.185:0/884109313 conn(0x1646e40 msgr2=0x1693dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- 172.21.15.185:0/884109313 shutdown_connections 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- 172.21.15.185:0/884109313 wait complete. 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 -- start start 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.054+0000 7fa11efbb740 1 Processor -- start 2020-11-16T22:21:48.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x1697480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x16c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15c36e0 con 0x1696da0 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x15c3820 con 0x14787b0 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x15c4a90 con 0x140abb0 2020-11-16T22:21:48.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 10 client.0 ll_register_callbacks cb 0x15b9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:55992/0 (socket says 172.21.15.185:55992) 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 -- 172.21.15.185:0/3327287882 learned_addr learned my addr 172.21.15.185:0/3327287882 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10dac5700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x1697480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10eac7700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x16c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 -- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 msgr2=0x1697480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x1697480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 -- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 msgr2=0x16c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x16c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 -- 172.21.15.185:0/3327287882 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15b4280 con 0x140abb0 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fa100032040 tx=0x7fa100015000).ready entity=mon.2 client_cookie=804ef1ed7ca084a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:21:48.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100048dc0 con 0x140abb0 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa100048f20 con 0x140abb0 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16776c0 con 0x140abb0 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa10001aa70 con 0x140abb0 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 10 client.6704 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x13e9fd0 con 0x140abb0 2020-11-16T22:21:48.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16776c0 con 0x140abb0 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fa10001abd0 con 0x140abb0 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 10 client.6704 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 10 client.6704 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 20 client.6704 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1692640 0x16de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 4 client.6704 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:48.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x14b4c50 con 0x1692640 2020-11-16T22:21:48.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10dac5700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1692640 0x16de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa0fcff9700 10 client.6704 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:48.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.058+0000 7fa10dac5700 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1692640 0x16de280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=5c5d7c7789b88662 server_cookie=c75188f756ab5956 in_seq=0 out_seq=0 2020-11-16T22:21:48.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa0fcff9700 1 -- 172.21.15.185:0/3327287882 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x14b4c50 con 0x1692640 2020-11-16T22:21:48.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa0fcff9700 10 client.6704 handle_command_reply: tid=0 2020-11-16T22:21:48.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 client.6704 shutdown 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 msgr2=0x16db000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fa100032040 tx=0x7fa100015000).stop 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 shutdown_connections 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1692640 0x16de280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1696da0 0x16c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x16db000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 --2- 172.21.15.185:0/3327287882 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x1697480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 >> 172.21.15.185:0/3327287882 conn(0x1646e40 msgr2=0x16c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa0fcff9700 0 client.6704 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 shutdown_connections 2020-11-16T22:21:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 1 -- 172.21.15.185:0/3327287882 wait complete. 2020-11-16T22:21:48.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.062+0000 7fa11efbb740 20 client.6704 trim_cache size 0 max 16384 2020-11-16T22:21:48.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 Processor -- start 2020-11-16T22:21:48.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 -- start start 2020-11-16T22:21:48.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x1699600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x14787b0 0x1699af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x159be40 con 0x14787b0 2020-11-16T22:21:48.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1692b00 con 0x1646e40 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x15f3680 con 0x140abb0 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10dac5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x1699600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34324/0 (socket says 172.21.15.185:34324) 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 -- 172.21.15.185:0/1246845682 learned_addr learned my addr 172.21.15.185:0/1246845682 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10eac7700 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x14787b0 0x1699af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 -- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 msgr2=0x1699600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x1699600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 -- 172.21.15.185:0/1246845682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x14787b0 msgr2=0x1699af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x14787b0 0x1699af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 -- 172.21.15.185:0/1246845682 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0081d0 con 0x1646e40 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10dac5700 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x1699600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:48.074 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fa100049b80 tx=0x7fa10003b000).ready entity=mon.1 client_cookie=d6ef4d2a95acbc7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa0fffff700 1 -- 172.21.15.185:0/1246845682 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10006e320 con 0x1646e40 2020-11-16T22:21:48.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa0fffff700 1 -- 172.21.15.185:0/1246845682 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100048dc0 con 0x1646e40 2020-11-16T22:21:48.075 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.078+0000 7fa0fffff700 1 -- 172.21.15.185:0/1246845682 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100049140 con 0x1646e40 2020-11-16T22:21:48.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 msgr2=0x1693060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fa100049b80 tx=0x7fa10003b000).stop 2020-11-16T22:21:48.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- 172.21.15.185:0/1246845682 shutdown_connections 2020-11-16T22:21:48.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1646e40 0x1693060 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x14787b0 0x1699af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- 172.21.15.185:0/1246845682 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x140abb0 0x1699600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- 172.21.15.185:0/1246845682 >> 172.21.15.185:0/1246845682 conn(0x7fa0dc002320 msgr2=0x16c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- 172.21.15.185:0/1246845682 shutdown_connections 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- 172.21.15.185:0/1246845682 wait complete. 2020-11-16T22:21:48.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- start start 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 Processor -- start 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x140abb0 0x172fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x172c340 con 0x140abb0 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d3680 con 0x14787b0 2020-11-16T22:21:48.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c9f30 con 0x1646e40 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa11efbb740 10 client.0 ll_register_callbacks cb 0x16993d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10dac5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10eac7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10eac7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56010/0 (socket says 172.21.15.185:56010) 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10eac7700 1 -- 172.21.15.185:0/17781116 learned_addr learned my addr 172.21.15.185:0/17781116 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10dac5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34332/0 (socket says 172.21.15.185:34332) 2020-11-16T22:21:48.081 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.082+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x140abb0 0x172fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 -- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 msgr2=0x16dd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 -- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x140abb0 msgr2=0x172fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x140abb0 0x172fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 -- 172.21.15.185:0/17781116 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1428720 con 0x14787b0 2020-11-16T22:21:48.082 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10eac7700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10dac5700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fa0f800b3b0 tx=0x7fa0f8019040).ready entity=mon.1 client_cookie=4d64ad80e3e48046 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8029070 con 0x14787b0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0f802fc50 con 0x14787b0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1000492a0 con 0x14787b0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f8024ea0 con 0x14787b0 2020-11-16T22:21:48.083 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 10 client.7489 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x1433e00 con 0x14787b0 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1000492a0 con 0x14787b0 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fa0f8035020 con 0x14787b0 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 10 client.7489 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 10 client.7489 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 20 client.7489 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:48.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1696da0 0x16dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 4 client.7489 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:21:48.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x14bbf50 con 0x1696da0 2020-11-16T22:21:48.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1696da0 0x16dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa0fcff9700 10 client.7489 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:48.085 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.086+0000 7fa10e2c6700 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1696da0 0x16dde70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=4f0df08cfc210772 server_cookie=52c06fb364ca7fde in_seq=0 out_seq=0 2020-11-16T22:21:48.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa0fcff9700 1 -- 172.21.15.185:0/17781116 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x14bbf50 con 0x1696da0 2020-11-16T22:21:48.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa0fcff9700 10 client.7489 handle_command_reply: tid=0 2020-11-16T22:21:48.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 client.7489 shutdown 2020-11-16T22:21:48.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 msgr2=0x172ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fa0f800b3b0 tx=0x7fa0f8019040).stop 2020-11-16T22:21:48.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 shutdown_connections 2020-11-16T22:21:48.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x1696da0 0x16dde70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1646e40 0x16dd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa0fcff9700 0 client.7489 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x140abb0 0x172fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 --2- 172.21.15.185:0/17781116 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14787b0 0x172ff20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 >> 172.21.15.185:0/17781116 conn(0x7fa0dc002320 msgr2=0x16cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 shutdown_connections 2020-11-16T22:21:48.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 1 -- 172.21.15.185:0/17781116 wait complete. 2020-11-16T22:21:48.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa11efbb740 20 client.7489 trim_cache size 0 max 16384 2020-11-16T22:21:48.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 msgr2=0x7fa0e4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 0x7fa0e4057520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fa108147780 tx=0x7fa0f8006df0).stop 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 msgr2=0x7fa1081570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fa108063040 tx=0x7fa100016000).stop 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 shutdown_connections 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0e4055060 0x7fa0e4057520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1080b2e80 0x7fa1081570c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa10813dcf0 0x7fa10813f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 --2- 172.21.15.185:0/2061371681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1080b3920 0x7fa10813e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 >> 172.21.15.185:0/2061371681 conn(0x7fa108060130 msgr2=0x7fa1080b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 shutdown_connections 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.106+0000 7fa10f2c8700 1 -- 172.21.15.185:0/2061371681 wait complete. 2020-11-16T22:21:48.103 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:48.104 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:21:48.104 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:21:48.104 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:48.116 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:48.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 Processor -- start 2020-11-16T22:21:48.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 -- start start 2020-11-16T22:21:48.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b18065fe0 con 0x7f1b18067070 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b18066280 con 0x7f1b180b2950 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b1d9fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18062e70 con 0x7f1b181417d0 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56016/0 (socket says 172.21.15.185:56016) 2020-11-16T22:21:48.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 -- 172.21.15.185:0/1308048380 learned_addr learned my addr 172.21.15.185:0/1308048380 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b16ffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f1b180b2950 0x7f1b18062760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56008/0 (socket says 172.21.15.185:56008) 2020-11-16T22:21:48.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 -- 172.21.15.185:0/1308048380 >> v1:172.21.15.185:6789/0 conn(0x7f1b180b2950 legacy=0x7f1b18062760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 -- 172.21.15.185:0/1308048380 >> v1:172.21.15.96:6789/0 conn(0x7f1b18067070 legacy=0x7f1b18063710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 -- 172.21.15.185:0/1308048380 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18066520 con 0x7f1b181417d0 2020-11-16T22:21:48.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b167fc700 1 --2- 172.21.15.185:0/1308048380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f1b08009d80 tx=0x7f1b08009290).ready entity=mon.2 client_cookie=db4054f44c151d90 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.250+0000 7f1b15ffb700 1 -- 172.21.15.185:0/1308048380 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800bc80 con 0x7f1b181417d0 2020-11-16T22:21:48.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b15ffb700 1 -- 172.21.15.185:0/1308048380 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800bde0 con 0x7f1b181417d0 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b15ffb700 1 -- 172.21.15.185:0/1308048380 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08010680 con 0x7f1b181417d0 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/1308048380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 msgr2=0x7f1b180b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/1308048380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f1b08009d80 tx=0x7f1b08009290).stop 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/1308048380 shutdown_connections 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/1308048380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b181417d0 0x7f1b180b2d30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/1308048380 >> 172.21.15.185:0/1308048380 conn(0x7f1b180b4410 msgr2=0x7f1b18060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/1308048380 shutdown_connections 2020-11-16T22:21:48.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.254+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/1308048380 wait complete. 2020-11-16T22:21:48.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 Processor -- start 2020-11-16T22:21:48.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- start start 2020-11-16T22:21:48.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b180b2950 0x7f1b180b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.251 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b181417d0 0x7f1b18144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1814fe60 con 0x7f1b180b2950 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18143f00 con 0x7f1b181417d0 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b181443a0 con 0x7f1b18067070 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56020/0 (socket says 172.21.15.185:56020) 2020-11-16T22:21:48.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 -- 172.21.15.185:0/2802321669 learned_addr learned my addr 172.21.15.185:0/2802321669 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b177fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b181417d0 0x7f1b18144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b167fc700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b180b2950 0x7f1b180b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 -- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b181417d0 msgr2=0x7f1b18144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b181417d0 0x7f1b18144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 -- 172.21.15.185:0/2802321669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b180b2950 msgr2=0x7f1b180b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.253 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b180b2950 0x7f1b180b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 -- 172.21.15.185:0/2802321669 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1814fb50 con 0x7f1b18067070 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b16ffd700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f1b0000c710 tx=0x7f1b00007c40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000072e0 con 0x7f1b18067070 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00005530 con 0x7f1b18067070 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000058b0 con 0x7f1b18067070 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b181425b0 con 0x7f1b18067070 2020-11-16T22:21:48.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.258+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b18143760 con 0x7f1b18067070 2020-11-16T22:21:48.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1805d4e0 con 0x7f1b18067070 2020-11-16T22:21:48.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1b00010070 con 0x7f1b18067070 2020-11-16T22:21:48.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b0effd700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 0x7f1af4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b167fc700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 0x7f1af4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b0006d390 con 0x7f1b18067070 2020-11-16T22:21:48.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.262+0000 7f1b167fc700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 0x7f1af4057520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f1b080092d0 tx=0x7f1b08018000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.266+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1b000435f0 con 0x7f1b18067070 2020-11-16T22:21:48.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.470+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b1814ffd0 con 0x7f1b18067070 2020-11-16T22:21:48.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.470+0000 7f1b0effd700 1 -- 172.21.15.185:0/2802321669 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+2522 (secure 0 0 0) 0x7f1b00046c10 con 0x7f1b18067070 2020-11-16T22:21:48.463 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:48.464 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:44.248693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:48.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 msgr2=0x7f1af4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 0x7f1af4057520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f1b080092d0 tx=0x7f1b08018000).stop 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 msgr2=0x7f1b180b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f1b0000c710 tx=0x7f1b00007c40).stop 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 shutdown_connections 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1af4055060 0x7f1af4057520 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1b181417d0 0x7f1b18144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1b18067070 0x7f1b180b0940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 --2- 172.21.15.185:0/2802321669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b180b2950 0x7f1b180b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 >> 172.21.15.185:0/2802321669 conn(0x7f1b180b4410 msgr2=0x7f1b18140d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 shutdown_connections 2020-11-16T22:21:48.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.474+0000 7f1b1d9fe700 1 -- 172.21.15.185:0/2802321669 wait complete. 2020-11-16T22:21:48.467 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 90 2020-11-16T22:21:48.482 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:21:48.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 Processor -- start 2020-11-16T22:21:48.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 -- start start 2020-11-16T22:21:48.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 0x7fb4180b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180b1310 con 0x7fb418143ef0 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41813e400 con 0x7fb4180b3920 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41daa1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180ac070 con 0x7fb4180b2e80 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb417fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb4180b3920 0x7fb41813dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56024/0 (socket says 172.21.15.185:56024) 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb417fff700 1 -- 172.21.15.185:0/2696945666 learned_addr learned my addr 172.21.15.185:0/2696945666 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 0x7fb4180b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb4177fe700 1 -- 172.21.15.185:0/2696945666 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446838631 0 0) 0x7fb41813e400 con 0x7fb4180b3920 2020-11-16T22:21:48.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41ca9f700 1 -- 172.21.15.185:0/2696945666 >> v1:172.21.15.185:6789/0 conn(0x7fb4180b3920 legacy=0x7fb41813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41ca9f700 1 -- 172.21.15.185:0/2696945666 >> v1:172.21.15.96:6789/0 conn(0x7fb418143ef0 legacy=0x7fb4180b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41ca9f700 1 -- 172.21.15.185:0/2696945666 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180ac310 con 0x7fb4180b2e80 2020-11-16T22:21:48.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.618+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/2696945666 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 0x7fb4180b3260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fb41000b9d0 tx=0x7fb410006df0).ready entity=mon.2 client_cookie=891d6ef8b24d9585 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb4177fe700 1 -- 172.21.15.185:0/2696945666 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410009770 con 0x7fb4180b2e80 2020-11-16T22:21:48.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb4177fe700 1 -- 172.21.15.185:0/2696945666 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4100098d0 con 0x7fb4180b2e80 2020-11-16T22:21:48.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb4177fe700 1 -- 172.21.15.185:0/2696945666 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410009c50 con 0x7fb4180b2e80 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- 172.21.15.185:0/2696945666 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 msgr2=0x7fb4180b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 --2- 172.21.15.185:0/2696945666 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 0x7fb4180b3260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fb41000b9d0 tx=0x7fb410006df0).stop 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- 172.21.15.185:0/2696945666 shutdown_connections 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 --2- 172.21.15.185:0/2696945666 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b2e80 0x7fb4180b3260 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- 172.21.15.185:0/2696945666 >> 172.21.15.185:0/2696945666 conn(0x7fb418060130 msgr2=0x7fb418060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- 172.21.15.185:0/2696945666 shutdown_connections 2020-11-16T22:21:48.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- 172.21.15.185:0/2696945666 wait complete. 2020-11-16T22:21:48.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 Processor -- start 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- start start 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 0x7fb4180afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb418143ef0 0x7fb418150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418146ba0 con 0x7fb4180b2e80 2020-11-16T22:21:48.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180b6180 con 0x7fb418143ef0 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41daa1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb418154460 con 0x7fb4180b3920 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb417fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41d2a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb418143ef0 0x7fb418150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb417fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56036/0 (socket says 172.21.15.185:56036) 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb417fff700 1 -- 172.21.15.185:0/3460006795 learned_addr learned my addr 172.21.15.185:0/3460006795 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.622+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 0x7fb4180afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 -- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb418143ef0 msgr2=0x7fb418150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb418143ef0 0x7fb418150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 -- 172.21.15.185:0/3460006795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 msgr2=0x7fb4180afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 0x7fb4180afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 -- 172.21.15.185:0/3460006795 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418147340 con 0x7fb4180b3920 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 0x7fb4180afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:48.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb417fff700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb40000c760 tx=0x7fb4000079a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 -- 172.21.15.185:0/3460006795 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000072e0 con 0x7fb4180b3920 2020-11-16T22:21:48.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 -- 172.21.15.185:0/3460006795 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400007440 con 0x7fb4180b3920 2020-11-16T22:21:48.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4181475e0 con 0x7fb4180b3920 2020-11-16T22:21:48.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 -- 172.21.15.185:0/3460006795 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400005820 con 0x7fb4180b3920 2020-11-16T22:21:48.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4180b5e00 con 0x7fb4180b3920 2020-11-16T22:21:48.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 -- 172.21.15.185:0/3460006795 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb400010070 con 0x7fb4180b3920 2020-11-16T22:21:48.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 0x7fb3f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 0x7fb3f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb414ff9700 1 -- 172.21.15.185:0/3460006795 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb40006d570 con 0x7fb4180b3920 2020-11-16T22:21:48.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.626+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 0x7fb3f4057520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fb41000b920 tx=0x7fb410015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 Processor -- start 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 -- start start 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x20fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x20fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f8a4b0 con 0x1e72bb0 2020-11-16T22:21:48.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x203e930 con 0x1ee07b0 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1efe6f0 con 0x20feda0 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb41d2a0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x20fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x20fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34366/0 (socket says 172.21.15.185:34366) 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb41d2a0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56046/0 (socket says 172.21.15.185:56046) 2020-11-16T22:21:48.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb41ca9f700 1 -- 172.21.15.185:0/4161651961 learned_addr learned my addr 172.21.15.185:0/4161651961 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.630+0000 7fb417fff700 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x20fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 -- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 msgr2=0x20fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x20fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 -- 172.21.15.185:0/4161651961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 msgr2=0x20fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x20fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 -- 172.21.15.185:0/4161651961 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ef95a0 con 0x20feda0 2020-11-16T22:21:48.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fb418064dc0 tx=0x7fb408009040).ready entity=mon.2 client_cookie=db1664e44a1d88c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb40dffb700 1 -- 172.21.15.185:0/4161651961 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408001810 con 0x20feda0 2020-11-16T22:21:48.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb40dffb700 1 -- 172.21.15.185:0/4161651961 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40800fbe0 con 0x20feda0 2020-11-16T22:21:48.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb40dffb700 1 -- 172.21.15.185:0/4161651961 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408014450 con 0x20feda0 2020-11-16T22:21:48.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 msgr2=0x20fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fb418064dc0 tx=0x7fb408009040).stop 2020-11-16T22:21:48.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/4161651961 shutdown_connections 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x20fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x20fd380 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- 172.21.15.185:0/4161651961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x20fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/4161651961 >> 172.21.15.185:0/4161651961 conn(0x20aee40 msgr2=0x20fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/4161651961 shutdown_connections 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/4161651961 wait complete. 2020-11-16T22:21:48.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- start start 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 Processor -- start 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x2142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x212cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x202b6e0 con 0x1e72bb0 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x202b820 con 0x1ee07b0 2020-11-16T22:21:48.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x202ca90 con 0x20feda0 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 10 client.0 ll_register_callbacks cb 0x2021ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34372/0 (socket says 172.21.15.185:34372) 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 -- 172.21.15.185:0/1643494366 learned_addr learned my addr 172.21.15.185:0/1643494366 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x212cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x2142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 -- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 msgr2=0x212cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x212cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 -- 172.21.15.185:0/1643494366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 msgr2=0x2142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x2142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 -- 172.21.15.185:0/1643494366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x201c280 con 0x1ee07b0 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x2142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:48.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb417fff700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7fb400047080 tx=0x7fb400009000).ready entity=mon.1 client_cookie=fd4d0707a2e0f646 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40004d070 con 0x1ee07b0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb40006d0d0 con 0x1ee07b0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20df6c0 con 0x1ee07b0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4000105e0 con 0x1ee07b0 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 10 client.7504 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:48.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x1e51fd0 con 0x1ee07b0 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20df6c0 con 0x1ee07b0 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fb40001a030 con 0x1ee07b0 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 10 client.7504 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 10 client.7504 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 20 client.7504 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x21468d0 0x2132950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.637 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 4 client.7504 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.634+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1f1cc50 con 0x21468d0 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x21468d0 0x2132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb3e6ffd700 10 client.7504 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x21468d0 0x2132950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=b76f14df5dcd3074 server_cookie=762d59e6de1ef33d in_seq=0 out_seq=0 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/1643494366 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1f1cc50 con 0x21468d0 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb3e6ffd700 10 client.7504 handle_command_reply: tid=0 2020-11-16T22:21:48.638 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 client.7504 shutdown 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 msgr2=0x21463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7fb400047080 tx=0x7fb400009000).stop 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 shutdown_connections 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x21468d0 0x2132950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20feda0 0x212cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.639 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb3e6ffd700 0 client.7504 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x2142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 --2- 172.21.15.185:0/1643494366 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x21463e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 >> 172.21.15.185:0/1643494366 conn(0x20aee40 msgr2=0x20fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 shutdown_connections 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 1 -- 172.21.15.185:0/1643494366 wait complete. 2020-11-16T22:21:48.640 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.638+0000 7fb42d794740 20 client.7504 trim_cache size 0 max 16384 2020-11-16T22:21:48.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 Processor -- start 2020-11-16T22:21:48.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 -- start start 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x212d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x212c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2003e40 con 0x1e72bb0 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d8800 con 0x1ee07b0 2020-11-16T22:21:48.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20fb7f0 con 0x20aee40 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56056/0 (socket says 172.21.15.185:56056) 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 -- 172.21.15.185:0/3560344521 learned_addr learned my addr 172.21.15.185:0/3560344521 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x212c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb417fff700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x212d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 -- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 msgr2=0x212c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x212c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 -- 172.21.15.185:0/3560344521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 msgr2=0x212d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x212d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 -- 172.21.15.185:0/3560344521 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec008160 con 0x20aee40 2020-11-16T22:21:48.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x212c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:48.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.654+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fb418063040 tx=0x7fb410013000).ready entity=mon.2 client_cookie=ae112c059a8049d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb40e7fc700 1 -- 172.21.15.185:0/3560344521 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410011e00 con 0x20aee40 2020-11-16T22:21:48.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb40e7fc700 1 -- 172.21.15.185:0/3560344521 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410031070 con 0x20aee40 2020-11-16T22:21:48.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb40e7fc700 1 -- 172.21.15.185:0/3560344521 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41002d4f0 con 0x20aee40 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 msgr2=0x2142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fb418063040 tx=0x7fb410013000).stop 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3560344521 shutdown_connections 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ee07b0 0x212c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20aee40 0x2142370 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- 172.21.15.185:0/3560344521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1e72bb0 0x212d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3560344521 >> 172.21.15.185:0/3560344521 conn(0x7fb3ec002320 msgr2=0x2133440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3560344521 shutdown_connections 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3560344521 wait complete. 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- start start 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 Processor -- start 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 0x2194240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 0x2194730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2142f30 con 0x20aee40 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x20faa00 con 0x1e72bb0 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2142c90 con 0x1ee07b0 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 10 client.0 ll_register_callbacks cb 0x20f9f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41ca9f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34386/0 (socket says 172.21.15.185:34386) 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41ca9f700 1 -- 172.21.15.185:0/3396130322 learned_addr learned my addr 172.21.15.185:0/3396130322 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 0x2194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 0x2194730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 -- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 msgr2=0x20fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 -- 172.21.15.185:0/3396130322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 msgr2=0x2194730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:48.657 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 0x2194730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 -- 172.21.15.185:0/3396130322 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e90720 con 0x1ee07b0 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41d2a0700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 0x2194730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb417fff700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 0x2194240 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x2131c00 tx=0x7fb40004d000).ready entity=mon.2 client_cookie=d7f66ba006516a43 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb3e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:21:48.658 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400046ec0 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb40006d0d0 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb40800fbe0 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4000465c0 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 10 client.6734 fetch_fsmap learned FSMap version 90 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.658+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x1e9be00 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40006b080 con 0x1ee07b0 2020-11-16T22:21:48.659 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 90) v1 ==== 1485+0+0 (secure 0 0 0) 0x7fb40001a040 con 0x1ee07b0 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 10 client.6734 fetch_fsmap finished waiting for FSMap version 90 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 10 client.6734 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 20 client.6734 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x20feda0 0x2132010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 4 client.6734 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1f23f50 con 0x20feda0 2020-11-16T22:21:48.660 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x20feda0 0x2132010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:48.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb3e6ffd700 10 client.6734 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:48.661 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.662+0000 7fb41ca9f700 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x20feda0 0x2132010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=1b4088d2e6abe399 server_cookie=541030c695a509b4 in_seq=0 out_seq=0 2020-11-16T22:21:48.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb3e6ffd700 1 -- 172.21.15.185:0/3396130322 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1f23f50 con 0x20feda0 2020-11-16T22:21:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb3e6ffd700 10 client.6734 handle_command_reply: tid=0 2020-11-16T22:21:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 client.6734 shutdown 2020-11-16T22:21:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 msgr2=0x2194240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 0x2194240 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x2131c00 tx=0x7fb40004d000).stop 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 shutdown_connections 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x20feda0 0x2132010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1e72bb0 0x20fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20aee40 0x2194730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 --2- 172.21.15.185:0/3396130322 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ee07b0 0x2194240 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb3e6ffd700 0 client.6734 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 >> 172.21.15.185:0/3396130322 conn(0x7fb3ec002320 msgr2=0x212c880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 shutdown_connections 2020-11-16T22:21:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 1 -- 172.21.15.185:0/3396130322 wait complete. 2020-11-16T22:21:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.682+0000 7fb42d794740 20 client.6734 trim_cache size 0 max 16384 2020-11-16T22:21:48.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 msgr2=0x7fb3f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 0x7fb3f4057520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fb41000b920 tx=0x7fb410015000).stop 2020-11-16T22:21:48.680 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 msgr2=0x7fb4180b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fb40000c760 tx=0x7fb4000079a0).stop 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 shutdown_connections 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb3f4055060 0x7fb3f4057520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb418143ef0 0x7fb418150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4180b2e80 0x7fb4180afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 --2- 172.21.15.185:0/3460006795 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb4180b3920 0x7fb4180b00c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:48.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 >> 172.21.15.185:0/3460006795 conn(0x7fb418060130 msgr2=0x7fb418066720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:48.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 shutdown_connections 2020-11-16T22:21:48.682 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:48.686+0000 7fb41daa1700 1 -- 172.21.15.185:0/3460006795 wait complete. 2020-11-16T22:21:48.682 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:48.682 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:21:48.682 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:21:48.683 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:48.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:21:48.693 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:21:48.756 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/ 2020-11-16T22:21:48.756 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4 2020-11-16T22:21:48.756 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5 2020-11-16T22:21:48.757 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:21:48.757 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:21:48.757 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:21:48.761 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:48.762 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:48.762 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:48.762 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:48.763 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:21:48.763 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:48.875 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627778 2020-11-16T22:21:49.064 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:49.064 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:49.065 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:49.065 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:49.065 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:21:49.066 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:49.229 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627779 2020-11-16T22:21:49.419 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:49.419 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:49.420 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:49.420 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:49.420 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:21:49.420 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:49.532 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627780 2020-11-16T22:21:49.722 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:49.723 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:49.723 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:49.724 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:49.724 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:21:49.724 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:49.836 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627781 2020-11-16T22:21:50.025 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:50.025 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:50.026 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:50.026 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:50.026 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:21:50.026 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:50.181 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627782 2020-11-16T22:21:50.372 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:21:50.372 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:21:50.372 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:21:50.373 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:21:50.373 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:21:50.373 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:21:50.490 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627783 2020-11-16T22:21:50.680 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:21:50.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.722+0000 7ff4976d8c40 1 Processor -- start 2020-11-16T22:21:50.715 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.722+0000 7ff4976d8c40 1 -- start start 2020-11-16T22:21:50.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4aa5c60 0x5598f4b6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 0x5598f4b7c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4a541f0 con 0x5598f4aa5c60 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4a22be0 con 0x5598f4ae08d0 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598f4ae3b70 con 0x5598f4b6eb30 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4894df700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 0x5598f4b7c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34402/0 (socket says 172.21.15.185:34402) 2020-11-16T22:21:50.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 -- 172.21.15.185:0/3385959436 learned_addr learned my addr 172.21.15.185:0/3385959436 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4894df700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 0x5598f4b7c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56080/0 (socket says 172.21.15.185:56080) 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff488cde700 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4aa5c60 0x5598f4b6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 -- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 msgr2=0x5598f4b7c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 0x5598f4b7c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 -- 172.21.15.185:0/3385959436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4aa5c60 msgr2=0x5598f4b6fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4aa5c60 0x5598f4b6fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 -- 172.21.15.185:0/3385959436 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f4a4c8a0 con 0x5598f4ae08d0 2020-11-16T22:21:50.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4884dd700 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff47000b5b0 tx=0x7ff470001720).ready entity=mon.1 client_cookie=77569a434ffae4ab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4874db700 1 -- 172.21.15.185:0/3385959436 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000e430 con 0x5598f4ae08d0 2020-11-16T22:21:50.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4874db700 1 -- 172.21.15.185:0/3385959436 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47000e5b0 con 0x5598f4ae08d0 2020-11-16T22:21:50.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4874db700 1 -- 172.21.15.185:0/3385959436 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000e970 con 0x5598f4ae08d0 2020-11-16T22:21:50.722 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 msgr2=0x5598f4b71c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff47000b5b0 tx=0x7ff470001720).stop 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- 172.21.15.185:0/3385959436 shutdown_connections 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4b6eb30 0x5598f4b7c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4aa5c60 0x5598f4b6fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/3385959436 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4ae08d0 0x5598f4b71c40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- 172.21.15.185:0/3385959436 >> 172.21.15.185:0/3385959436 conn(0x5598f4ae5490 msgr2=0x5598f4ac9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.723 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- 172.21.15.185:0/3385959436 shutdown_connections 2020-11-16T22:21:50.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.726+0000 7ff4976d8c40 1 -- 172.21.15.185:0/3385959436 wait complete. 2020-11-16T22:21:50.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 Processor -- start 2020-11-16T22:21:50.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 -- start start 2020-11-16T22:21:50.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 0x5598f4adf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4ae08d0 0x5598f4abea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 0x5598f4adc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4b6da20 con 0x5598f4ae08d0 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4ade8b0 con 0x5598f4b6eb30 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4976d8c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598f4b70960 con 0x5598f4aa5c60 2020-11-16T22:21:50.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 0x5598f4adf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4894df700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 0x5598f4adc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 0x5598f4adf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56084/0 (socket says 172.21.15.185:56084) 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4894df700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 0x5598f4adc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34410/0 (socket says 172.21.15.185:34410) 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 -- 172.21.15.185:0/2815945966 learned_addr learned my addr 172.21.15.185:0/2815945966 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4884dd700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4ae08d0 0x5598f4abea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 -- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 msgr2=0x5598f4adc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 0x5598f4adc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 -- 172.21.15.185:0/2815945966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4ae08d0 msgr2=0x5598f4abea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4ae08d0 0x5598f4abea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff488cde700 1 -- 172.21.15.185:0/2815945966 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f4a48280 con 0x5598f4aa5c60 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4884dd700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598f4ae08d0 0x5598f4abea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:50.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.730+0000 7ff4894df700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5598f4b6eb30 0x5598f4adc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff488cde700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 0x5598f4adf4c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x5598f4a407c0 tx=0x7ff478013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff485cd8700 1 -- 172.21.15.185:0/2815945966 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478012ea0 con 0x5598f4aa5c60 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff485cd8700 1 -- 172.21.15.185:0/2815945966 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47801f470 con 0x5598f4aa5c60 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff485cd8700 1 -- 172.21.15.185:0/2815945966 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47801f7f0 con 0x5598f4aa5c60 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598f4a26620 con 0x5598f4aa5c60 2020-11-16T22:21:50.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598f4b705f0 con 0x5598f4aa5c60 2020-11-16T22:21:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff485cd8700 1 -- 172.21.15.185:0/2815945966 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff47801fb60 con 0x5598f4aa5c60 2020-11-16T22:21:50.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.734+0000 7ff485cd8700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff468053ce0 0x7ff4680561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff485cd8700 1 -- 172.21.15.185:0/2815945966 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff47806f650 con 0x5598f4aa5c60 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4884dd700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff468053ce0 0x7ff4680561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x5598f4b803c0 0x5598f4b8cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- osd_op(unknown.0.0:1 13.2 13:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x5598f4b8d270 con 0x5598f4b803c0 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4884dd700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff468053ce0 0x7ff4680561a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ff470000f70 tx=0x7ff470001720).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4894df700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x5598f4b803c0 0x5598f4b8cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.738+0000 7ff4894df700 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x5598f4b803c0 0x5598f4b8cd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4894df700 1 -- 172.21.15.185:0/2815945966 <== osd.1 v2:172.21.15.185:6812/15205 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v74'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff48000cc40 con 0x5598f4b803c0 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x5598f4b803c0 msgr2=0x5598f4b8cd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x5598f4b803c0 0x5598f4b8cd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff468053ce0 msgr2=0x7ff4680561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff468053ce0 0x7ff4680561a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7ff470000f70 tx=0x7ff470001720).stop 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 msgr2=0x5598f4adf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 --2- 172.21.15.185:0/2815945966 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5598f4aa5c60 0x5598f4adf4c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x5598f4a407c0 tx=0x7ff478013000).stop 2020-11-16T22:21:50.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4894df700 1 -- 172.21.15.185:0/2815945966 reap_dead start 2020-11-16T22:21:50.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 shutdown_connections 2020-11-16T22:21:50.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 >> 172.21.15.185:0/2815945966 conn(0x5598f4ae5490 msgr2=0x5598f4ae2600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 shutdown_connections 2020-11-16T22:21:50.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.742+0000 7ff4976d8c40 1 -- 172.21.15.185:0/2815945966 wait complete. 2020-11-16T22:21:50.737 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:21:50.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 Processor -- start 2020-11-16T22:21:50.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 -- start start 2020-11-16T22:21:50.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f97f69a40 0x558f97f8a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f98032490 0x558f9803b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f97f15190 con 0x558f97f69a40 2020-11-16T22:21:50.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f97f161f0 con 0x558f97f9ee40 2020-11-16T22:21:50.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f94040dbc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f97ee4be0 con 0x558f98032490 2020-11-16T22:21:50.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34424/0 (socket says 172.21.15.185:34424) 2020-11-16T22:21:50.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 -- 172.21.15.185:0/969087055 learned_addr learned my addr 172.21.15.185:0/969087055 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f5ee2700 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f98032490 0x558f9803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f56e1700 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f97f69a40 0x558f97f8a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 -- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f98032490 msgr2=0x558f9803b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f98032490 0x558f9803b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 -- 172.21.15.185:0/969087055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f97f69a40 msgr2=0x558f97f8a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f97f69a40 0x558f97f8a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 -- 172.21.15.185:0/969087055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f97f0a280 con 0x558f97f9ee40 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f93e800b640 tx=0x7f93e8001820).ready entity=mon.1 client_cookie=e929601d9228a254 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93ef7fe700 1 -- 172.21.15.185:0/969087055 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e800e430 con 0x558f97f9ee40 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93ef7fe700 1 -- 172.21.15.185:0/969087055 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e800e590 con 0x558f97f9ee40 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.822+0000 7f93ef7fe700 1 -- 172.21.15.185:0/969087055 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e800e910 con 0x558f97f9ee40 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 msgr2=0x558f97f67550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f93e800b640 tx=0x7f93e8001820).stop 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- 172.21.15.185:0/969087055 shutdown_connections 2020-11-16T22:21:50.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f98032490 0x558f9803b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f97f69a40 0x558f97f8a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- 172.21.15.185:0/969087055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f97f67550 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- 172.21.15.185:0/969087055 >> 172.21.15.185:0/969087055 conn(0x558f97fa2370 msgr2=0x558f97f67f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- 172.21.15.185:0/969087055 shutdown_connections 2020-11-16T22:21:50.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- 172.21.15.185:0/969087055 wait complete. 2020-11-16T22:21:50.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 Processor -- start 2020-11-16T22:21:50.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- start start 2020-11-16T22:21:50.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f97f69a40 0x558f97f89c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f98030470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f98032490 0x558f98030960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f97f88ef0 con 0x558f98032490 2020-11-16T22:21:50.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9803ca50 con 0x558f97f9ee40 2020-11-16T22:21:50.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f94040dbc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f9803be40 con 0x558f97f69a40 2020-11-16T22:21:50.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f93f4ee0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f98030470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f93f4ee0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f98030470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34430/0 (socket says 172.21.15.185:34430) 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f93f4ee0700 1 -- 172.21.15.185:0/1444624203 learned_addr learned my addr 172.21.15.185:0/1444624203 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.826+0000 7f93f56e1700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f97f69a40 0x558f97f89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f5ee2700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f98032490 0x558f98030960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 -- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f97f69a40 msgr2=0x558f97f89c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f97f69a40 0x558f97f89c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 -- 172.21.15.185:0/1444624203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f98032490 msgr2=0x558f98030960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f98032490 0x558f98030960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 -- 172.21.15.185:0/1444624203 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f97ee8620 con 0x558f97f9ee40 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f56e1700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x558f97f69a40 0x558f97f89c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f4ee0700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f98030470 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f93e8004130 tx=0x7f93e8014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 -- 172.21.15.185:0/1444624203 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e801ac10 con 0x558f97f9ee40 2020-11-16T22:21:50.825 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 -- 172.21.15.185:0/1444624203 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e801f070 con 0x558f97f9ee40 2020-11-16T22:21:50.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 -- 172.21.15.185:0/1444624203 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e801ad90 con 0x558f97f9ee40 2020-11-16T22:21:50.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f9803c070 con 0x558f97f9ee40 2020-11-16T22:21:50.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f9803c5b0 con 0x558f97f9ee40 2020-11-16T22:21:50.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 -- 172.21.15.185:0/1444624203 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f93e802b070 con 0x558f97f9ee40 2020-11-16T22:21:50.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f93d8053ce0 0x7f93d80561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93edffb700 1 -- 172.21.15.185:0/1444624203 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f93e806ed80 con 0x558f97f9ee40 2020-11-16T22:21:50.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f56e1700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f93d8053ce0 0x7f93d80561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f94040dbc40 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x558f97f89110 0x558f9804e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- osd_op(unknown.0.0:1 13.2 13:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x558f9804ed50 con 0x558f97f89110 2020-11-16T22:21:50.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f5ee2700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x558f97f89110 0x558f9804e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.830+0000 7f93f56e1700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f93d8053ce0 0x7f93d80561a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f93e000c010 tx=0x7f93e0013000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f93f5ee2700 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x558f97f89110 0x558f9804e860 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f93f5ee2700 1 -- 172.21.15.185:0/1444624203 <== osd.1 v2:172.21.15.185:6812/15205 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v74'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f93f0006c40 con 0x558f97f89110 2020-11-16T22:21:50.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x558f97f89110 msgr2=0x558f9804e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x558f97f89110 0x558f9804e860 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f93d8053ce0 msgr2=0x7f93d80561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f93d8053ce0 0x7f93d80561a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f93e000c010 tx=0x7f93e0013000).stop 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 msgr2=0x558f98030470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 --2- 172.21.15.185:0/1444624203 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x558f97f9ee40 0x558f98030470 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f93e8004130 tx=0x7f93e8014000).stop 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f93f5ee2700 1 -- 172.21.15.185:0/1444624203 reap_dead start 2020-11-16T22:21:50.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 shutdown_connections 2020-11-16T22:21:50.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 >> 172.21.15.185:0/1444624203 conn(0x558f97fa2370 msgr2=0x558f97f68090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 shutdown_connections 2020-11-16T22:21:50.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.834+0000 7f94040dbc40 1 -- 172.21.15.185:0/1444624203 wait complete. 2020-11-16T22:21:50.832 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:21:50.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 Processor -- start 2020-11-16T22:21:50.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 -- start start 2020-11-16T22:21:50.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041295fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x55904130b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590411d4be0 con 0x559041257fc0 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041323c00 con 0x55904128ca80 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x559041326b30 con 0x559041290620 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b6b97700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x55904130b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56124/0 (socket says 172.21.15.185:56124) 2020-11-16T22:21:50.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b6b97700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x55904130b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34446/0 (socket says 172.21.15.185:34446) 2020-11-16T22:21:50.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7398700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041295fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 -- 172.21.15.185:0/3129870450 learned_addr learned my addr 172.21.15.185:0/3129870450 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 -- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 msgr2=0x55904130b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x55904130b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 -- 172.21.15.185:0/3129870450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 msgr2=0x559041295fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041295fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.918+0000 7f05b7b99700 1 -- 172.21.15.185:0/3129870450 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590411fd480 con 0x559041290620 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f05b000ba20 tx=0x7f05b0005d00).ready entity=mon.2 client_cookie=65234164c6a90489 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b5b95700 1 -- 172.21.15.185:0/3129870450 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000d8a0 con 0x559041290620 2020-11-16T22:21:50.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b5b95700 1 -- 172.21.15.185:0/3129870450 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000da00 con 0x559041290620 2020-11-16T22:21:50.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b5b95700 1 -- 172.21.15.185:0/3129870450 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000dda0 con 0x559041290620 2020-11-16T22:21:50.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 msgr2=0x559041294cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f05b000ba20 tx=0x7f05b0005d00).stop 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3129870450 shutdown_connections 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041294cd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041295fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3129870450 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x55904130b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3129870450 >> 172.21.15.185:0/3129870450 conn(0x559041298be0 msgr2=0x55904128e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3129870450 shutdown_connections 2020-11-16T22:21:50.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3129870450 wait complete. 2020-11-16T22:21:50.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 Processor -- start 2020-11-16T22:21:50.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- start start 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041291f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x559041322a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041322f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041257c90 con 0x559041257fc0 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x559041333400 con 0x55904128ca80 2020-11-16T22:21:50.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05c5d92c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590413336a0 con 0x559041290620 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041322f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041322f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56130/0 (socket says 172.21.15.185:56130) 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 -- 172.21.15.185:0/3771241255 learned_addr learned my addr 172.21.15.185:0/3771241255 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b6b97700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x559041322a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7398700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041291f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 -- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 msgr2=0x559041322a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55904128ca80 0x559041322a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 -- 172.21.15.185:0/3771241255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 msgr2=0x559041291f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041291f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7b99700 1 -- 172.21.15.185:0/3771241255 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590411fe8a0 con 0x559041290620 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.922+0000 7f05b7398700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559041257fc0 0x559041291f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05b7b99700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041322f40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f05b000c010 tx=0x7f05b0018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 -- 172.21.15.185:0/3771241255 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000d8a0 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 -- 172.21.15.185:0/3771241255 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0011e50 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 -- 172.21.15.185:0/3771241255 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b001c410 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590411fa280 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590411d8620 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 -- 172.21.15.185:0/3771241255 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f05b002c070 con 0x559041290620 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f059c053ce0 0x7f059c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05b7398700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f059c053ce0 0x7f059c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05affff700 1 -- 172.21.15.185:0/3771241255 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05b006ee90 con 0x559041290620 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x559041323430 0x55904133f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- osd_op(unknown.0.0:1 13.5 13:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x559041334d90 con 0x559041323430 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05b7398700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f059c053ce0 0x7f059c0561a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x559041299190 tx=0x7f05a000c040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05b6b97700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x559041323430 0x55904133f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:50.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.926+0000 7f05b6b97700 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x559041323430 0x55904133f100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:50.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05b6b97700 1 -- 172.21.15.185:0/3771241255 <== osd.2 v2:172.21.15.185:6800/15193 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v74'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f05a8000b80 con 0x559041323430 2020-11-16T22:21:50.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x559041323430 msgr2=0x55904133f100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x559041323430 0x55904133f100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f059c053ce0 msgr2=0x7f059c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f059c053ce0 0x7f059c0561a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x559041299190 tx=0x7f05a000c040).stop 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 msgr2=0x559041322f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 --2- 172.21.15.185:0/3771241255 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x559041290620 0x559041322f40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f05b000c010 tx=0x7f05b0018000).stop 2020-11-16T22:21:50.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05b7b99700 1 -- 172.21.15.185:0/3771241255 reap_dead start 2020-11-16T22:21:50.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 shutdown_connections 2020-11-16T22:21:50.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 >> 172.21.15.185:0/3771241255 conn(0x559041298be0 msgr2=0x559041324df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:50.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 shutdown_connections 2020-11-16T22:21:50.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:50.930+0000 7f05c5d92c40 1 -- 172.21.15.185:0/3771241255 wait complete. 2020-11-16T22:21:50.928 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:21:51.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 Processor -- start 2020-11-16T22:21:51.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- start start 2020-11-16T22:21:51.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55863412b8d0 0x5586341b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863409e190 con 0x55863412b8d0 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863409f1f0 con 0x5586340f0c60 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55863406dbe0 con 0x5586341b9b30 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67f7c2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67f7c2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34466/0 (socket says 172.21.15.185:34466) 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67f7c2700 1 -- 172.21.15.185:0/2207721301 learned_addr learned my addr 172.21.15.185:0/2207721301 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67efc1700 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55863412b8d0 0x5586341b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 -- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 msgr2=0x5586341b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 -- 172.21.15.185:0/2207721301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55863412b8d0 msgr2=0x5586341b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55863412b8d0 0x5586341b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 -- 172.21.15.185:0/2207721301 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558634093280 con 0x5586341b9b30 2020-11-16T22:21:51.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67f7c2700 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341be310 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fc67800b4d0 tx=0x7fc678001830).ready entity=mon.2 client_cookie=501bfeddfb849ef1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67dfbf700 1 -- 172.21.15.185:0/2207721301 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800be70 con 0x5586341b9b30 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67dfbf700 1 -- 172.21.15.185:0/2207721301 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67800d490 con 0x5586341b9b30 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc67dfbf700 1 -- 172.21.15.185:0/2207721301 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800d810 con 0x5586341b9b30 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 msgr2=0x5586341be310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341be310 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fc67800b4d0 tx=0x7fc678001830).stop 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/2207721301 shutdown_connections 2020-11-16T22:21:51.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5586340f0c60 0x5586341b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341be310 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/2207721301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55863412b8d0 0x5586341b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/2207721301 >> 172.21.15.185:0/2207721301 conn(0x558634130490 msgr2=0x5586341b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/2207721301 shutdown_connections 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.014+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/2207721301 wait complete. 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 Processor -- start 2020-11-16T22:21:51.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- start start 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586340f0c60 0x5586341c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55863412b8d0 0x5586341be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586341bd900 con 0x5586340f0c60 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586341c9b70 con 0x55863412b8d0 2020-11-16T22:21:51.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586341bdb30 con 0x5586341b9b30 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67efc1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55863412b8d0 0x5586341be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67efc1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55863412b8d0 0x5586341be7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34474/0 (socket says 172.21.15.185:34474) 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56152/0 (socket says 172.21.15.185:56152) 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 -- 172.21.15.185:0/1957363482 learned_addr learned my addr 172.21.15.185:0/1957363482 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67f7c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586340f0c60 0x5586341c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 -- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55863412b8d0 msgr2=0x5586341be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55863412b8d0 0x5586341be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 -- 172.21.15.185:0/1957363482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586340f0c60 msgr2=0x5586341c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586340f0c60 0x5586341c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 -- 172.21.15.185:0/1957363482 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558634071620 con 0x5586341b9b30 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc67ffc3700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341b5fa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc67800f000 tx=0x7fc678001830).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc667fff700 1 -- 172.21.15.185:0/1957363482 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678001ea0 con 0x5586341b9b30 2020-11-16T22:21:51.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc667fff700 1 -- 172.21.15.185:0/1957363482 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67800d810 con 0x5586341b9b30 2020-11-16T22:21:51.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5586341bddd0 con 0x5586341b9b30 2020-11-16T22:21:51.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc667fff700 1 -- 172.21.15.185:0/1957363482 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011b00 con 0x5586341b9b30 2020-11-16T22:21:51.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.018+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586341be310 con 0x5586341b9b30 2020-11-16T22:21:51.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc667fff700 1 -- 172.21.15.185:0/1957363482 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc678024400 con 0x5586341b9b30 2020-11-16T22:21:51.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc667fff700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc66c053ce0 0x7fc66c0561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc67f7c2700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc66c053ce0 0x7fc66c0561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc667fff700 1 -- 172.21.15.185:0/1957363482 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc67806e080 con 0x5586341b9b30 2020-11-16T22:21:51.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x55863412e820 0x5586341b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- osd_op(unknown.0.0:1 13.1 13:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x5586341d5540 con 0x55863412e820 2020-11-16T22:21:51.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc67f7c2700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc66c053ce0 0x7fc66c0561a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fc67000ae50 tx=0x7fc670012000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc67efc1700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x55863412e820 0x5586341b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.022+0000 7fc67efc1700 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x55863412e820 0x5586341b3070 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc67efc1700 1 -- 172.21.15.185:0/1957363482 <== osd.1 v2:172.21.15.185:6812/15205 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v74'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc6740015e0 con 0x55863412e820 2020-11-16T22:21:51.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x55863412e820 msgr2=0x5586341b3070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x55863412e820 0x5586341b3070 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc66c053ce0 msgr2=0x7fc66c0561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc66c053ce0 0x7fc66c0561a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fc67000ae50 tx=0x7fc670012000).stop 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 msgr2=0x5586341b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 --2- 172.21.15.185:0/1957363482 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5586341b9b30 0x5586341b5fa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fc67800f000 tx=0x7fc678001830).stop 2020-11-16T22:21:51.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc67ffc3700 1 -- 172.21.15.185:0/1957363482 reap_dead start 2020-11-16T22:21:51.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 shutdown_connections 2020-11-16T22:21:51.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 >> 172.21.15.185:0/1957363482 conn(0x558634130490 msgr2=0x558634114ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 shutdown_connections 2020-11-16T22:21:51.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.026+0000 7fc68e1bcc40 1 -- 172.21.15.185:0/1957363482 wait complete. 2020-11-16T22:21:51.022 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:21:51.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.106+0000 7f7a0e3a1c40 1 Processor -- start 2020-11-16T22:21:51.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.106+0000 7f7a0e3a1c40 1 -- start start 2020-11-16T22:21:51.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb6265fc0 0x557bb632b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 0x557bb6339ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb62141f0 con 0x557bb6265fc0 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb61e2be0 con 0x557bb629e620 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bb6334dd0 con 0x557bb629aa80 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 0x557bb6339ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 0x557bb6339ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56166/0 (socket says 172.21.15.185:56166) 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34492/0 (socket says 172.21.15.185:34492) 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 -- 172.21.15.185:0/411359903 learned_addr learned my addr 172.21.15.185:0/411359903 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff9a7700 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb6265fc0 0x557bb632b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 -- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 msgr2=0x557bb6339ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.103 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 0x557bb6339ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 -- 172.21.15.185:0/411359903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb6265fc0 msgr2=0x557bb632b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb6265fc0 0x557bb632b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 -- 172.21.15.185:0/411359903 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb620c8a0 con 0x557bb629e620 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f79f800b500 tx=0x7f79f80016c0).ready entity=mon.1 client_cookie=60f9cea3dcf6182 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79fe1a4700 1 -- 172.21.15.185:0/411359903 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8010420 con 0x557bb629e620 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79fe1a4700 1 -- 172.21.15.185:0/411359903 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8010580 con 0x557bb629e620 2020-11-16T22:21:51.104 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79fe1a4700 1 -- 172.21.15.185:0/411359903 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8010940 con 0x557bb629e620 2020-11-16T22:21:51.106 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 msgr2=0x557bb6339fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f79f800b500 tx=0x7f79f80016c0).stop 2020-11-16T22:21:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/411359903 shutdown_connections 2020-11-16T22:21:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629e620 0x557bb6339fa0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb6265fc0 0x557bb632b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.107 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/411359903 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb629aa80 0x557bb6339ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/411359903 >> 172.21.15.185:0/411359903 conn(0x557bb62a6be0 msgr2=0x557bb629c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/411359903 shutdown_connections 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/411359903 wait complete. 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 Processor -- start 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- start start 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb6265fc0 0x557bb6339c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.108 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 0x557bb632fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb629e620 0x557bb632ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb63418f0 con 0x557bb629e620 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb6341b90 con 0x557bb629aa80 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a0e3a1c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bb63389c0 con 0x557bb6265fc0 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 0x557bb632fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 0x557bb632fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34498/0 (socket says 172.21.15.185:34498) 2020-11-16T22:21:51.109 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff1a6700 1 -- 172.21.15.185:0/3813448909 learned_addr learned my addr 172.21.15.185:0/3813448909 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f7a001a8700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb629e620 0x557bb632ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.110+0000 7f79ff9a7700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb6265fc0 0x557bb6339c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 -- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb6265fc0 msgr2=0x557bb6339c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557bb6265fc0 0x557bb6339c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 -- 172.21.15.185:0/3813448909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb629e620 msgr2=0x557bb632ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb629e620 0x557bb632ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 -- 172.21.15.185:0/3813448909 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb6208280 con 0x557bb629aa80 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f7a001a8700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bb629e620 0x557bb632ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79ff1a6700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 0x557bb632fab0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f79f0008d50 tx=0x7f79f000c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79fc9a1700 1 -- 172.21.15.185:0/3813448909 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000f040 con 0x557bb629aa80 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79fc9a1700 1 -- 172.21.15.185:0/3813448909 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0010b60 con 0x557bb629aa80 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557bb61e6620 con 0x557bb629aa80 2020-11-16T22:21:51.111 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f79fc9a1700 1 -- 172.21.15.185:0/3813448909 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0010ee0 con 0x557bb629aa80 2020-11-16T22:21:51.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.114+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bb6337370 con 0x557bb629aa80 2020-11-16T22:21:51.112 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f79fc9a1700 1 -- 172.21.15.185:0/3813448909 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f79f0015680 con 0x557bb629aa80 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f79fc9a1700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79e0053ce0 0x7f79e00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f79fc9a1700 1 -- 172.21.15.185:0/3813448909 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79f006d960 con 0x557bb629aa80 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f79ff9a7700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79e0053ce0 0x7f79e00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x557bb6338e60 0x557bb634d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- osd_op(unknown.0.0:1 13.5 13:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x557bb6335180 con 0x557bb6338e60 2020-11-16T22:21:51.113 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f79ff9a7700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79e0053ce0 0x7f79e00561a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f79e80081b0 tx=0x7f79e8014000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f7a001a8700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x557bb6338e60 0x557bb634d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.118+0000 7f7a001a8700 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x557bb6338e60 0x557bb634d610 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a001a8700 1 -- 172.21.15.185:0/3813448909 <== osd.2 v2:172.21.15.185:6800/15193 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v74'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f79f801b3f0 con 0x557bb6338e60 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x557bb6338e60 msgr2=0x557bb634d610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x557bb6338e60 0x557bb634d610 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79e0053ce0 msgr2=0x7f79e00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f79e0053ce0 0x7f79e00561a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f79e80081b0 tx=0x7f79e8014000).stop 2020-11-16T22:21:51.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 msgr2=0x557bb632fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 --2- 172.21.15.185:0/3813448909 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557bb629aa80 0x557bb632fab0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f79f0008d50 tx=0x7f79f000c040).stop 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a001a8700 1 -- 172.21.15.185:0/3813448909 reap_dead start 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 shutdown_connections 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 >> 172.21.15.185:0/3813448909 conn(0x557bb62a6be0 msgr2=0x557bb632b280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 shutdown_connections 2020-11-16T22:21:51.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.122+0000 7f7a0e3a1c40 1 -- 172.21.15.185:0/3813448909 wait complete. 2020-11-16T22:21:51.117 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:21:51.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 Processor -- start 2020-11-16T22:21:51.193 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 -- start start 2020-11-16T22:21:51.194 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a71ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd19b5be0 con 0x55ecd1b01b30 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1a72580 con 0x55ecd1a38c60 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1b037e0 con 0x55ecd1a3c750 2020-11-16T22:21:51.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34510/0 (socket says 172.21.15.185:34510) 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 -- 172.21.15.185:0/2828634881 learned_addr learned my addr 172.21.15.185:0/2828634881 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeeea39700 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 -- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 msgr2=0x55ecd1a71ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a71ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 -- 172.21.15.185:0/2828634881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 msgr2=0x55ecd1a72090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a72090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 -- 172.21.15.185:0/2828634881 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd19de480 con 0x55ecd1a38c60 2020-11-16T22:21:51.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.202+0000 7fdeee238700 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fdee000bb40 tx=0x7fdee0005d00).ready entity=mon.1 client_cookie=78b827e121a7bd0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.206+0000 7fdeeca35700 1 -- 172.21.15.185:0/2828634881 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000da90 con 0x55ecd1a38c60 2020-11-16T22:21:51.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.206+0000 7fdeeca35700 1 -- 172.21.15.185:0/2828634881 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee000dbf0 con 0x55ecd1a38c60 2020-11-16T22:21:51.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.206+0000 7fdeeca35700 1 -- 172.21.15.185:0/2828634881 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00113f0 con 0x55ecd1a38c60 2020-11-16T22:21:51.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 msgr2=0x55ecd1a767d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fdee000bb40 tx=0x7fdee0005d00).stop 2020-11-16T22:21:51.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/2828634881 shutdown_connections 2020-11-16T22:21:51.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a767d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/2828634881 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a71ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:21:51.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/2828634881 >> 172.21.15.185:0/2828634881 conn(0x55ecd1a78490 msgr2=0x55ecd1b04060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/2828634881 shutdown_connections 2020-11-16T22:21:51.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/2828634881 wait complete. 2020-11-16T22:21:51.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 Processor -- start 2020-11-16T22:21:51.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- start start 2020-11-16T22:21:51.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a4d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a4d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1a73780 con 0x55ecd1b01b30 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1a50140 con 0x55ecd1a38c60 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1a503a0 con 0x55ecd1a3c750 2020-11-16T22:21:51.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a4d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a4d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56192/0 (socket says 172.21.15.185:56192) 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 -- 172.21.15.185:0/443587394 learned_addr learned my addr 172.21.15.185:0/443587394 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeee238700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeea39700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 -- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 msgr2=0x55ecd1a726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55ecd1a38c60 0x55ecd1a726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 -- 172.21.15.185:0/443587394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 msgr2=0x55ecd1a4d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a4d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 -- 172.21.15.185:0/443587394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd19df8a0 con 0x55ecd1a3c750 2020-11-16T22:21:51.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeea39700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ecd1b01b30 0x55ecd1a4d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:51.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdeeda37700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a4d450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fded8005b10 tx=0x7fded8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdee6ffd700 1 -- 172.21.15.185:0/443587394 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8012b30 con 0x55ecd1a3c750 2020-11-16T22:21:51.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdee6ffd700 1 -- 172.21.15.185:0/443587394 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8012c90 con 0x55ecd1a3c750 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ecd19db280 con 0x55ecd1a3c750 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdee6ffd700 1 -- 172.21.15.185:0/443587394 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded801c620 con 0x55ecd1a3c750 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.210+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecd19b9620 con 0x55ecd1a3c750 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdee6ffd700 1 -- 172.21.15.185:0/443587394 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fded801a040 con 0x55ecd1a3c750 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdee6ffd700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fded0053d10 0x7fded00561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdee6ffd700 1 -- 172.21.15.185:0/443587394 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fded806e8c0 con 0x55ecd1a3c750 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdeee238700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fded0053d10 0x7fded00561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55ecd1a72bb0 0x55ecd1b13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 --> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] -- osd_op(unknown.0.0:1 13.5 13:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e74) v8 -- 0x55ecd1a4de30 con 0x55ecd1a72bb0 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdeee238700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fded0053d10 0x7fded00561d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fdee000c010 tx=0x7fdee0018000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdeeea39700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55ecd1a72bb0 0x55ecd1b13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.214+0000 7fdeeea39700 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55ecd1a72bb0 0x55ecd1b13ee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdeeea39700 1 -- 172.21.15.185:0/443587394 <== osd.2 v2:172.21.15.185:6800/15193 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v74'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdee8012040 con 0x55ecd1a72bb0 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55ecd1a72bb0 msgr2=0x55ecd1b13ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6800/15193,v1:172.21.15.185:6801/15193] conn(0x55ecd1a72bb0 0x55ecd1b13ee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fded0053d10 msgr2=0x7fded00561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fded0053d10 0x7fded00561d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fdee000c010 tx=0x7fdee0018000).stop 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 msgr2=0x55ecd1a4d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 --2- 172.21.15.185:0/443587394 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55ecd1a3c750 0x55ecd1a4d450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fded8005b10 tx=0x7fded8015000).stop 2020-11-16T22:21:51.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdeeea39700 1 -- 172.21.15.185:0/443587394 reap_dead start 2020-11-16T22:21:51.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 shutdown_connections 2020-11-16T22:21:51.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 >> 172.21.15.185:0/443587394 conn(0x55ecd1a78490 msgr2=0x55ecd1b02a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 shutdown_connections 2020-11-16T22:21:51.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.218+0000 7fdefcc32c40 1 -- 172.21.15.185:0/443587394 wait complete. 2020-11-16T22:21:51.213 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:51.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 Processor -- start 2020-11-16T22:21:51.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 -- start start 2020-11-16T22:21:51.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 0x7f2784066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f27840acc10 0x7f2784147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 0x7f2784147a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840abf90 con 0x7f2784061e60 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840ac230 con 0x7f27840acc10 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f278974f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840ac4d0 con 0x7f27840669a0 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2783fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 0x7f2784066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f27840acc10 0x7f2784147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2783fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 0x7f2784066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56202/0 (socket says 172.21.15.185:56202) 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2783fff700 1 -- 172.21.15.185:0/2041322997 learned_addr learned my addr 172.21.15.185:0/2041322997 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2788f4e700 1 --2- 172.21.15.185:0/2041322997 >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 0x7f2784147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 -- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 msgr2=0x7f2784066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 --2- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 0x7f2784066d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 -- 172.21.15.185:0/2041322997 >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 msgr2=0x7f2784147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 --2- 172.21.15.185:0/2041322997 >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 0x7f2784147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 -- 172.21.15.185:0/2041322997 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840ac770 con 0x7f27840acc10 2020-11-16T22:21:51.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2788f4e700 1 --2- 172.21.15.185:0/2041322997 >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 0x7f2784147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f27837fe700 1 --2- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840acc10 0x7f2784147550 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f27840b4070 tx=0x7f276c006df0).ready entity=mon.1 client_cookie=b2917af98ffdfcfa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2782ffd700 1 -- 172.21.15.185:0/2041322997 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c009640 con 0x7f27840acc10 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2782ffd700 1 -- 172.21.15.185:0/2041322997 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c0097a0 con 0x7f27840acc10 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.394+0000 7f2782ffd700 1 -- 172.21.15.185:0/2041322997 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c009b20 con 0x7f27840acc10 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840acc10 msgr2=0x7f2784147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840acc10 0x7f2784147550 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f27840b4070 tx=0x7f276c006df0).stop 2020-11-16T22:21:51.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- 172.21.15.185:0/2041322997 shutdown_connections 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- 172.21.15.185:0/2041322997 >> v2:172.21.15.96:3300/0 conn(0x7f2784061e60 0x7f2784147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f27840669a0 0x7f2784066d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- 172.21.15.185:0/2041322997 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840acc10 0x7f2784147550 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- 172.21.15.185:0/2041322997 >> 172.21.15.185:0/2041322997 conn(0x7f2784060130 msgr2=0x7f2784060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- 172.21.15.185:0/2041322997 shutdown_connections 2020-11-16T22:21:51.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- 172.21.15.185:0/2041322997 wait complete. 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 Processor -- start 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- start start 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 0x7f2784154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27840669a0 0x7f27841476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27841406d0 con 0x7f27840669a0 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784154980 con 0x7f27840b1860 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f278974f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784114230 con 0x7f2784061e60 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2783fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 0x7f2784154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34532/0 (socket says 172.21.15.185:34532) 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2783fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 0x7f2784154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56206/0 (socket says 172.21.15.185:56206) 2020-11-16T22:21:51.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 -- 172.21.15.185:0/1257351422 learned_addr learned my addr 172.21.15.185:0/1257351422 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f27837fe700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27840669a0 0x7f27841476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 -- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 msgr2=0x7f2784154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 0x7f2784154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 -- 172.21.15.185:0/1257351422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27840669a0 msgr2=0x7f27841476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27840669a0 0x7f27841476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 -- 172.21.15.185:0/1257351422 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27841144d0 con 0x7f27840b1860 2020-11-16T22:21:51.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f2788f4e700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f277c00ef50 tx=0x7f277c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.398+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00cbf0 con 0x7f27840b1860 2020-11-16T22:21:51.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c00cd50 con 0x7f27840b1860 2020-11-16T22:21:51.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c012620 con 0x7f27840b1860 2020-11-16T22:21:51.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2784114770 con 0x7f27840b1860 2020-11-16T22:21:51.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2784147bc0 con 0x7f27840b1860 2020-11-16T22:21:51.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f277c022070 con 0x7f27840b1860 2020-11-16T22:21:51.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f277bfff700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 0x7f2760057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278405d200 con 0x7f27840b1860 2020-11-16T22:21:51.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f277c06e5f0 con 0x7f27840b1860 2020-11-16T22:21:51.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f2783fff700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 0x7f2760057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.402+0000 7f2783fff700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 0x7f2760057520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f2784113ec0 tx=0x7f2774008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.400 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.406+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f277c044850 con 0x7f27840b1860 2020-11-16T22:21:51.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.606+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27840b36e0 con 0x7f27840b1860 2020-11-16T22:21:51.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.606+0000 7f277bfff700 1 -- 172.21.15.185:0/1257351422 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f277c016090 con 0x7f27840b1860 2020-11-16T22:21:51.600 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:51.600 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:49.075839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:51.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 msgr2=0x7f2760057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 0x7f2760057520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f2784113ec0 tx=0x7f2774008040).stop 2020-11-16T22:21:51.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 msgr2=0x7f27840ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f277c00ef50 tx=0x7f277c0091d0).stop 2020-11-16T22:21:51.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 shutdown_connections 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f2760055060 0x7f2760057520 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f27840b1860 0x7f27840ad540 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f2784061e60 0x7f2784154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 --2- 172.21.15.185:0/1257351422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27840669a0 0x7f27841476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 >> 172.21.15.185:0/1257351422 conn(0x7f2784060130 msgr2=0x7f2784141100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 shutdown_connections 2020-11-16T22:21:51.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.610+0000 7f278974f700 1 -- 172.21.15.185:0/1257351422 wait complete. 2020-11-16T22:21:51.604 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 91 2020-11-16T22:21:51.615 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub start /d1/d2/d3 recursive force 2020-11-16T22:21:51.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 Processor -- start 2020-11-16T22:21:51.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 -- start start 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3c140b7170 0x7f3c140b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3c140acee0 0x7f3c140b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c140ad370 con 0x7f3c14148160 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14144f80 con 0x7f3c140acee0 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1ace2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14144170 con 0x7f3c140b7170 2020-11-16T22:21:51.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3c140b7170 0x7f3c140b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3c140b7170 0x7f3c140b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34538/0 (socket says 172.21.15.185:34538) 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 -- 172.21.15.185:0/1421818306 learned_addr learned my addr 172.21.15.185:0/1421818306 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/1421818306 >> v2:172.21.15.96:3300/0 conn(0x7f3c140acee0 0x7f3c140b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 -- 172.21.15.185:0/1421818306 >> v2:172.21.15.96:3300/0 conn(0x7f3c140acee0 msgr2=0x7f3c140b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1421818306 >> v2:172.21.15.96:3300/0 conn(0x7f3c140acee0 0x7f3c140b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 -- 172.21.15.185:0/1421818306 >> v1:172.21.15.96:6789/0 conn(0x7f3c14148160 legacy=0x7f3c14144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 -- 172.21.15.185:0/1421818306 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140b6ed0 con 0x7f3c140b7170 2020-11-16T22:21:51.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1421818306 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c140b7170 0x7f3c140b7550 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f3c140bb920 tx=0x7f3c10006e50).ready entity=mon.1 client_cookie=78dd6caa3e9e6d02 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c18cde700 1 -- 172.21.15.185:0/1421818306 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10006030 con 0x7f3c140b7170 2020-11-16T22:21:51.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c18cde700 1 -- 172.21.15.185:0/1421818306 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10006190 con 0x7f3c140b7170 2020-11-16T22:21:51.749 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.754+0000 7f3c18cde700 1 -- 172.21.15.185:0/1421818306 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10006510 con 0x7f3c140b7170 2020-11-16T22:21:51.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1421818306 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c140b7170 msgr2=0x7f3c140b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1421818306 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c140b7170 0x7f3c140b7550 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f3c140bb920 tx=0x7f3c10006e50).stop 2020-11-16T22:21:51.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1421818306 shutdown_connections 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1421818306 >> v2:172.21.15.96:3300/0 conn(0x7f3c140acee0 0x7f3c140b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1421818306 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c140b7170 0x7f3c140b7550 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1421818306 >> 172.21.15.185:0/1421818306 conn(0x7f3c14067860 msgr2=0x7f3c14067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1421818306 shutdown_connections 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1421818306 wait complete. 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 Processor -- start 2020-11-16T22:21:51.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- start start 2020-11-16T22:21:51.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 0x7f3c140afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 0x7f3c14152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141535c0 con 0x7f3c140acee0 2020-11-16T22:21:51.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140ba930 con 0x7f3c14148160 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1ace2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1413e700 con 0x7f3c140b7170 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1a4e1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1a4e1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34546/0 (socket says 172.21.15.185:34546) 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/1351737534 learned_addr learned my addr 172.21.15.185:0/1351737534 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c194df700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 0x7f3c14152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.758+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 0x7f3c140afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 msgr2=0x7f3c14152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 0x7f3c14152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/1351737534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 msgr2=0x7f3c140afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 0x7f3c140afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/1351737534 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140bb5d0 con 0x7f3c14148160 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 0x7f3c140afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:51.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c194df700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 0x7f3c14152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3c0c00ef80 tx=0x7f3c0c0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 -- 172.21.15.185:0/1351737534 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0096f0 con 0x7f3c14148160 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 -- 172.21.15.185:0/1351737534 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c00cd40 con 0x7f3c14148160 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c14154200 con 0x7f3c14148160 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 -- 172.21.15.185:0/1351737534 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c012640 con 0x7f3c14148160 2020-11-16T22:21:51.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c140b0120 con 0x7f3c14148160 2020-11-16T22:21:51.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 -- 172.21.15.185:0/1351737534 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3c0c022070 con 0x7f3c14148160 2020-11-16T22:21:51.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 0x7f3bec057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c01ffb700 1 -- 172.21.15.185:0/1351737534 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c0c06e630 con 0x7f3c14148160 2020-11-16T22:21:51.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.762+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 0x7f3bec057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 0x7f3bec057520 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f3c1000b1b0 tx=0x7f3c10015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 Processor -- start 2020-11-16T22:21:51.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 -- start start 2020-11-16T22:21:51.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2618f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x2619430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x24a6560 con 0x238ec60 2020-11-16T22:21:51.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x255a9e0 con 0x23fc860 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x241a7a0 con 0x261ae50 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34552/0 (socket says 172.21.15.185:34552) 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3106300022 learned_addr learned my addr 172.21.15.185:0/3106300022 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x2619430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.766+0000 7f3c194df700 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2618f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 msgr2=0x2619430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x2619430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3106300022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 msgr2=0x2618f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2618f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3106300022 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2415650 con 0x23fc860 2020-11-16T22:21:51.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f3c141442b0 tx=0x7f3c1001e000).ready entity=mon.1 client_cookie=ce0d21de482409aa server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/3106300022 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100058e0 con 0x23fc860 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/3106300022 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10005a40 con 0x23fc860 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3bf2ffd700 1 -- 172.21.15.185:0/3106300022 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10005dc0 con 0x23fc860 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 msgr2=0x261a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f3c141442b0 tx=0x7f3c1001e000).stop 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3106300022 shutdown_connections 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261a960 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x2619430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3106300022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2618f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3106300022 >> 172.21.15.185:0/3106300022 conn(0x25caef0 msgr2=0x2617e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3106300022 shutdown_connections 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3106300022 wait complete. 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- start start 2020-11-16T22:21:51.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 Processor -- start 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2616670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x26626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2547790 con 0x238ec60 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x25478d0 con 0x23fc860 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2548b40 con 0x261ae50 2020-11-16T22:21:51.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c2a9d5740 10 client.0 ll_register_callbacks cb 0x253db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:51.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34560/0 (socket says 172.21.15.185:34560) 2020-11-16T22:21:51.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 -- 172.21.15.185:0/4011636145 learned_addr learned my addr 172.21.15.185:0/4011636145 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x26626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2616670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 -- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 msgr2=0x26626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x26626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 -- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 msgr2=0x2616670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2616670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.770+0000 7f3c194df700 1 -- 172.21.15.185:0/4011636145 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2538330 con 0x23fc860 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c194df700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f3c04011580 tx=0x7f3c04009040).ready entity=mon.1 client_cookie=4585f82026b36729 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:51.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003d20 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25fb770 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c04013540 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c04013910 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 10 client.7588 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x236e080 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25fb770 con 0x23fc860 2020-11-16T22:21:51.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3c0401c020 con 0x23fc860 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 10 client.7588 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 10 client.7588 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 20 client.7588 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x26490d0 0x264ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 4 client.7588 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2438d00 con 0x26490d0 2020-11-16T22:21:51.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x26490d0 0x264ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 10 client.7588 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x26490d0 0x264ea00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a1389a43914fd493 server_cookie=71fc7fbda85ca271 in_seq=0 out_seq=0 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 1 -- 172.21.15.185:0/4011636145 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2438d00 con 0x26490d0 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 10 client.7588 handle_command_reply: tid=0 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 client.7588 shutdown 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 msgr2=0x26621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f3c04011580 tx=0x7f3c04009040).stop 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 shutdown_connections 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x26490d0 0x264ea00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x261ae50 0x26626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x2616670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3be7fff700 0 client.7588 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/4011636145 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x26621e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.775 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 >> 172.21.15.185:0/4011636145 conn(0x25caef0 msgr2=0x264a440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 shutdown_connections 2020-11-16T22:21:51.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/4011636145 wait complete. 2020-11-16T22:21:51.776 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.774+0000 7f3c2a9d5740 20 client.7588 trim_cache size 0 max 16384 2020-11-16T22:21:51.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 Processor -- start 2020-11-16T22:21:51.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- start start 2020-11-16T22:21:51.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x261d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x251fef0 con 0x238ec60 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x265f210 con 0x23fc860 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x261aae0 con 0x25caef0 2020-11-16T22:21:51.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56242/0 (socket says 172.21.15.185:56242) 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3875568396 learned_addr learned my addr 172.21.15.185:0/3875568396 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c194df700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x261d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 msgr2=0x2617a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/3875568396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 msgr2=0x261d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x261d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 -- 172.21.15.185:0/3875568396 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0008160 con 0x23fc860 2020-11-16T22:21:51.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261da60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3c0c044a50 tx=0x7f3c0c028040).ready entity=mon.1 client_cookie=21badbbcde759fe2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3bf37fe700 1 -- 172.21.15.185:0/3875568396 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c02b040 con 0x23fc860 2020-11-16T22:21:51.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3bf37fe700 1 -- 172.21.15.185:0/3875568396 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0469c0 con 0x23fc860 2020-11-16T22:21:51.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3bf37fe700 1 -- 172.21.15.185:0/3875568396 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c047c50 con 0x23fc860 2020-11-16T22:21:51.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 msgr2=0x261da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261da60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3c0c044a50 tx=0x7f3c0c028040).stop 2020-11-16T22:21:51.797 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3875568396 shutdown_connections 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x23fc860 0x261da60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x2617a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3875568396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x238ec60 0x261d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3875568396 >> 172.21.15.185:0/3875568396 conn(0x7f3be0002320 msgr2=0x26621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3875568396 shutdown_connections 2020-11-16T22:21:51.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.798+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3875568396 wait complete. 2020-11-16T22:21:51.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- start start 2020-11-16T22:21:51.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 Processor -- start 2020-11-16T22:21:51.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 0x26b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x26ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b4d70 con 0x23fc860 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x26b4fa0 con 0x238ec60 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x26b1a70 con 0x25caef0 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 10 client.0 ll_register_callbacks cb 0x261d4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c1a4e1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x26ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34572/0 (socket says 172.21.15.185:34572) 2020-11-16T22:21:51.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3277160633 learned_addr learned my addr 172.21.15.185:0/3277160633 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c194df700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 0x26b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 msgr2=0x26ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x26ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 msgr2=0x26b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 0x26b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 -- 172.21.15.185:0/3277160633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x23ac7d0 con 0x238ec60 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c1a4e1700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x26ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c194df700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 0x26b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:51.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3be7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c19ce0700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f3c10006fd0 tx=0x7f3c1002e040).ready entity=mon.1 client_cookie=fa8655d3a8c41e65 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10031040 con 0x238ec60 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c10023070 con 0x238ec60 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c10005a40 con 0x238ec60 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c10005e90 con 0x238ec60 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 10 client.7594 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:51.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.802+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x23b7eb0 con 0x238ec60 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10005a40 con 0x238ec60 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3c1003f420 con 0x238ec60 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 10 client.7594 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 10 client.7594 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 20 client.7594 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x261ae50 0x26ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 4 client.7594 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:21:51.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x2440000 con 0x261ae50 2020-11-16T22:21:51.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c194df700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x261ae50 0x26ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.807 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:21:51.809+0000 7fd96c605700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v21 ap=1 f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1) n(v4 rc2020-11-16T22:21:28.420895+0000 6=0+6)/n(v3 rc2020-11-16T22:21:28.420895+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@5},l=7251 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x563afb0aa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:21:28.420895+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.807 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:21:51.813+0000 7fd972611700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1) n(v3 rc2020-11-16T22:21:28.420895+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7251=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x563afb0aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:21:28.420895+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.807 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:21:51.813+0000 7fe1a1be7700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1) n(v2 rc2020-11-16T22:21:28.420895+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@4},l=7251 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a326cb4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:21:28.420895+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.808 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:21:51.813+0000 7fe1a7bf3700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1) n(v1 rc2020-11-16T22:21:28.420895+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7251=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a326cb4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:21:28.420895+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 10 client.7594 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:51.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c194df700 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x261ae50 0x26ab7b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=861c6a524ea106cf server_cookie=ad3e3a766bea48a6 in_seq=0 out_seq=0 2020-11-16T22:21:51.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 1 -- 172.21.15.185:0/3277160633 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x2440000 con 0x261ae50 2020-11-16T22:21:51.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 10 client.7594 handle_command_reply: tid=0 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 client.7594 shutdown 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 msgr2=0x2662290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f3c10006fd0 tx=0x7f3c1002e040).stop 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 shutdown_connections 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x261ae50 0x26ab7b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x238ec60 0x2662290 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3be7fff700 0 client.7594 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x25caef0 0x26ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 --2- 172.21.15.185:0/3277160633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x23fc860 0x26b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 >> 172.21.15.185:0/3277160633 conn(0x7f3be0002320 msgr2=0x2617f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 shutdown_connections 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 1 -- 172.21.15.185:0/3277160633 wait complete. 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.806+0000 7f3c2a9d5740 20 client.7594 trim_cache size 0 max 16384 2020-11-16T22:21:51.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 msgr2=0x7f3bec057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 0x7f3bec057520 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f3c1000b1b0 tx=0x7f3c10015000).stop 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 msgr2=0x7f3c141530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3c0c00ef80 tx=0x7f3c0c0091d0).stop 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 shutdown_connections 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bec055060 0x7f3bec057520 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3c14148160 0x7f3c141530d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c140acee0 0x7f3c140afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 --2- 172.21.15.185:0/1351737534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3c140b7170 0x7f3c14152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 >> 172.21.15.185:0/1351737534 conn(0x7f3c14067860 msgr2=0x7f3c14143770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 shutdown_connections 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.810+0000 7f3c1ace2700 1 -- 172.21.15.185:0/1351737534 wait complete. 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:51.812 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0, 2020-11-16T22:21:51.813 INFO:teuthology.orchestra.run.smithi185.stdout: "scrub_tag": "785a2670-69d6-4efa-ada4-4d2d650f3105", 2020-11-16T22:21:51.813 INFO:teuthology.orchestra.run.smithi185.stdout: "mode": "asynchronous" 2020-11-16T22:21:51.813 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:51.813 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:21:51.810+0000 7efdaa137700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1) n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@4},l=7251 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556b546c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.813 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:21:51.810+0000 7efdaa137700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/AsLsXsFsx@2},l=7251 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556b5489e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:21:51.818 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:51.945 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 Processor -- start 2020-11-16T22:21:51.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 -- start start 2020-11-16T22:21:51.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbcd40ad110 0x7fbcd4147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd4061fe0 con 0x7fbcd40b3090 2020-11-16T22:21:51.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40b0fe0 con 0x7fbcd40b1d30 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd9fab700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40b2b60 con 0x7fbcd40ad110 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd3fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbcd40b1d30 0x7fbcd41477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56250/0 (socket says 172.21.15.185:56250) 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd3fff700 1 -- 172.21.15.185:0/3974462722 learned_addr learned my addr 172.21.15.185:0/3974462722 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/3974462722 >> v2:172.21.15.96:3300/0 conn(0x7fbcd40ad110 0x7fbcd4147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091654368 0 0) 0x7fbcd40b0fe0 con 0x7fbcd40b1d30 2020-11-16T22:21:51.949 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb8003240 con 0x7fbcd40b1d30 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 78925720 0 0) 0x7fbcb8003240 con 0x7fbcd40b1d30 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 >> v2:172.21.15.96:3300/0 conn(0x7fbcd40ad110 msgr2=0x7fbcd4147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 --2- 172.21.15.185:0/3974462722 >> v2:172.21.15.96:3300/0 conn(0x7fbcd40ad110 0x7fbcd4147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 >> v1:172.21.15.96:6789/0 conn(0x7fbcd40b3090 legacy=0x7fbcd414b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40b1280 con 0x7fbcd40b1d30 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7fbcbc001860 con 0x7fbcd40b1d30 2020-11-16T22:21:51.950 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 >> v1:172.21.15.185:6789/0 conn(0x7fbcd40b1d30 legacy=0x7fbcd41477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb8003730 0x7fbcb8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcb80044e0 0x7fbcb8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcb800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b2b60 con 0x7fbcb8003730 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80041d0 con 0x7fbcb8004e50 2020-11-16T22:21:51.951 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb800bc20 con 0x7fbcb80044e0 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcb800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd3fff700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcb80044e0 0x7fbcb8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd8fa9700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb8003730 0x7fbcb8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 -- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcb80044e0 msgr2=0x7fbcb8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcb80044e0 0x7fbcb8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.952 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 -- 172.21.15.185:0/3974462722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb8003730 msgr2=0x7fbcb8003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcb8003730 0x7fbcb8003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd97aa700 1 -- 172.21.15.185:0/3974462722 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8003240 con 0x7fbcb8004e50 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.954+0000 7fbcd3fff700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcb80044e0 0x7fbcb8004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd97aa700 1 -- 172.21.15.185:0/3974462722 reap_dead start 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcb800b730 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fbcc400bbf0 tx=0x7fbcc4006df0).ready entity=mon.1 client_cookie=30f993455e81d820 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.953 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4016070 con 0x7fbcb8004e50 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400b300 con 0x7fbcb8004e50 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd37fe700 1 -- 172.21.15.185:0/3974462722 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400ec60 con 0x7fbcb8004e50 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 msgr2=0x7fbcb800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcb800b730 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fbcc400bbf0 tx=0x7fbcc4006df0).stop 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- 172.21.15.185:0/3974462722 shutdown_connections 2020-11-16T22:21:51.956 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/3974462722 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcb800b730 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- 172.21.15.185:0/3974462722 >> 172.21.15.185:0/3974462722 conn(0x7fbcd4060130 msgr2=0x7fbcd4060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- 172.21.15.185:0/3974462722 shutdown_connections 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- 172.21.15.185:0/3974462722 wait complete. 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 Processor -- start 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- start start 2020-11-16T22:21:51.957 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcd4157650 0x7fbcd41448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b0320 con 0x7fbcd4157650 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b0580 con 0x7fbcb8004e50 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd9fab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b0820 con 0x7fbcd414af00 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd8fa9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd8fa9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34592/0 (socket says 172.21.15.185:34592) 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd8fa9700 1 -- 172.21.15.185:0/113613819 learned_addr learned my addr 172.21.15.185:0/113613819 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56270/0 (socket says 172.21.15.185:56270) 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd97aa700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcd4157650 0x7fbcd41448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 -- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 msgr2=0x7fbcd414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 -- 172.21.15.185:0/113613819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcd4157650 msgr2=0x7fbcd41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcd4157650 0x7fbcd41448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd3fff700 1 -- 172.21.15.185:0/113613819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd414bc40 con 0x7fbcd414af00 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.958+0000 7fbcd8fa9700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd3fff700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fbcd40650f0 tx=0x7fbcd40b1410).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00fc40 con 0x7fbcd414af00 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd414bee0 con 0x7fbcd414af00 2020-11-16T22:21:51.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc00fda0 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc012560 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd414b430 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd405d200 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbcbc0128d0 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 0x7fbca4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbcbc06d6f0 con 0x7fbcd414af00 2020-11-16T22:21:51.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd8fa9700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 0x7fbca4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:51.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.962+0000 7fbcd8fa9700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 0x7fbca4057520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fbccc007fd0 tx=0x7fbccc007b70).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:51.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:51.970+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbcbc043a00 con 0x7fbcd414af00 2020-11-16T22:21:52.160 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.166+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcd40b2ca0 con 0x7fbcd414af00 2020-11-16T22:21:52.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd0ff9700 1 -- 172.21.15.185:0/113613819 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+2524 (secure 0 0 0) 0x7fbcbc019020 con 0x7fbcd414af00 2020-11-16T22:21:52.161 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:52.161 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:49.075839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:52.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 msgr2=0x7fbca4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 0x7fbca4057520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fbccc007fd0 tx=0x7fbccc007b70).stop 2020-11-16T22:21:52.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 msgr2=0x7fbcd40ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fbcd40650f0 tx=0x7fbcd40b1410).stop 2020-11-16T22:21:52.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 shutdown_connections 2020-11-16T22:21:52.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbca4055060 0x7fbca4057520 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbcb8004e50 0x7fbcd414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcd4157650 0x7fbcd41448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 --2- 172.21.15.185:0/113613819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbcd414af00 0x7fbcd40ad690 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 >> 172.21.15.185:0/113613819 conn(0x7fbcd4060130 msgr2=0x7fbcd40b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 shutdown_connections 2020-11-16T22:21:52.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.170+0000 7fbcd9fab700 1 -- 172.21.15.185:0/113613819 wait complete. 2020-11-16T22:21:52.165 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 91 2020-11-16T22:21:52.175 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub pause 2020-11-16T22:21:52.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 Processor -- start 2020-11-16T22:21:52.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 -- start start 2020-11-16T22:21:52.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 0x7f82fc0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc14c660 con 0x7f82fc0b3920 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc0ac120 con 0x7f82fc0b3080 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0ac3c0 con 0x7f82fc0b13d0 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56280/0 (socket says 172.21.15.185:56280) 2020-11-16T22:21:52.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 -- 172.21.15.185:0/1696716035 learned_addr learned my addr 172.21.15.185:0/1696716035 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83015b4700 1 --2- 172.21.15.185:0/1696716035 >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 0x7f82fc0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 -- 172.21.15.185:0/1696716035 >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 msgr2=0x7f82fc0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 --2- 172.21.15.185:0/1696716035 >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 0x7f82fc0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 -- 172.21.15.185:0/1696716035 >> v1:172.21.15.96:6789/0 conn(0x7f82fc0b3920 legacy=0x7f82fc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 -- 172.21.15.185:0/1696716035 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0acd50 con 0x7f82fc0b13d0 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83015b4700 1 --2- 172.21.15.185:0/1696716035 >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 0x7f82fc0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:52.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f8300db3700 1 --2- 172.21.15.185:0/1696716035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f82f0009d50 tx=0x7f82f0009290).ready entity=mon.2 client_cookie=1a4b710c34a9f9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f82effff700 1 -- 172.21.15.185:0/1696716035 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000bc80 con 0x7f82fc0b13d0 2020-11-16T22:21:52.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f82effff700 1 -- 172.21.15.185:0/1696716035 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f000bde0 con 0x7f82fc0b13d0 2020-11-16T22:21:52.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f82effff700 1 -- 172.21.15.185:0/1696716035 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0011610 con 0x7f82fc0b13d0 2020-11-16T22:21:52.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 -- 172.21.15.185:0/1696716035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 msgr2=0x7f82fc0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.314+0000 7f83025b6700 1 --2- 172.21.15.185:0/1696716035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f82f0009d50 tx=0x7f82f0009290).stop 2020-11-16T22:21:52.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/1696716035 shutdown_connections 2020-11-16T22:21:52.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 --2- 172.21.15.185:0/1696716035 >> v2:172.21.15.96:3300/0 conn(0x7f82fc0b3080 0x7f82fc0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 --2- 172.21.15.185:0/1696716035 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0b1bf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/1696716035 >> 172.21.15.185:0/1696716035 conn(0x7f82fc060130 msgr2=0x7f82fc060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/1696716035 shutdown_connections 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/1696716035 wait complete. 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 Processor -- start 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- start start 2020-11-16T22:21:52.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82fc0b3920 0x7f82fc1123a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0aebd0 con 0x7f82fc0b3920 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0aff00 con 0x7f82fc0b3080 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0af280 con 0x7f82fc0b13d0 2020-11-16T22:21:52.314 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83015b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34608/0 (socket says 172.21.15.185:34608) 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 -- 172.21.15.185:0/3744354761 learned_addr learned my addr 172.21.15.185:0/3744354761 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8301db5700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82fc0b3920 0x7f82fc1123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 -- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 msgr2=0x7f82fc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 -- 172.21.15.185:0/3744354761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82fc0b3920 msgr2=0x7f82fc1123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82fc0b3920 0x7f82fc1123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 -- 172.21.15.185:0/3744354761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0af520 con 0x7f82fc0b3080 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f8300db3700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f82f000c010 tx=0x7f82f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f82ed7fa700 1 -- 172.21.15.185:0/3744354761 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f001c680 con 0x7f82fc0b3080 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f82ed7fa700 1 -- 172.21.15.185:0/3744354761 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f001c7e0 con 0x7f82fc0b3080 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0af7c0 con 0x7f82fc0b3080 2020-11-16T22:21:52.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f82ed7fa700 1 -- 172.21.15.185:0/3744354761 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f001cb80 con 0x7f82fc0b3080 2020-11-16T22:21:52.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.318+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc113250 con 0x7f82fc0b3080 2020-11-16T22:21:52.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.322+0000 7f82ed7fa700 1 -- 172.21.15.185:0/3744354761 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f82f0017070 con 0x7f82fc0b3080 2020-11-16T22:21:52.318 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.322+0000 7f82ed7fa700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 0x7f82d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.322+0000 7f82ed7fa700 1 -- 172.21.15.185:0/3744354761 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82f006fb30 con 0x7f82fc0b3080 2020-11-16T22:21:52.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.322+0000 7f83015b4700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 0x7f82d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.322+0000 7f83015b4700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 0x7f82d4057520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f82f400b710 tx=0x7f82f4008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 Processor -- start 2020-11-16T22:21:52.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- start start 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b30d90 0x2b2f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29bc4a0 con 0x29127a0 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a70920 con 0x2b30d90 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29306e0 con 0x28a4ba0 2020-11-16T22:21:52.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56290/0 (socket says 172.21.15.185:56290) 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 -- 172.21.15.185:0/107831534 learned_addr learned my addr 172.21.15.185:0/107831534 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8301db5700 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b30d90 0x2b2f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83015b4700 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 -- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b30d90 msgr2=0x2b2f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b30d90 0x2b2f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 -- 172.21.15.185:0/107831534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 msgr2=0x2b308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 -- 172.21.15.185:0/107831534 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x292b590 con 0x28a4ba0 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f8300db3700 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f82f00092d0 tx=0x7f82f0012000).ready entity=mon.2 client_cookie=b32392d8fc7042ab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f82de7fc700 1 -- 172.21.15.185:0/107831534 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0025cd0 con 0x28a4ba0 2020-11-16T22:21:52.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f82de7fc700 1 -- 172.21.15.185:0/107831534 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0025e30 con 0x28a4ba0 2020-11-16T22:21:52.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f82de7fc700 1 -- 172.21.15.185:0/107831534 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0049150 con 0x28a4ba0 2020-11-16T22:21:52.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 msgr2=0x2b2ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f82f00092d0 tx=0x7f82f0012000).stop 2020-11-16T22:21:52.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- 172.21.15.185:0/107831534 shutdown_connections 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2b30d90 0x2b2f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 --2- 172.21.15.185:0/107831534 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b2ee80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- 172.21.15.185:0/107831534 >> 172.21.15.185:0/107831534 conn(0x2ae0e30 msgr2=0x2b2ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- 172.21.15.185:0/107831534 shutdown_connections 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.326+0000 7f83122a9740 1 -- 172.21.15.185:0/107831534 wait complete. 2020-11-16T22:21:52.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- start start 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 Processor -- start 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b31e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b30d90 0x2b5eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a5d6d0 con 0x29127a0 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a5d810 con 0x28a4ba0 2020-11-16T22:21:52.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a5ea80 con 0x2b30d90 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 10 client.0 ll_register_callbacks cb 0x2a53ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34620/0 (socket says 172.21.15.185:34620) 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 -- 172.21.15.185:0/3081212683 learned_addr learned my addr 172.21.15.185:0/3081212683 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f8301db5700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b30d90 0x2b5eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f8300db3700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b31e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 -- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b30d90 msgr2=0x2b5eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b30d90 0x2b5eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 -- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 msgr2=0x2b31e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b31e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 -- 172.21.15.185:0/3081212683 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2a4e270 con 0x28a4ba0 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83015b4700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f82f4010af0 tx=0x7f82f4013000).ready entity=mon.1 client_cookie=ae90c4490a4cd85a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011710 con 0x28a4ba0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2b116b0 con 0x28a4ba0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82f4011870 con 0x28a4ba0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f4011c40 con 0x28a4ba0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 10 client.7609 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x2883fc0 con 0x28a4ba0 2020-11-16T22:21:52.331 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2b116b0 con 0x28a4ba0 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f82f401ea30 con 0x28a4ba0 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 10 client.7609 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 10 client.7609 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 20 client.7609 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b743e0 0x2b78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 4 client.7609 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:52.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x294ec40 con 0x2b743e0 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.330+0000 7f8300db3700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b743e0 0x2b78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f82cb7fe700 10 client.7609 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f8300db3700 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b743e0 0x2b78190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=43424ea1df68aac5 server_cookie=6982a498d9ff93b3 in_seq=0 out_seq=0 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f82cb7fe700 1 -- 172.21.15.185:0/3081212683 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x294ec40 con 0x2b743e0 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f82cb7fe700 10 client.7609 handle_command_reply: tid=0 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 client.7609 shutdown 2020-11-16T22:21:52.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 msgr2=0x2b31910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f82f4010af0 tx=0x7f82f4013000).stop 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 shutdown_connections 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b743e0 0x2b78190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b31910 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f82cb7fe700 0 client.7609 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2b30d90 0x2b5eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 --2- 172.21.15.185:0/3081212683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29127a0 0x2b31e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 >> 172.21.15.185:0/3081212683 conn(0x2ae0e30 msgr2=0x2b2ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 shutdown_connections 2020-11-16T22:21:52.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 1 -- 172.21.15.185:0/3081212683 wait complete. 2020-11-16T22:21:52.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.334+0000 7f83122a9740 20 client.7609 trim_cache size 0 max 16384 2020-11-16T22:21:52.348 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 Processor -- start 2020-11-16T22:21:52.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- start start 2020-11-16T22:21:52.349 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2b2ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 0x2b607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a35e30 con 0x2ae0e30 2020-11-16T22:21:52.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a8d670 con 0x28a4ba0 2020-11-16T22:21:52.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b66de0 con 0x29127a0 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8301db5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 0x2b607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34630/0 (socket says 172.21.15.185:34630) 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 -- 172.21.15.185:0/1622548341 learned_addr learned my addr 172.21.15.185:0/1622548341 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8301db5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 0x2b607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56308/0 (socket says 172.21.15.185:56308) 2020-11-16T22:21:52.351 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83015b4700 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2b2ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 -- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 msgr2=0x2b607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 0x2b607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 -- 172.21.15.185:0/1622548341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 msgr2=0x2b2ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2b2ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 -- 172.21.15.185:0/1622548341 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00081d0 con 0x28a4ba0 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f8300db3700 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f82f00497e0 tx=0x7f82f001e070).ready entity=mon.1 client_cookie=cb53c5fb248a0c9b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f82deffd700 1 -- 172.21.15.185:0/1622548341 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0025cd0 con 0x28a4ba0 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f82deffd700 1 -- 172.21.15.185:0/1622548341 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f00493b0 con 0x28a4ba0 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f82deffd700 1 -- 172.21.15.185:0/1622548341 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0049150 con 0x28a4ba0 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 msgr2=0x2b60300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f82f00497e0 tx=0x7f82f001e070).stop 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- 172.21.15.185:0/1622548341 shutdown_connections 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29127a0 0x2b607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2b2ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 --2- 172.21.15.185:0/1622548341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a4ba0 0x2b60300 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- 172.21.15.185:0/1622548341 >> 172.21.15.185:0/1622548341 conn(0x7f82d0002320 msgr2=0x2b31910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- 172.21.15.185:0/1622548341 shutdown_connections 2020-11-16T22:21:52.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.358+0000 7f83122a9740 1 -- 172.21.15.185:0/1622548341 wait complete. 2020-11-16T22:21:52.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 -- start start 2020-11-16T22:21:52.356 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 Processor -- start 2020-11-16T22:21:52.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 0x2bca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2bca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b33ef0 con 0x2ae0e30 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b69ac0 con 0x29127a0 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b64e10 con 0x28a4ba0 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83122a9740 10 client.0 ll_register_callbacks cb 0x2b2e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83015b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83015b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56310/0 (socket says 172.21.15.185:56310) 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83015b4700 1 -- 172.21.15.185:0/1015539963 learned_addr learned my addr 172.21.15.185:0/1015539963 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 0x2bca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8301db5700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2bca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 -- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 msgr2=0x2b668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 -- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 msgr2=0x2bca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2bca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 -- 172.21.15.185:0/1015539963 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x28c2710 con 0x29127a0 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f83015b4700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f8300db3700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 0x2bca1e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f82f0049980 tx=0x7f82f003b000).ready entity=mon.1 client_cookie=d71c90c1cc588b38 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f82cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.362+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f006f790 con 0x29127a0 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82f006f8f0 con 0x29127a0 2020-11-16T22:21:52.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f0025e30 con 0x29127a0 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f00255d0 con 0x29127a0 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 10 client.7615 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x28cddf0 con 0x29127a0 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0025e30 con 0x29127a0 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f82f0025730 con 0x29127a0 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 10 client.7615 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:52.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 10 client.7615 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 20 client.7615 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b30d90 0x2b747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 4 client.7615 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0x2955f40 con 0x2b30d90 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83015b4700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b30d90 0x2b747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83015b4700 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b30d90 0x2b747c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e800951fdc7df9b7 server_cookie=4137e4847418e3a6 in_seq=0 out_seq=0 2020-11-16T22:21:52.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 10 client.7615 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:52.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 1 -- 172.21.15.185:0/1015539963 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x2955f40 con 0x2b30d90 2020-11-16T22:21:52.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 10 client.7615 handle_command_reply: tid=0 2020-11-16T22:21:52.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 client.7615 shutdown 2020-11-16T22:21:52.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 msgr2=0x2bca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 0x2bca1e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f82f0049980 tx=0x7f82f003b000).stop 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 shutdown_connections 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x2b30d90 0x2b747c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2ae0e30 0x2bca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f82cb7fe700 0 client.7615 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:52.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a4ba0 0x2b668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 --2- 172.21.15.185:0/1015539963 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29127a0 0x2bca1e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 >> 172.21.15.185:0/1015539963 conn(0x7f82d0002320 msgr2=0x2b2ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 shutdown_connections 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 1 -- 172.21.15.185:0/1015539963 wait complete. 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83122a9740 20 client.7615 trim_cache size 0 max 16384 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 msgr2=0x7f82d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 0x7f82d4057520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f82f400b710 tx=0x7f82f4008040).stop 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 msgr2=0x7f82fc10f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f82f000c010 tx=0x7f82f0016000).stop 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 shutdown_connections 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f82d4055060 0x7f82d4057520 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82fc0b3920 0x7f82fc1123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f82fc0b13d0 0x7f82fc0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 --2- 172.21.15.185:0/3744354761 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f82fc0b3080 0x7f82fc10f7e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.366+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 >> 172.21.15.185:0/3744354761 conn(0x7f82fc060130 msgr2=0x7f82fc1467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.370+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 shutdown_connections 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.370+0000 7f83025b6700 1 -- 172.21.15.185:0/3744354761 wait complete. 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:21:52.369 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:52.378 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:52.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.510+0000 7f9f674d1700 1 Processor -- start 2020-11-16T22:21:52.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.510+0000 7f9f674d1700 1 -- start start 2020-11-16T22:21:52.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9f60067070 0x7f9f60143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f60067510 con 0x7f9f600682b0 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600677b0 con 0x7f9f60067070 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600b17a0 con 0x7f9f600b7470 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56318/0 (socket says 172.21.15.185:56318) 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f65cce700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f9f60067070 0x7f9f60143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.507 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 -- 172.21.15.185:0/2262272991 learned_addr learned my addr 172.21.15.185:0/2262272991 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 -- 172.21.15.185:0/2262272991 >> v2:172.21.15.185:3300/0 conn(0x7f9f60067070 msgr2=0x7f9f60143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 --2- 172.21.15.185:0/2262272991 >> v2:172.21.15.185:3300/0 conn(0x7f9f60067070 0x7f9f60143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 -- 172.21.15.185:0/2262272991 >> v1:172.21.15.96:6789/0 conn(0x7f9f600682b0 legacy=0x7f9f600b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 -- 172.21.15.185:0/2262272991 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600b1a40 con 0x7f9f600b7470 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f664cf700 1 --2- 172.21.15.185:0/2262272991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f9f5c006fd0 tx=0x7f9f5c006eb0).ready entity=mon.2 client_cookie=4da6726f1ead3755 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f654cd700 1 -- 172.21.15.185:0/2262272991 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c009810 con 0x7f9f600b7470 2020-11-16T22:21:52.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f654cd700 1 -- 172.21.15.185:0/2262272991 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c009970 con 0x7f9f600b7470 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f654cd700 1 -- 172.21.15.185:0/2262272991 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c009cf0 con 0x7f9f600b7470 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- 172.21.15.185:0/2262272991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 msgr2=0x7f9f600b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 --2- 172.21.15.185:0/2262272991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f9f5c006fd0 tx=0x7f9f5c006eb0).stop 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- 172.21.15.185:0/2262272991 shutdown_connections 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 --2- 172.21.15.185:0/2262272991 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f600b7470 0x7f9f600b7850 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 --2- 172.21.15.185:0/2262272991 >> v2:172.21.15.185:3300/0 conn(0x7f9f60067070 0x7f9f60143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.514+0000 7f9f674d1700 1 -- 172.21.15.185:0/2262272991 >> 172.21.15.185:0/2262272991 conn(0x7f9f600637b0 msgr2=0x7f9f600b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- 172.21.15.185:0/2262272991 shutdown_connections 2020-11-16T22:21:52.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- 172.21.15.185:0/2262272991 wait complete. 2020-11-16T22:21:52.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 Processor -- start 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- start start 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f60067070 0x7f9f6014b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f600682b0 0x7f9f6013cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600b0b20 con 0x7f9f60067070 2020-11-16T22:21:52.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6013d860 con 0x7f9f600682b0 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600b06c0 con 0x7f9f6013d0c0 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56326/0 (socket says 172.21.15.185:56326) 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 -- 172.21.15.185:0/3750506889 learned_addr learned my addr 172.21.15.185:0/3750506889 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f65cce700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f600682b0 0x7f9f6013cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f664cf700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f60067070 0x7f9f6014b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 -- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f600682b0 msgr2=0x7f9f6013cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f600682b0 0x7f9f6013cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 -- 172.21.15.185:0/3750506889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f60067070 msgr2=0x7f9f6014b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f60067070 0x7f9f6014b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 -- 172.21.15.185:0/3750506889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600af480 con 0x7f9f6013d0c0 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f66cd0700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9f5800cac0 tx=0x7f9f58007eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800cea0 con 0x7f9f6013d0c0 2020-11-16T22:21:52.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5800a6d0 con 0x7f9f6013d0c0 2020-11-16T22:21:52.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800aa70 con 0x7f9f6013d0c0 2020-11-16T22:21:52.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.518+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600afa60 con 0x7f9f6013d0c0 2020-11-16T22:21:52.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600af1e0 con 0x7f9f6013d0c0 2020-11-16T22:21:52.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6005d200 con 0x7f9f6013d0c0 2020-11-16T22:21:52.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f9f58010070 con 0x7f9f6013d0c0 2020-11-16T22:21:52.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f4e7fc700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 0x7f9f38057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f5806d960 con 0x7f9f6013d0c0 2020-11-16T22:21:52.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f664cf700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 0x7f9f38057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.522+0000 7f9f664cf700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 0x7f9f38057520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9f600b7d50 tx=0x7f9f5c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.530+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f9f58043b60 con 0x7f9f6013d0c0 2020-11-16T22:21:52.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.734+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f600b0850 con 0x7f9f6013d0c0 2020-11-16T22:21:52.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.734+0000 7f9f4e7fc700 1 -- 172.21.15.185:0/3750506889 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f9f58047180 con 0x7f9f6013d0c0 2020-11-16T22:21:52.727 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:52.727 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:49.075839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:52.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 msgr2=0x7f9f38057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 0x7f9f38057520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9f600b7d50 tx=0x7f9f5c015000).stop 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 msgr2=0x7f9f60139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f9f5800cac0 tx=0x7f9f58007eb0).stop 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 shutdown_connections 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9f38055060 0x7f9f38057520 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.729 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f9f6013d0c0 0x7f9f60139160 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f60067070 0x7f9f6014b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 --2- 172.21.15.185:0/3750506889 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f9f600682b0 0x7f9f6013cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 >> 172.21.15.185:0/3750506889 conn(0x7f9f600637b0 msgr2=0x7f9f60140e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 shutdown_connections 2020-11-16T22:21:52.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.738+0000 7f9f674d1700 1 -- 172.21.15.185:0/3750506889 wait complete. 2020-11-16T22:21:52.731 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 91 2020-11-16T22:21:52.741 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:21:52.869 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 Processor -- start 2020-11-16T22:21:52.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 -- start start 2020-11-16T22:21:52.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3230144d50 con 0x7f323013dea0 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32300ad490 con 0x7f32300b7440 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f32369fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ad730 con 0x7f32300afc60 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56334/0 (socket says 172.21.15.185:56334) 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34656/0 (socket says 172.21.15.185:34656) 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 -- 172.21.15.185:0/1779932509 learned_addr learned my addr 172.21.15.185:0/1779932509 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 -- 172.21.15.185:0/1779932509 >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 msgr2=0x7f32300b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1779932509 >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 -- 172.21.15.185:0/1779932509 >> v1:172.21.15.96:6789/0 conn(0x7f323013dea0 legacy=0x7f32300ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 -- 172.21.15.185:0/1779932509 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300ad9d0 con 0x7f32300afc60 2020-11-16T22:21:52.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322ffff700 1 --2- 172.21.15.185:0/1779932509 >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:52.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1779932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f3218006fd0 tx=0x7f3218006eb0).ready entity=mon.2 client_cookie=af6499a32db6169 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322effd700 1 -- 172.21.15.185:0/1779932509 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218009810 con 0x7f32300afc60 2020-11-16T22:21:52.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322effd700 1 -- 172.21.15.185:0/1779932509 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218009970 con 0x7f32300afc60 2020-11-16T22:21:52.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.878+0000 7f322effd700 1 -- 172.21.15.185:0/1779932509 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218009cf0 con 0x7f32300afc60 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- 172.21.15.185:0/1779932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 msgr2=0x7f3230144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- 172.21.15.185:0/1779932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f3218006fd0 tx=0x7f3218006eb0).stop 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- 172.21.15.185:0/1779932509 shutdown_connections 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- 172.21.15.185:0/1779932509 >> v2:172.21.15.185:3300/0 conn(0x7f32300b7440 0x7f32300b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- 172.21.15.185:0/1779932509 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f32300afc60 0x7f3230144860 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- 172.21.15.185:0/1779932509 >> 172.21.15.185:0/1779932509 conn(0x7f3230067860 msgr2=0x7f3230067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- 172.21.15.185:0/1779932509 shutdown_connections 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- 172.21.15.185:0/1779932509 wait complete. 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 Processor -- start 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- start start 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32300afc60 0x7f3230153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f323013dea0 0x7f3230140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300b4d20 con 0x7f32300afc60 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230153800 con 0x7f32300b7440 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f32369fe700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3230157800 con 0x7f323013dea0 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34664/0 (socket says 172.21.15.185:34664) 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f3234f9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f323013dea0 0x7f3230140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 -- 172.21.15.185:0/365785766 learned_addr learned my addr 172.21.15.185:0/365785766 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322ffff700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32300afc60 0x7f3230153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 -- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f323013dea0 msgr2=0x7f3230140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f323013dea0 0x7f3230140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 -- 172.21.15.185:0/365785766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32300afc60 msgr2=0x7f3230153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32300afc60 0x7f3230153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 -- 172.21.15.185:0/365785766 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230113ca0 con 0x7f32300b7440 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.882+0000 7f322f7fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f3218006ef0 tx=0x7f3218015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 -- 172.21.15.185:0/365785766 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011d50 con 0x7f32300b7440 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 -- 172.21.15.185:0/365785766 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321801e460 con 0x7f32300b7440 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 -- 172.21.15.185:0/365785766 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321801e7e0 con 0x7f32300b7440 2020-11-16T22:21:52.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3230113f40 con 0x7f32300b7440 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230154240 con 0x7f32300b7440 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 -- 172.21.15.185:0/365785766 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3218025030 con 0x7f32300b7440 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 0x7f320c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f322ffff700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 0x7f320c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f3227fff700 1 -- 172.21.15.185:0/365785766 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f321806f0a0 con 0x7f32300b7440 2020-11-16T22:21:52.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f322ffff700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 0x7f320c057520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3220006fd0 tx=0x7f3220006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f324548f740 1 Processor -- start 2020-11-16T22:21:52.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.886+0000 7f324548f740 1 -- start start 2020-11-16T22:21:52.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x170bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 0x170c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15634b0 con 0x17117f0 2020-11-16T22:21:52.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1617930 con 0x144bbb0 2020-11-16T22:21:52.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x14d76f0 con 0x14b97b0 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x170bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56346/0 (socket says 172.21.15.185:56346) 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322ffff700 1 -- 172.21.15.185:0/509186288 learned_addr learned my addr 172.21.15.185:0/509186288 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f3234f9b700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 0x170c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 -- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 msgr2=0x1711300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 -- 172.21.15.185:0/509186288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 msgr2=0x170c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 0x170c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 -- 172.21.15.185:0/509186288 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14d25a0 con 0x144bbb0 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f3234f9b700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 0x170c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:52.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322ffff700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f322f7fe700 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x170bf10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f3218000dd0 tx=0x7f3218011000).ready entity=mon.1 client_cookie=62126dfc479d6baf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f32257fa700 1 -- 172.21.15.185:0/509186288 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218047410 con 0x144bbb0 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f32257fa700 1 -- 172.21.15.185:0/509186288 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218045780 con 0x144bbb0 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f32257fa700 1 -- 172.21.15.185:0/509186288 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218048770 con 0x144bbb0 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 msgr2=0x170bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x170bf10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f3218000dd0 tx=0x7f3218011000).stop 2020-11-16T22:21:52.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- 172.21.15.185:0/509186288 shutdown_connections 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17117f0 0x170c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x1711300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 --2- 172.21.15.185:0/509186288 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x144bbb0 0x170bf10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- 172.21.15.185:0/509186288 >> 172.21.15.185:0/509186288 conn(0x1687e40 msgr2=0x1710bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- 172.21.15.185:0/509186288 shutdown_connections 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- 172.21.15.185:0/509186288 wait complete. 2020-11-16T22:21:52.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 -- start start 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.890+0000 7f324548f740 1 Processor -- start 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x171bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16046e0 con 0x144bbb0 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1604820 con 0x16f7160 2020-11-16T22:21:52.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1605a90 con 0x14b97b0 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 10 client.0 ll_register_callbacks cb 0x15faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34680/0 (socket says 172.21.15.185:34680) 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f322ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 -- 172.21.15.185:0/1386157228 learned_addr learned my addr 172.21.15.185:0/1386157228 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x171bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 -- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 msgr2=0x171bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x171bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 -- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x171b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 -- 172.21.15.185:0/1386157228 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x15f5280 con 0x16f7160 2020-11-16T22:21:52.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f3234f9b700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f3228010810 tx=0x7f322800c040).ready entity=mon.1 client_cookie=59d99b0ed340e219 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280125a0 con 0x16f7160 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16cce10 con 0x16f7160 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3228012700 con 0x16f7160 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3228012ad0 con 0x16f7160 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 10 client.7633 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:52.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x142afd0 con 0x16f7160 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16cce10 con 0x16f7160 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3228012c30 con 0x16f7160 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 10 client.7633 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 10 client.7633 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 20 client.7633 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x171f360 0x1727f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 4 client.7633 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x14f5c50 con 0x171f360 2020-11-16T22:21:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f322ffff700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x171f360 0x1727f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f322ffff700 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x171f360 0x1727f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=e6b44d7bf3c898a0 server_cookie=1972357fcc6452c1 in_seq=0 out_seq=0 2020-11-16T22:21:52.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 10 client.7633 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:52.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 1 -- 172.21.15.185:0/1386157228 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x14f5c50 con 0x171f360 2020-11-16T22:21:52.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f32067fc700 10 client.7633 handle_command_reply: tid=0 2020-11-16T22:21:52.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.894+0000 7f324548f740 1 client.7633 shutdown 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 msgr2=0x1706800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f3228010810 tx=0x7f322800c040).stop 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 shutdown_connections 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x171f360 0x1727f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x16f7160 0x1706800 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f32067fc700 0 client.7633 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 --2- 172.21.15.185:0/1386157228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x14b97b0 0x171bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 >> 172.21.15.185:0/1386157228 conn(0x1687e40 msgr2=0x170df30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 shutdown_connections 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 1 -- 172.21.15.185:0/1386157228 wait complete. 2020-11-16T22:21:52.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.898+0000 7f324548f740 20 client.7633 trim_cache size 0 max 16384 2020-11-16T22:21:52.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 Processor -- start 2020-11-16T22:21:52.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- start start 2020-11-16T22:21:52.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x1708a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x15dce40 con 0x144bbb0 2020-11-16T22:21:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x17071e0 con 0x14b97b0 2020-11-16T22:21:52.911 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x16d5a10 con 0x1687e40 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56360/0 (socket says 172.21.15.185:56360) 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 -- 172.21.15.185:0/258361207 learned_addr learned my addr 172.21.15.185:0/258361207 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f3234f9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x1708a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322f7fe700 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 -- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 msgr2=0x1708a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x1708a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 -- 172.21.15.185:0/258361207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x171ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 -- 172.21.15.185:0/258361207 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0081d0 con 0x1687e40 2020-11-16T22:21:52.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f322ffff700 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3220024880 tx=0x7f3220009000).ready entity=mon.2 client_cookie=34699fe6335be531 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f3225ffb700 1 -- 172.21.15.185:0/258361207 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220024cd0 con 0x1687e40 2020-11-16T22:21:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f3225ffb700 1 -- 172.21.15.185:0/258361207 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220024e30 con 0x1687e40 2020-11-16T22:21:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f3225ffb700 1 -- 172.21.15.185:0/258361207 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220013670 con 0x1687e40 2020-11-16T22:21:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 msgr2=0x16d6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3220024880 tx=0x7f3220009000).stop 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- 172.21.15.185:0/258361207 shutdown_connections 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x1708a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x16d6e90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 --2- 172.21.15.185:0/258361207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x171ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- 172.21.15.185:0/258361207 >> 172.21.15.185:0/258361207 conn(0x7f31fc002320 msgr2=0x16f5820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- 172.21.15.185:0/258361207 shutdown_connections 2020-11-16T22:21:52.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- 172.21.15.185:0/258361207 wait complete. 2020-11-16T22:21:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 -- start start 2020-11-16T22:21:52.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.918+0000 7f324548f740 1 Processor -- start 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x1707f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d13b0 con 0x144bbb0 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1707cd0 con 0x14b97b0 2020-11-16T22:21:52.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x171b5e0 con 0x1687e40 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34692/0 (socket says 172.21.15.185:34692) 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 -- 172.21.15.185:0/1174076620 learned_addr learned my addr 172.21.15.185:0/1174076620 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 10 client.0 ll_register_callbacks cb 0x16d1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f3234f9b700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x1707f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322ffff700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 -- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 msgr2=0x1707f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x1707f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 -- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 msgr2=0x16d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 -- 172.21.15.185:0/1174076620 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1469720 con 0x14b97b0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322f7fe700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f3218048fa0 tx=0x7f3218011f30).ready entity=mon.1 client_cookie=b736b8307614d8a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218037070 con 0x14b97b0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f32180458f0 con 0x14b97b0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3218045780 con 0x14b97b0 2020-11-16T22:21:52.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3218045070 con 0x14b97b0 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 10 client.7639 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1474e00 con 0x14b97b0 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218045780 con 0x14b97b0 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3218047fd0 con 0x14b97b0 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 10 client.7639 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:52.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 10 client.7639 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 20 client.7639 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x16f7160 0x1719f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 4 client.7639 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x14fcf50 con 0x16f7160 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322ffff700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x16f7160 0x1719f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:52.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f322ffff700 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x16f7160 0x1719f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=eb0f02eba0d91dd2 server_cookie=4fd75baf482ca7ba in_seq=0 out_seq=0 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.922+0000 7f32067fc700 10 client.7639 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32067fc700 1 -- 172.21.15.185:0/1174076620 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x14fcf50 con 0x16f7160 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32067fc700 10 client.7639 handle_command_reply: tid=0 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 client.7639 shutdown 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 msgr2=0x16d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f3218048fa0 tx=0x7f3218011f30).stop 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 shutdown_connections 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0x16f7160 0x1719f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1687e40 0x1707f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32067fc700 0 client.7639 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x144bbb0 0x16d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 --2- 172.21.15.185:0/1174076620 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x14b97b0 0x16d0bf0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 >> 172.21.15.185:0/1174076620 conn(0x7f31fc002320 msgr2=0x16d6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 shutdown_connections 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 1 -- 172.21.15.185:0/1174076620 wait complete. 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f324548f740 20 client.7639 trim_cache size 0 max 16384 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 msgr2=0x7f320c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 0x7f320c057520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3220006fd0 tx=0x7f3220006df0).stop 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 msgr2=0x7f3230157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f3218006ef0 tx=0x7f3218015000).stop 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 shutdown_connections 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f320c055060 0x7f320c057520 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f323013dea0 0x7f3230140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32300afc60 0x7f3230153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 --2- 172.21.15.185:0/365785766 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f32300b7440 0x7f3230157310 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:52.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 >> 172.21.15.185:0/365785766 conn(0x7f3230067860 msgr2=0x7f3230146ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:52.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 shutdown_connections 2020-11-16T22:21:52.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:52.926+0000 7f32369fe700 1 -- 172.21.15.185:0/365785766 wait complete. 2020-11-16T22:21:52.930 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:52.930 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: "785a2670-69d6-4efa-ada4-4d2d650f3105": { 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "785a2670-69d6-4efa-ada4-4d2d650f3105", 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:21:52.931 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:52.932 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:52.932 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:52.933 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:53.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 Processor -- start 2020-11-16T22:21:53.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- start start 2020-11-16T22:21:53.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 0x7f4f680b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f680b7e10 con 0x7f4f680b4b20 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680b4fd0 con 0x7f4f681418d0 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680b7670 con 0x7f4f680675b0 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56374/0 (socket says 172.21.15.185:56374) 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 -- 172.21.15.185:0/2840506761 learned_addr learned my addr 172.21.15.185:0/2840506761 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2840506761 >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 0x7f4f680b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 -- 172.21.15.185:0/2840506761 >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 msgr2=0x7f4f680b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 --2- 172.21.15.185:0/2840506761 >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 0x7f4f680b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 -- 172.21.15.185:0/2840506761 >> v1:172.21.15.96:6789/0 conn(0x7f4f680b4b20 legacy=0x7f4f6813de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 -- 172.21.15.185:0/2840506761 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f680b78d0 con 0x7f4f680675b0 2020-11-16T22:21:53.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2840506761 >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 0x7f4f680b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6dbcf700 1 --2- 172.21.15.185:0/2840506761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f4f5c006fd0 tx=0x7f4f5c006eb0).ready entity=mon.2 client_cookie=4b8c1a7208094f0b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6cbcd700 1 -- 172.21.15.185:0/2840506761 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0098d0 con 0x7f4f680675b0 2020-11-16T22:21:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6cbcd700 1 -- 172.21.15.185:0/2840506761 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c009a30 con 0x7f4f680675b0 2020-11-16T22:21:53.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6cbcd700 1 -- 172.21.15.185:0/2840506761 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c009db0 con 0x7f4f680675b0 2020-11-16T22:21:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2840506761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 msgr2=0x7f4f68068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2840506761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f4f5c006fd0 tx=0x7f4f5c006eb0).stop 2020-11-16T22:21:53.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2840506761 shutdown_connections 2020-11-16T22:21:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2840506761 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680675b0 0x7f4f68068460 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2840506761 >> v2:172.21.15.185:3300/0 conn(0x7f4f681418d0 0x7f4f680b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2840506761 >> 172.21.15.185:0/2840506761 conn(0x7f4f6805ba80 msgr2=0x7f4f6805be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2840506761 shutdown_connections 2020-11-16T22:21:53.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.058+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2840506761 wait complete. 2020-11-16T22:21:53.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 Processor -- start 2020-11-16T22:21:53.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 -- start start 2020-11-16T22:21:53.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 0x7f4f68142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 0x7f4f6813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6813e290 con 0x7f4f681418d0 2020-11-16T22:21:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6813dd50 con 0x7f4f680675b0 2020-11-16T22:21:53.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6ebd1700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6813dff0 con 0x7f4f680b4b20 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6d3ce700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6dbcf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 0x7f4f68142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6dbcf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 0x7f4f68142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34702/0 (socket says 172.21.15.185:34702) 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6dbcf700 1 -- 172.21.15.185:0/2813755708 learned_addr learned my addr 172.21.15.185:0/2813755708 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.062+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56380/0 (socket says 172.21.15.185:56380) 2020-11-16T22:21:53.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6e3d0700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 0x7f4f6813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 -- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 msgr2=0x7f4f68142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 0x7f4f68142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 -- 172.21.15.185:0/2813755708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 msgr2=0x7f4f6813d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 0x7f4f6813d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 -- 172.21.15.185:0/2813755708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68143340 con 0x7f4f680b4b20 2020-11-16T22:21:53.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6e3d0700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 0x7f4f6813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6d3ce700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4f68065f60 tx=0x7f4f60008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000c740 con 0x7f4f680b4b20 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f680b0f50 con 0x7f4f680b4b20 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6000c8a0 con 0x7f4f680b4b20 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000cc20 con 0x7f4f680b4b20 2020-11-16T22:21:53.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68144ef0 con 0x7f4f680b4b20 2020-11-16T22:21:53.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6805d200 con 0x7f4f680b4b20 2020-11-16T22:21:53.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4f60013640 con 0x7f4f680b4b20 2020-11-16T22:21:53.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 0x7f4f48057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f6dbcf700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 0x7f4f48057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.066+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f6006ddd0 con 0x7f4f680b4b20 2020-11-16T22:21:53.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.070+0000 7f4f6dbcf700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 0x7f4f48057520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4f680662e0 tx=0x7f4f5c015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.074+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4f60043fd0 con 0x7f4f680b4b20 2020-11-16T22:21:53.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.278+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f680ae440 con 0x7f4f680b4b20 2020-11-16T22:21:53.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.278+0000 7f4f59ffb700 1 -- 172.21.15.185:0/2813755708 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f4f600475f0 con 0x7f4f680b4b20 2020-11-16T22:21:53.272 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:53.272 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:49.075839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:53.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 msgr2=0x7f4f48057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 0x7f4f48057520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4f680662e0 tx=0x7f4f5c015000).stop 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 msgr2=0x7f4f680b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f4f68065f60 tx=0x7f4f60008040).stop 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 shutdown_connections 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4f48055060 0x7f4f48057520 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4f680675b0 0x7f4f68142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f681418d0 0x7f4f6813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 --2- 172.21.15.185:0/2813755708 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4f680b4b20 0x7f4f680b0a60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 >> 172.21.15.185:0/2813755708 conn(0x7f4f6805ba80 msgr2=0x7f4f680b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 shutdown_connections 2020-11-16T22:21:53.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.282+0000 7f4f6ebd1700 1 -- 172.21.15.185:0/2813755708 wait complete. 2020-11-16T22:21:53.276 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 91 2020-11-16T22:21:53.287 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:21:53.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.410+0000 7f54f3d1d700 1 Processor -- start 2020-11-16T22:21:53.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.410+0000 7f54f3d1d700 1 -- start start 2020-11-16T22:21:53.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f54ec063550 0x7f54ec063930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54ec0b2070 0x7f54ec144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec0b26a0 con 0x7f54ec0b3700 2020-11-16T22:21:53.405 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec1452b0 con 0x7f54ec063550 2020-11-16T22:21:53.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0b2940 con 0x7f54ec0b2070 2020-11-16T22:21:53.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54ec0b2070 0x7f54ec144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f54ec0b2070 0x7f54ec144dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34712/0 (socket says 172.21.15.185:34712) 2020-11-16T22:21:53.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 -- 172.21.15.185:0/3084264280 learned_addr learned my addr 172.21.15.185:0/3084264280 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/3084264280 >> v2:172.21.15.96:3300/0 conn(0x7f54ec063550 0x7f54ec063930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 -- 172.21.15.185:0/3084264280 >> v2:172.21.15.96:3300/0 conn(0x7f54ec063550 msgr2=0x7f54ec063930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 --2- 172.21.15.185:0/3084264280 >> v2:172.21.15.96:3300/0 conn(0x7f54ec063550 0x7f54ec063930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 -- 172.21.15.185:0/3084264280 >> v1:172.21.15.96:6789/0 conn(0x7f54ec0b3700 legacy=0x7f54ec1412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 -- 172.21.15.185:0/3084264280 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0b2be0 con 0x7f54ec0b2070 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f12b8700 1 --2- 172.21.15.185:0/3084264280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec0b2070 0x7f54ec144dc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f54ec0617c0 tx=0x7f54e0006eb0).ready entity=mon.1 client_cookie=ebe4e13ecf6fdbab server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f0ab7700 1 -- 172.21.15.185:0/3084264280 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0009810 con 0x7f54ec0b2070 2020-11-16T22:21:53.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f0ab7700 1 -- 172.21.15.185:0/3084264280 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e0009970 con 0x7f54ec0b2070 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f0ab7700 1 -- 172.21.15.185:0/3084264280 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0009cf0 con 0x7f54ec0b2070 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- 172.21.15.185:0/3084264280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec0b2070 msgr2=0x7f54ec144dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/3084264280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec0b2070 0x7f54ec144dc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f54ec0617c0 tx=0x7f54e0006eb0).stop 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- 172.21.15.185:0/3084264280 shutdown_connections 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/3084264280 >> v2:172.21.15.96:3300/0 conn(0x7f54ec063550 0x7f54ec063930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/3084264280 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec0b2070 0x7f54ec144dc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- 172.21.15.185:0/3084264280 >> 172.21.15.185:0/3084264280 conn(0x7f54ec0649a0 msgr2=0x7f54ec062e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- 172.21.15.185:0/3084264280 shutdown_connections 2020-11-16T22:21:53.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.414+0000 7f54f3d1d700 1 -- 172.21.15.185:0/3084264280 wait complete. 2020-11-16T22:21:53.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 Processor -- start 2020-11-16T22:21:53.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- start start 2020-11-16T22:21:53.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 0x7f54ec156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54ec0b3700 0x7f54ec157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec13eae0 con 0x7f54ec0b3700 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec13fc90 con 0x7f54ec063550 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec157920 con 0x7f54ec0b2070 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f12b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 0x7f54ec156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34716/0 (socket says 172.21.15.185:34716) 2020-11-16T22:21:53.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 -- 172.21.15.185:0/1529810732 learned_addr learned my addr 172.21.15.185:0/1529810732 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f22ba700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54ec0b3700 0x7f54ec157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 -- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 msgr2=0x7f54ec156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 0x7f54ec156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 -- 172.21.15.185:0/1529810732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54ec0b3700 msgr2=0x7f54ec157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54ec0b3700 0x7f54ec157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 -- 172.21.15.185:0/1529810732 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec1540a0 con 0x7f54ec063550 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f12b8700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 0x7f54ec156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f54dc00ef50 tx=0x7f54dc0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54e9ffb700 1 -- 172.21.15.185:0/1529810732 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00cc10 con 0x7f54ec063550 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54e9ffb700 1 -- 172.21.15.185:0/1529810732 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00cd70 con 0x7f54ec063550 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec154340 con 0x7f54ec063550 2020-11-16T22:21:53.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54e9ffb700 1 -- 172.21.15.185:0/1529810732 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc012600 con 0x7f54ec063550 2020-11-16T22:21:53.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.418+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec110070 con 0x7f54ec063550 2020-11-16T22:21:53.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.422+0000 7f54e9ffb700 1 -- 172.21.15.185:0/1529810732 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f54dc022070 con 0x7f54ec063550 2020-11-16T22:21:53.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.422+0000 7f54e9ffb700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 0x7f54c4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.422+0000 7f54e9ffb700 1 -- 172.21.15.185:0/1529810732 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54dc06e500 con 0x7f54ec063550 2020-11-16T22:21:53.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.422+0000 7f54f12b8700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 0x7f54c4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.422+0000 7f54f12b8700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 0x7f54c4057550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f54e0006ef0 tx=0x7f54e0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 Processor -- start 2020-11-16T22:21:53.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 -- start start 2020-11-16T22:21:53.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1da1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1da77f0 0x1da2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bf94b0 con 0x1ae1bb0 2020-11-16T22:21:53.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cad930 con 0x1da77f0 2020-11-16T22:21:53.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b6d6f0 con 0x1b4f7b0 2020-11-16T22:21:53.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f22ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1da77f0 0x1da2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56400/0 (socket says 172.21.15.185:56400) 2020-11-16T22:21:53.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 -- 172.21.15.185:0/437562769 learned_addr learned my addr 172.21.15.185:0/437562769 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.421 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f12b8700 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1da1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 -- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1da77f0 msgr2=0x1da2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1da77f0 0x1da2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 -- 172.21.15.185:0/437562769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 msgr2=0x1da1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1da1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 -- 172.21.15.185:0/437562769 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b685a0 con 0x1b4f7b0 2020-11-16T22:21:53.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f12b8700 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1da1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:53.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f54dc047fe0 tx=0x7f54dc015000).ready entity=mon.2 client_cookie=5e697c29c508c995 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54caffd700 1 -- 172.21.15.185:0/437562769 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc044b30 con 0x1b4f7b0 2020-11-16T22:21:53.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54caffd700 1 -- 172.21.15.185:0/437562769 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc044c90 con 0x1b4f7b0 2020-11-16T22:21:53.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.426+0000 7f54caffd700 1 -- 172.21.15.185:0/437562769 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc047b20 con 0x1b4f7b0 2020-11-16T22:21:53.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 msgr2=0x1da7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f54dc047fe0 tx=0x7f54dc015000).stop 2020-11-16T22:21:53.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/437562769 shutdown_connections 2020-11-16T22:21:53.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1da77f0 0x1da2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1da7300 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- 172.21.15.185:0/437562769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1da1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/437562769 >> 172.21.15.185:0/437562769 conn(0x1d1de40 msgr2=0x1da6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/437562769 shutdown_connections 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/437562769 wait complete. 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- start start 2020-11-16T22:21:53.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 Processor -- start 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 0x1db57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c9a6e0 con 0x1da6f30 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c9a820 con 0x1b4f7b0 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1c9ba90 con 0x1ae1bb0 2020-11-16T22:21:53.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 10 client.0 ll_register_callbacks cb 0x1c90ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34730/0 (socket says 172.21.15.185:34730) 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 -- 172.21.15.185:0/2636952533 learned_addr learned my addr 172.21.15.185:0/2636952533 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56408/0 (socket says 172.21.15.185:56408) 2020-11-16T22:21:53.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f22ba700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 0x1db57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 -- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 msgr2=0x1d8b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 -- 172.21.15.185:0/2636952533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 msgr2=0x1db57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 0x1db57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 -- 172.21.15.185:0/2636952533 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1c8b280 con 0x1b4f7b0 2020-11-16T22:21:53.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f22ba700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 0x1db57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54f12b8700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f54e0009940 tx=0x7f54e001f000).ready entity=mon.1 client_cookie=4e0c63c166a23117 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54bffff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e002b430 con 0x1b4f7b0 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d62e10 con 0x1b4f7b0 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54e0023040 con 0x1b4f7b0 2020-11-16T22:21:53.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54e002b7a0 con 0x1b4f7b0 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 10 client.7657 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.430+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1ac0fd0 con 0x1b4f7b0 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d62e10 con 0x1b4f7b0 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f54e0028030 con 0x1b4f7b0 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 10 client.7657 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 10 client.7657 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:21:53.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 20 client.7657 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1db1b60 0x1d9ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 4 client.7657 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1b8bc50 con 0x1db1b60 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1db1b60 0x1d9ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 10 client.7657 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:53.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1db1b60 0x1d9ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=23da9bd9baf20172 server_cookie=f158552df722f2fe in_seq=0 out_seq=0 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 1 -- 172.21.15.185:0/2636952533 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1b8bc50 con 0x1db1b60 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 10 client.7657 handle_command_reply: tid=0 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 client.7657 shutdown 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 msgr2=0x1da0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f54e0009940 tx=0x7f54e001f000).stop 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 shutdown_connections 2020-11-16T22:21:53.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1db1b60 0x1d9ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1da6f30 0x1db57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ae1bb0 0x1d8b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f54bffff700 0 client.7657 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 --2- 172.21.15.185:0/2636952533 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1da0300 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 >> 172.21.15.185:0/2636952533 conn(0x1d1de40 msgr2=0x1da2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 shutdown_connections 2020-11-16T22:21:53.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 1 -- 172.21.15.185:0/2636952533 wait complete. 2020-11-16T22:21:53.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.434+0000 7f55027ae740 20 client.7657 trim_cache size 0 max 16384 2020-11-16T22:21:53.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 Processor -- start 2020-11-16T22:21:53.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 -- start start 2020-11-16T22:21:53.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1d67f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c72e40 con 0x1ae1bb0 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d8c7c0 con 0x1b4f7b0 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.450+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d8d150 con 0x1d1de40 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f22ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56414/0 (socket says 172.21.15.185:56414) 2020-11-16T22:21:53.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 -- 172.21.15.185:0/3353273714 learned_addr learned my addr 172.21.15.185:0/3353273714 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f22ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34742/0 (socket says 172.21.15.185:34742) 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f12b8700 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1d67f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 -- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 msgr2=0x1d68440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 -- 172.21.15.185:0/3353273714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 msgr2=0x1d67f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1d67f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 -- 172.21.15.185:0/3353273714 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b80081d0 con 0x1d1de40 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f22ba700 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f54dc048350 tx=0x7f54dc06e030).ready entity=mon.2 client_cookie=ec2cd9e2387965b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54cb7fe700 1 -- 172.21.15.185:0/3353273714 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc044b30 con 0x1d1de40 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54cb7fe700 1 -- 172.21.15.185:0/3353273714 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc047d80 con 0x1d1de40 2020-11-16T22:21:53.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f54cb7fe700 1 -- 172.21.15.185:0/3353273714 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc044210 con 0x1d1de40 2020-11-16T22:21:53.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 msgr2=0x1da3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f54dc048350 tx=0x7f54dc06e030).stop 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- 172.21.15.185:0/3353273714 shutdown_connections 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b4f7b0 0x1d68440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d1de40 0x1da3610 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 --2- 172.21.15.185:0/3353273714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1d67f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- 172.21.15.185:0/3353273714 >> 172.21.15.185:0/3353273714 conn(0x7f54b8002320 msgr2=0x1da2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- 172.21.15.185:0/3353273714 shutdown_connections 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- 172.21.15.185:0/3353273714 wait complete. 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 -- start start 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.454+0000 7f55027ae740 1 Processor -- start 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1e06470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1e06960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d6b020 con 0x1ae1bb0 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d6ad80 con 0x1d1de40 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da8fc0 con 0x1b4f7b0 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 10 client.0 ll_register_callbacks cb 0x1da42e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f22ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1e06960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f22ba700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34748/0 (socket says 172.21.15.185:34748) 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f22ba700 1 -- 172.21.15.185:0/4263658160 learned_addr learned my addr 172.21.15.185:0/4263658160 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1e06470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 -- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 msgr2=0x1e10140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 -- 172.21.15.185:0/4263658160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 msgr2=0x1e06470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1e06470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 -- 172.21.15.185:0/4263658160 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1aff720 con 0x1b4f7b0 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f22ba700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f12b8700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1e06960 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f54e0009eb0 tx=0x7f54e002b000).ready entity=mon.2 client_cookie=f7a456d92df63787 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0009ae0 con 0x1b4f7b0 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54e0023040 con 0x1b4f7b0 2020-11-16T22:21:53.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54dc044c90 con 0x1b4f7b0 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54e002b7a0 con 0x1b4f7b0 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 10 client.6866 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1b0ae00 con 0x1b4f7b0 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc044c90 con 0x1b4f7b0 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f54e0028070 con 0x1b4f7b0 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 10 client.6866 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:53.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 10 client.6866 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 20 client.6866 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1da6f30 0x1e10630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 4 client.6866 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1b92f50 con 0x1da6f30 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1da6f30 0x1e10630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 10 client.6866 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:53.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54f1ab9700 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1da6f30 0x1e10630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=b9939441a9896ca0 server_cookie=1b4be4c039fefecc in_seq=0 out_seq=0 2020-11-16T22:21:53.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 1 -- 172.21.15.185:0/4263658160 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1b92f50 con 0x1da6f30 2020-11-16T22:21:53.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f54bffff700 10 client.6866 handle_command_reply: tid=0 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.458+0000 7f55027ae740 1 client.6866 shutdown 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 msgr2=0x1e06960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1e06960 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f54e0009eb0 tx=0x7f54e002b000).stop 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 shutdown_connections 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0x1da6f30 0x1e10630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1d1de40 0x1e10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54bffff700 0 client.6866 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1ae1bb0 0x1e06470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 --2- 172.21.15.185:0/4263658160 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b4f7b0 0x1e06960 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 >> 172.21.15.185:0/4263658160 conn(0x7f54b8002320 msgr2=0x1da2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 shutdown_connections 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 1 -- 172.21.15.185:0/4263658160 wait complete. 2020-11-16T22:21:53.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f55027ae740 20 client.6866 trim_cache size 0 max 16384 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 msgr2=0x7f54c4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 0x7f54c4057550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f54e0006ef0 tx=0x7f54e0015000).stop 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 msgr2=0x7f54ec0b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f54dc00ef50 tx=0x7f54dc0091d0).stop 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 shutdown_connections 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54c4055090 0x7f54c4057550 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f54ec063550 0x7f54ec0b04a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54ec0b3700 0x7f54ec157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 --2- 172.21.15.185:0/1529810732 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f54ec0b2070 0x7f54ec156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 >> 172.21.15.185:0/1529810732 conn(0x7f54ec0649a0 msgr2=0x7f54ec141050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 shutdown_connections 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.462+0000 7f54f3d1d700 1 -- 172.21.15.185:0/1529810732 wait complete. 2020-11-16T22:21:53.464 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "785a2670-69d6-4efa-ada4-4d2d650f3105": { 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "785a2670-69d6-4efa-ada4-4d2d650f3105", 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:21:53.465 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:53.466 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:53.466 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:53.471 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:53.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.602+0000 7f395ffff700 1 Processor -- start 2020-11-16T22:21:53.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.602+0000 7f395ffff700 1 -- start start 2020-11-16T22:21:53.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f39600b2920 0x7f396013e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3960066ad0 con 0x7f396013e720 2020-11-16T22:21:53.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3960140d10 con 0x7f39600b2920 2020-11-16T22:21:53.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960144d90 con 0x7f39600b7470 2020-11-16T22:21:53.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56428/0 (socket says 172.21.15.185:56428) 2020-11-16T22:21:53.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 -- 172.21.15.185:0/659194698 learned_addr learned my addr 172.21.15.185:0/659194698 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f39567fc700 1 --2- 172.21.15.185:0/659194698 >> v2:172.21.15.185:3300/0 conn(0x7f39600b2920 0x7f396013e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 -- 172.21.15.185:0/659194698 >> v2:172.21.15.185:3300/0 conn(0x7f39600b2920 msgr2=0x7f396013e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 --2- 172.21.15.185:0/659194698 >> v2:172.21.15.185:3300/0 conn(0x7f39600b2920 0x7f396013e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 -- 172.21.15.185:0/659194698 >> v1:172.21.15.185:6789/0 conn(0x7f396013e720 legacy=0x7f3960141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 -- 172.21.15.185:0/659194698 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960144ff0 con 0x7f39600b7470 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395effd700 1 --2- 172.21.15.185:0/659194698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3950006fd0 tx=0x7f3950006f40).ready entity=mon.2 client_cookie=acf35e47079a541e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395e7fc700 1 -- 172.21.15.185:0/659194698 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000de60 con 0x7f39600b7470 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395e7fc700 1 -- 172.21.15.185:0/659194698 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3950010470 con 0x7f39600b7470 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395e7fc700 1 -- 172.21.15.185:0/659194698 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39500107f0 con 0x7f39600b7470 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- 172.21.15.185:0/659194698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 msgr2=0x7f39600b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 --2- 172.21.15.185:0/659194698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f3950006fd0 tx=0x7f3950006f40).stop 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- 172.21.15.185:0/659194698 shutdown_connections 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 --2- 172.21.15.185:0/659194698 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b7470 0x7f39600b7850 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 --2- 172.21.15.185:0/659194698 >> v2:172.21.15.185:3300/0 conn(0x7f39600b2920 0x7f396013e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- 172.21.15.185:0/659194698 >> 172.21.15.185:0/659194698 conn(0x7f39600637b0 msgr2=0x7f39600b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- 172.21.15.185:0/659194698 shutdown_connections 2020-11-16T22:21:53.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.606+0000 7f395ffff700 1 -- 172.21.15.185:0/659194698 wait complete. 2020-11-16T22:21:53.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.610+0000 7f395ffff700 1 Processor -- start 2020-11-16T22:21:53.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.610+0000 7f395ffff700 1 -- start start 2020-11-16T22:21:53.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39600b7470 0x7f3960140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 0x7f3960153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b1ba0 con 0x7f396013e720 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b1e40 con 0x7f39600b7470 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960152a10 con 0x7f39600b2920 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56434/0 (socket says 172.21.15.185:56434) 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f39567fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39600b7470 0x7f3960140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 -- 172.21.15.185:0/1369913523 learned_addr learned my addr 172.21.15.185:0/1369913523 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395f7fe700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 0x7f3960153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 -- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39600b7470 msgr2=0x7f3960140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39600b7470 0x7f3960140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 -- 172.21.15.185:0/1369913523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 msgr2=0x7f3960153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 0x7f3960153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 -- 172.21.15.185:0/1369913523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960152cb0 con 0x7f39600b2920 2020-11-16T22:21:53.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395f7fe700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 0x7f3960153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:53.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395effd700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f395000b430 tx=0x7f39600b3010).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950018860 con 0x7f39600b2920 2020-11-16T22:21:53.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39500189c0 con 0x7f39600b2920 2020-11-16T22:21:53.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950018d60 con 0x7f39600b2920 2020-11-16T22:21:53.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3960140a20 con 0x7f39600b2920 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39600b6210 con 0x7f39600b2920 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.614+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396005d200 con 0x7f39600b2920 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.618+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f395002b070 con 0x7f39600b2920 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.618+0000 7f39577fe700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.618+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3950071870 con 0x7f39600b2920 2020-11-16T22:21:53.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.618+0000 7f39567fc700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.618+0000 7f39567fc700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3960067b50 tx=0x7f3948008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.622+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3950047b50 con 0x7f39600b2920 2020-11-16T22:21:53.815 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.822+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39600617b0 con 0x7f39600b2920 2020-11-16T22:21:53.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.822+0000 7f39577fe700 1 -- 172.21.15.185:0/1369913523 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+2524 (secure 0 0 0) 0x7f395004b170 con 0x7f39600b2920 2020-11-16T22:21:53.816 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:53.816 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:49.075839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":6,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:53.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 msgr2=0x7f393c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f3960067b50 tx=0x7f3948008040).stop 2020-11-16T22:21:53.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 msgr2=0x7f396014f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f395000b430 tx=0x7f39600b3010).stop 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 shutdown_connections 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f393c055060 0x7f393c057520 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f396013e720 0x7f3960153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f39600b2920 0x7f396014f3a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 --2- 172.21.15.185:0/1369913523 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f39600b7470 0x7f3960140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 >> 172.21.15.185:0/1369913523 conn(0x7f39600637b0 msgr2=0x7f39600b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 shutdown_connections 2020-11-16T22:21:53.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.826+0000 7f395ffff700 1 -- 172.21.15.185:0/1369913523 wait complete. 2020-11-16T22:21:53.820 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 91 2020-11-16T22:21:53.831 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:21:53.959 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.966+0000 7f51f7177700 1 Processor -- start 2020-11-16T22:21:53.960 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.966+0000 7f51f7177700 1 -- start start 2020-11-16T22:21:53.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51f00b5c40 0x7f51f00b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 0x7f51f0144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.961 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f01446e0 con 0x7f51f013dd30 2020-11-16T22:21:53.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f01444f0 con 0x7f51f00b5c40 2020-11-16T22:21:53.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0144980 con 0x7f51f00ba800 2020-11-16T22:21:53.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51f00b5c40 0x7f51f00b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5974700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 0x7f51f0144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5974700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 0x7f51f0144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56442/0 (socket says 172.21.15.185:56442) 2020-11-16T22:21:53.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5974700 1 -- 172.21.15.185:0/4021383341 learned_addr learned my addr 172.21.15.185:0/4021383341 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f51f00b5c40 0x7f51f00b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34770/0 (socket says 172.21.15.185:34770) 2020-11-16T22:21:53.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 -- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 msgr2=0x7f51f0144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 --2- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 0x7f51f0144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 -- 172.21.15.185:0/4021383341 >> v1:172.21.15.96:6789/0 conn(0x7f51f013dd30 legacy=0x7f51f00b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 -- 172.21.15.185:0/4021383341 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0144c20 con 0x7f51f00b5c40 2020-11-16T22:21:53.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f6175700 1 --2- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00b5c40 0x7f51f00b6020 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f51e8009c90 tx=0x7f51e80091d0).ready entity=mon.1 client_cookie=d91f7aa95ee1a0b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5173700 1 -- 172.21.15.185:0/4021383341 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800ba70 con 0x7f51f00b5c40 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5173700 1 -- 172.21.15.185:0/4021383341 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e800bbd0 con 0x7f51f00b5c40 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f5173700 1 -- 172.21.15.185:0/4021383341 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8011400 con 0x7f51f00b5c40 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00b5c40 msgr2=0x7f51f00b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 --2- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00b5c40 0x7f51f00b6020 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f51e8009c90 tx=0x7f51e80091d0).stop 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- 172.21.15.185:0/4021383341 shutdown_connections 2020-11-16T22:21:53.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 --2- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00b5c40 0x7f51f00b6020 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 --2- 172.21.15.185:0/4021383341 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f00ba800 0x7f51f0144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- 172.21.15.185:0/4021383341 >> 172.21.15.185:0/4021383341 conn(0x7f51f005ba80 msgr2=0x7f51f005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- 172.21.15.185:0/4021383341 shutdown_connections 2020-11-16T22:21:53.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- 172.21.15.185:0/4021383341 wait complete. 2020-11-16T22:21:53.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 Processor -- start 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.970+0000 7f51f7177700 1 -- start start 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f00b5c40 0x7f51f00ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f013dd30 0x7f51f01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00b0270 con 0x7f51f00b5c40 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00adbc0 con 0x7f51f00ba800 2020-11-16T22:21:53.969 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f01547c0 con 0x7f51f013dd30 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f6976700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f013dd30 0x7f51f01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34772/0 (socket says 172.21.15.185:34772) 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 -- 172.21.15.185:0/3676415439 learned_addr learned my addr 172.21.15.185:0/3676415439 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f6175700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f00b5c40 0x7f51f00ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.970 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 -- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f013dd30 msgr2=0x7f51f01435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f013dd30 0x7f51f01435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 -- 172.21.15.185:0/3676415439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f00b5c40 msgr2=0x7f51f00ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f00b5c40 0x7f51f00ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 -- 172.21.15.185:0/3676415439 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00b0fc0 con 0x7f51f00ba800 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f5974700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f51ec00f860 tx=0x7f51ec008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51e27fc700 1 -- 172.21.15.185:0/3676415439 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec014670 con 0x7f51f00ba800 2020-11-16T22:21:53.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51e27fc700 1 -- 172.21.15.185:0/3676415439 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec0147d0 con 0x7f51f00ba800 2020-11-16T22:21:53.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f01538f0 con 0x7f51f00ba800 2020-11-16T22:21:53.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51e27fc700 1 -- 172.21.15.185:0/3676415439 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec014b50 con 0x7f51f00ba800 2020-11-16T22:21:53.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0143ab0 con 0x7f51f00ba800 2020-11-16T22:21:53.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.974+0000 7f51e27fc700 1 -- 172.21.15.185:0/3676415439 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f51ec024070 con 0x7f51f00ba800 2020-11-16T22:21:53.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f51e27fc700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 0x7f51c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f51e27fc700 1 -- 172.21.15.185:0/3676415439 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51ec06e1d0 con 0x7f51f00ba800 2020-11-16T22:21:53.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f51f6175700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 0x7f51c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f51f6175700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 0x7f51c8057520 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f51e800b770 tx=0x7f51e8016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f5206e6a740 1 Processor -- start 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.978+0000 7f5206e6a740 1 -- start start 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x17704b0 con 0x1658bb0 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1824930 con 0x18e4da0 2020-11-16T22:21:53.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x16e46f0 con 0x16c67b0 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34784/0 (socket says 172.21.15.185:34784) 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 -- 172.21.15.185:0/1890458683 learned_addr learned my addr 172.21.15.185:0/1890458683 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56456/0 (socket says 172.21.15.185:56456) 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f5974700 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 -- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 msgr2=0x18e48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 -- 172.21.15.185:0/1890458683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 msgr2=0x18e2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 -- 172.21.15.185:0/1890458683 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16df5a0 con 0x18e4da0 2020-11-16T22:21:53.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51f6976700 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f51e4003990 tx=0x7f51e4007aa0).ready entity=mon.1 client_cookie=ef6b50919f438647 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51cf7fe700 1 -- 172.21.15.185:0/1890458683 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400f840 con 0x18e4da0 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51cf7fe700 1 -- 172.21.15.185:0/1890458683 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e400f9a0 con 0x18e4da0 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f51cf7fe700 1 -- 172.21.15.185:0/1890458683 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400fd20 con 0x18e4da0 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 msgr2=0x18e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f51e4003990 tx=0x7f51e4007aa0).stop 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- 172.21.15.185:0/1890458683 shutdown_connections 2020-11-16T22:21:53.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18e4da0 0x18e3380 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 --2- 172.21.15.185:0/1890458683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- 172.21.15.185:0/1890458683 >> 172.21.15.185:0/1890458683 conn(0x1894e40 msgr2=0x18e1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- 172.21.15.185:0/1890458683 shutdown_connections 2020-11-16T22:21:53.980 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- 172.21.15.185:0/1890458683 wait complete. 2020-11-16T22:21:53.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 -- start start 2020-11-16T22:21:53.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.982+0000 7f5206e6a740 1 Processor -- start 2020-11-16T22:21:53.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x192c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18e4da0 0x1912ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x18116e0 con 0x16c67b0 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1811820 con 0x1658bb0 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1812a90 con 0x18e4da0 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 10 client.0 ll_register_callbacks cb 0x1807ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:53.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34786/0 (socket says 172.21.15.185:34786) 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 -- 172.21.15.185:0/4145788803 learned_addr learned my addr 172.21.15.185:0/4145788803 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6976700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18e4da0 0x1912ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f5974700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x192c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 -- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18e4da0 msgr2=0x1912ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18e4da0 0x1912ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 -- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 msgr2=0x192c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x192c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 -- 172.21.15.185:0/4145788803 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1802280 con 0x1658bb0 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f6175700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f51e8011000 tx=0x7f51e801e000).ready entity=mon.1 client_cookie=575598498bad01ed server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8011e00 con 0x1658bb0 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51e8029420 con 0x1658bb0 2020-11-16T22:21:53.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18c56c0 con 0x1658bb0 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e80297f0 con 0x1658bb0 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 10 client.7684 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1637fd0 con 0x1658bb0 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18c56c0 con 0x1658bb0 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f51e801c3e0 con 0x1658bb0 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 10 client.7684 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:53.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 10 client.7684 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:53.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 20 client.7684 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:53.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e1560 0x1918950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:53.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 4 client.7684 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:21:53.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1702c50 con 0x18e1560 2020-11-16T22:21:53.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f5974700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e1560 0x1918950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51ccff9700 10 client.7684 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.986+0000 7f51f5974700 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e1560 0x1918950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bcc07bdc0f08720b server_cookie=8b22dc548ed54830 in_seq=0 out_seq=0 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f51ccff9700 1 -- 172.21.15.185:0/4145788803 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1702c50 con 0x18e1560 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f51ccff9700 10 client.7684 handle_command_reply: tid=0 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 client.7684 shutdown 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 msgr2=0x1928830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:53.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f51e8011000 tx=0x7f51e801e000).stop 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 shutdown_connections 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e1560 0x1918950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1928830 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18e4da0 0x1912ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 --2- 172.21.15.185:0/4145788803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x192c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:53.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f51ccff9700 0 client.7684 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:53.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 >> 172.21.15.185:0/4145788803 conn(0x1894e40 msgr2=0x18e2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:53.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 shutdown_connections 2020-11-16T22:21:53.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 1 -- 172.21.15.185:0/4145788803 wait complete. 2020-11-16T22:21:53.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:53.990+0000 7f5206e6a740 20 client.7684 trim_cache size 0 max 16384 2020-11-16T22:21:53.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 Processor -- start 2020-11-16T22:21:53.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 -- start start 2020-11-16T22:21:54.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 0x18e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x17e9e40 con 0x1658bb0 2020-11-16T22:21:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1919c50 con 0x1894e40 2020-11-16T22:21:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1841680 con 0x16c67b0 2020-11-16T22:21:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 0x18e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.006+0000 7f51f6976700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 0x18e7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34798/0 (socket says 172.21.15.185:34798) 2020-11-16T22:21:54.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 -- 172.21.15.185:0/3719797078 learned_addr learned my addr 172.21.15.185:0/3719797078 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:54.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f5974700 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 -- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 msgr2=0x18e7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 0x18e7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 -- 172.21.15.185:0/3719797078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 msgr2=0x18e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.005 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 -- 172.21.15.185:0/3719797078 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc008160 con 0x16c67b0 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e0200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f51e4003990 tx=0x7f51e4018000).ready entity=mon.2 client_cookie=38d9c8cbe0b52c95 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51cffff700 1 -- 172.21.15.185:0/3719797078 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e40075b0 con 0x16c67b0 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51cffff700 1 -- 172.21.15.185:0/3719797078 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4024070 con 0x16c67b0 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51cffff700 1 -- 172.21.15.185:0/3719797078 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400fb10 con 0x16c67b0 2020-11-16T22:21:54.006 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 msgr2=0x18e0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e0200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f51e4003990 tx=0x7f51e4018000).stop 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- 172.21.15.185:0/3719797078 shutdown_connections 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1894e40 0x18e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x16c67b0 0x18e0200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3719797078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1658bb0 0x18e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.007 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- 172.21.15.185:0/3719797078 >> 172.21.15.185:0/3719797078 conn(0x7f51bc002320 msgr2=0x18e19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- 172.21.15.185:0/3719797078 shutdown_connections 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- 172.21.15.185:0/3719797078 wait complete. 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- start start 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 Processor -- start 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.008 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x1921c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1894e40 0x1982e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x18e09b0 con 0x16c67b0 2020-11-16T22:21:54.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19287f0 con 0x1658bb0 2020-11-16T22:21:54.009 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1928550 con 0x1894e40 2020-11-16T22:21:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34800/0 (socket says 172.21.15.185:34800) 2020-11-16T22:21:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6976700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1894e40 0x1982e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f5206e6a740 10 client.0 ll_register_callbacks cb 0x1917ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:21:54.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 -- 172.21.15.185:0/3339485856 learned_addr learned my addr 172.21.15.185:0/3339485856 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f5974700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x1921c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 -- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1894e40 msgr2=0x1982e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.010+0000 7f51f6175700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1894e40 0x1982e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f6175700 1 -- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 msgr2=0x1921c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f6175700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x1921c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f6175700 1 -- 172.21.15.185:0/3339485856 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1676720 con 0x1658bb0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f6175700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f51e80033b0 tx=0x7f51e802a000).ready entity=mon.1 client_cookie=58ba6ff20d2e9390 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8011560 con 0x1658bb0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f51e801c890 con 0x1658bb0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e400f9a0 con 0x1658bb0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e801ca50 con 0x1658bb0 2020-11-16T22:21:54.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 10 client.7690 fetch_fsmap learned FSMap version 91 2020-11-16T22:21:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1681e00 con 0x1658bb0 2020-11-16T22:21:54.013 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e400f9a0 con 0x1658bb0 2020-11-16T22:21:54.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 91) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f51e803c430 con 0x1658bb0 2020-11-16T22:21:54.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 10 client.7690 fetch_fsmap finished waiting for FSMap version 91 2020-11-16T22:21:54.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 10 client.7690 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:21:54.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 20 client.7690 populate_metadata read hostname 'smithi185' 2020-11-16T22:21:54.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e4da0 0x1983300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 4 client.7690 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x1709f50 con 0x18e4da0 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f5974700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e4da0 0x1983300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 10 client.7690 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51f5974700 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e4da0 0x1983300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=77cf52bb39459fd7 server_cookie=88a1b3903da2ef51 in_seq=0 out_seq=0 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 1 -- 172.21.15.185:0/3339485856 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x1709f50 con 0x18e4da0 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 10 client.7690 handle_command_reply: tid=0 2020-11-16T22:21:54.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 client.7690 shutdown 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 msgr2=0x1922a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f51e80033b0 tx=0x7f51e802a000).stop 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 shutdown_connections 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x18e4da0 0x1983300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1658bb0 0x1922a40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f51ccff9700 0 client.7690 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1894e40 0x1982e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 --2- 172.21.15.185:0/3339485856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x16c67b0 0x1921c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 >> 172.21.15.185:0/3339485856 conn(0x7f51bc002320 msgr2=0x18e7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 shutdown_connections 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 1 -- 172.21.15.185:0/3339485856 wait complete. 2020-11-16T22:21:54.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.014+0000 7f5206e6a740 20 client.7690 trim_cache size 0 max 16384 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 msgr2=0x7f51c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 0x7f51c8057520 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f51e800b770 tx=0x7f51e8016000).stop 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 msgr2=0x7f51f01542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f51ec00f860 tx=0x7f51ec008040).stop 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 shutdown_connections 2020-11-16T22:21:54.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f51c8055060 0x7f51c8057520 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f51f013dd30 0x7f51f01435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f00b5c40 0x7f51f00ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 --2- 172.21.15.185:0/3676415439 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f51f00ba800 0x7f51f01542d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 >> 172.21.15.185:0/3676415439 conn(0x7f51f005ba80 msgr2=0x7f51f013d600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 shutdown_connections 2020-11-16T22:21:54.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.018+0000 7f51f7177700 1 -- 172.21.15.185:0/3676415439 wait complete. 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout: "785a2670-69d6-4efa-ada4-4d2d650f3105": { 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:21:54.022 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "785a2670-69d6-4efa-ada4-4d2d650f3105", 2020-11-16T22:21:54.023 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:21:54.023 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:54.023 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:21:54.023 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:21:54.026 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:21:54.027 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:54.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 Processor -- start 2020-11-16T22:21:54.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- start start 2020-11-16T22:21:54.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba8066f80 con 0x7f7ba8148550 2020-11-16T22:21:54.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba81478d0 con 0x7f7ba80b37e0 2020-11-16T22:21:54.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8147b70 con 0x7f7ba80b0f90 2020-11-16T22:21:54.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56486/0 (socket says 172.21.15.185:56486) 2020-11-16T22:21:54.147 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 -- 172.21.15.185:0/3430569929 learned_addr learned my addr 172.21.15.185:0/3430569929 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:54.148 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 -- 172.21.15.185:0/3430569929 >> v1:172.21.15.185:6789/0 conn(0x7f7ba80b37e0 legacy=0x7f7ba8141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 -- 172.21.15.185:0/3430569929 >> v1:172.21.15.96:6789/0 conn(0x7f7ba8148550 legacy=0x7f7ba8144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 -- 172.21.15.185:0/3430569929 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8147e10 con 0x7f7ba80b0f90 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba7fff700 1 --2- 172.21.15.185:0/3430569929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f7ba000ba20 tx=0x7f7ba0006eb0).ready entity=mon.2 client_cookie=831fa21a171a7b8f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba67fc700 1 -- 172.21.15.185:0/3430569929 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00098d0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba67fc700 1 -- 172.21.15.185:0/3430569929 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba0009a30 con 0x7f7ba80b0f90 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7ba67fc700 1 -- 172.21.15.185:0/3430569929 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0009db0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.149 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- 172.21.15.185:0/3430569929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 msgr2=0x7f7ba80b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 --2- 172.21.15.185:0/3430569929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f7ba000ba20 tx=0x7f7ba0006eb0).stop 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- 172.21.15.185:0/3430569929 shutdown_connections 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 --2- 172.21.15.185:0/3430569929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba80b0f90 0x7f7ba80b0530 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.154+0000 7f7badaa2700 1 -- 172.21.15.185:0/3430569929 >> 172.21.15.185:0/3430569929 conn(0x7f7ba80604a0 msgr2=0x7f7ba80608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- 172.21.15.185:0/3430569929 shutdown_connections 2020-11-16T22:21:54.150 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- 172.21.15.185:0/3430569929 wait complete. 2020-11-16T22:21:54.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 Processor -- start 2020-11-16T22:21:54.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- start start 2020-11-16T22:21:54.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 0x7f7ba8148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba8148550 0x7f7ba813c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba809c190 con 0x7f7ba80b37e0 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80962b0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7badaa2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8096550 con 0x7f7ba8148550 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.153 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34812/0 (socket says 172.21.15.185:34812) 2020-11-16T22:21:54.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 -- 172.21.15.185:0/435893492 learned_addr learned my addr 172.21.15.185:0/435893492 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:54.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba7fff700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba8148550 0x7f7ba813c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba6ffd700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 0x7f7ba8148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 -- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba8148550 msgr2=0x7f7ba813c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba8148550 0x7f7ba813c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 -- 172.21.15.185:0/435893492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 msgr2=0x7f7ba8148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 0x7f7ba8148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 -- 172.21.15.185:0/435893492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80967f0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba6ffd700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 0x7f7ba8148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7ba77fe700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f7b9800cb10 tx=0x7f7b98007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.158+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800a7f0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8096a90 con 0x7f7ba80b0f90 2020-11-16T22:21:54.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800a950 con 0x7f7ba80b0f90 2020-11-16T22:21:54.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800acd0 con 0x7f7ba80b0f90 2020-11-16T22:21:54.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8068170 con 0x7f7ba80b0f90 2020-11-16T22:21:54.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba8068450 con 0x7f7ba80b0f90 2020-11-16T22:21:54.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7b98010070 con 0x7f7ba80b0f90 2020-11-16T22:21:54.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7b9f7fe700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 0x7f7b84057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b9806dc10 con 0x7f7ba80b0f90 2020-11-16T22:21:54.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7ba6ffd700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 0x7f7b84057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.162+0000 7f7ba6ffd700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 0x7f7b84057550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7ba8067d60 tx=0x7f7b90009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.170+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7b98043e70 con 0x7f7ba80b0f90 2020-11-16T22:21:54.247 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:21:54.257+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:21:54.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.374+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ba8096c20 con 0x7f7ba80b0f90 2020-11-16T22:21:54.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.374+0000 7f7b9f7fe700 1 -- 172.21.15.185:0/435893492 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f7b98047490 con 0x7f7ba80b0f90 2020-11-16T22:21:54.366 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:54.367 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:54.261092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:54.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 msgr2=0x7f7b84057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 0x7f7b84057550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7ba8067d60 tx=0x7f7b90009040).stop 2020-11-16T22:21:54.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 msgr2=0x7f7ba809b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f7b9800cb10 tx=0x7f7b98007f20).stop 2020-11-16T22:21:54.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 shutdown_connections 2020-11-16T22:21:54.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7b84055090 0x7f7b84057550 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ba80b0f90 0x7f7ba809b140 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ba8148550 0x7f7ba813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 --2- 172.21.15.185:0/435893492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ba80b37e0 0x7f7ba8148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 >> 172.21.15.185:0/435893492 conn(0x7f7ba80604a0 msgr2=0x7f7ba80b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 shutdown_connections 2020-11-16T22:21:54.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.378+0000 7f7badaa2700 1 -- 172.21.15.185:0/435893492 wait complete. 2020-11-16T22:21:54.375 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 92 2020-11-16T22:21:54.385 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'a'] 2020-11-16T22:21:54.385 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:21:54.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.510+0000 7f498f06c700 1 Processor -- start 2020-11-16T22:21:54.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.510+0000 7f498f06c700 1 -- start start 2020-11-16T22:21:54.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f49880b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49880a9bd0 0x7f498813f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4988141430 con 0x7f49880b70a0 2020-11-16T22:21:54.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880664c0 con 0x7f49880a9bd0 2020-11-16T22:21:54.505 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4988066760 con 0x7f49880b2fc0 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d869700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f49880b70a0 0x7f4988066ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56490/0 (socket says 172.21.15.185:56490) 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d869700 1 -- 172.21.15.185:0/783078100 learned_addr learned my addr 172.21.15.185:0/783078100 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f49880b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e86b700 1 --2- 172.21.15.185:0/783078100 >> v2:172.21.15.96:3300/0 conn(0x7f49880a9bd0 0x7f498813f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801714499 0 0) 0x7f4988141430 con 0x7f49880b70a0 2020-11-16T22:21:54.506 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4970003240 con 0x7f49880b70a0 2020-11-16T22:21:54.508 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2161416214 0 0) 0x7f4970003240 con 0x7f49880b70a0 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 msgr2=0x7f49880b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f49880b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 >> v2:172.21.15.96:3300/0 conn(0x7f49880a9bd0 msgr2=0x7f498813f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 --2- 172.21.15.185:0/783078100 >> v2:172.21.15.96:3300/0 conn(0x7f49880a9bd0 0x7f498813f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988066a00 con 0x7f49880b70a0 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f49780014e0 con 0x7f49880b70a0 2020-11-16T22:21:54.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 >> v1:172.21.15.185:6789/0 conn(0x7f49880b70a0 legacy=0x7f4988066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49700037e0 0x7f4970003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49700044e0 0x7f4970004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f4970004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49880664c0 con 0x7f49700037e0 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988066760 con 0x7f49700044e0 2020-11-16T22:21:54.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970005340 con 0x7f49880b2fc0 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e86b700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f4970004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49700037e0 0x7f4970003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d869700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49700044e0 0x7f4970004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 -- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 msgr2=0x7f4970004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880b2fc0 0x7f4970004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 -- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49700044e0 msgr2=0x7f4970004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49700044e0 0x7f4970004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 -- 172.21.15.185:0/783078100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970003240 con 0x7f49700037e0 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e06a700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49700037e0 0x7f4970003c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f498400bbf0 tx=0x7f4984006df0).ready entity=mon.0 client_cookie=9b4d9ff59d50df7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400b300 con 0x7f49700037e0 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498400ea30 con 0x7f49700037e0 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498d068700 1 -- 172.21.15.185:0/783078100 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400edb0 con 0x7f49700037e0 2020-11-16T22:21:54.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- 172.21.15.185:0/783078100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49700037e0 msgr2=0x7f4970003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 --2- 172.21.15.185:0/783078100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49700037e0 0x7f4970003c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f498400bbf0 tx=0x7f4984006df0).stop 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498e86b700 1 -- 172.21.15.185:0/783078100 reap_dead start 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- 172.21.15.185:0/783078100 shutdown_connections 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.514+0000 7f498f06c700 1 -- 172.21.15.185:0/783078100 >> 172.21.15.185:0/783078100 conn(0x7f49880657c0 msgr2=0x7f4988065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- 172.21.15.185:0/783078100 shutdown_connections 2020-11-16T22:21:54.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- 172.21.15.185:0/783078100 wait complete. 2020-11-16T22:21:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 Processor -- start 2020-11-16T22:21:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- start start 2020-11-16T22:21:54.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49880aade0 0x7f49880ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4988144130 0x7f49880ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49880ac5d0 con 0x7f49880aade0 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49880ad580 con 0x7f4988144130 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498f06c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498809d5a0 con 0x7f49880a8db0 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.518+0000 7f498e86b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56512/0 (socket says 172.21.15.185:56512) 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 -- 172.21.15.185:0/1070136692 learned_addr learned my addr 172.21.15.185:0/1070136692 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e06a700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49880aade0 0x7f49880ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498d869700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4988144130 0x7f49880ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 -- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4988144130 msgr2=0x7f49880ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4988144130 0x7f49880ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 -- 172.21.15.185:0/1070136692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49880aade0 msgr2=0x7f49880ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49880aade0 0x7f49880ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 -- 172.21.15.185:0/1070136692 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880ab9c0 con 0x7f49880a8db0 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e86b700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4988064fb0 tx=0x7f49800091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000f8b0 con 0x7f49880a8db0 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498000fa30 con 0x7f49880a8db0 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000fdf0 con 0x7f49880a8db0 2020-11-16T22:21:54.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49881500b0 con 0x7f49880a8db0 2020-11-16T22:21:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49880aa280 con 0x7f49880a8db0 2020-11-16T22:21:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4988150240 con 0x7f49880a8db0 2020-11-16T22:21:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4980023070 con 0x7f49880a8db0 2020-11-16T22:21:54.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 0x7f4960057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f498006e170 con 0x7f49880a8db0 2020-11-16T22:21:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.522+0000 7f498e06a700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 0x7f4960057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:54.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.526+0000 7f498e06a700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 0x7f4960057550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f4984006d00 tx=0x7f4984019000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:54.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.534+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f49800443d0 con 0x7f49880a8db0 2020-11-16T22:21:54.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.738+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f498809d730 con 0x7f49880a8db0 2020-11-16T22:21:54.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.738+0000 7f49767fc700 1 -- 172.21.15.185:0/1070136692 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1750 (secure 0 0 0) 0x7f4980021020 con 0x7f49880a8db0 2020-11-16T22:21:54.732 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:54.732 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":104859,"bytes_used":57098240,"bytes_avail":773037015040,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6653,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":6545},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7080},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7083}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2020-11-16T22:21:53.189237+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:21:54.733 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 msgr2=0x7f4960057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 0x7f4960057550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f4984006d00 tx=0x7f4984019000).stop 2020-11-16T22:21:54.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 msgr2=0x7f49880a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:54.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4988064fb0 tx=0x7f49800091d0).stop 2020-11-16T22:21:54.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 shutdown_connections 2020-11-16T22:21:54.734 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4960055090 0x7f4960057550 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f49880a8db0 0x7f49880a91f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49880aade0 0x7f49880ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 --2- 172.21.15.185:0/1070136692 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4988144130 0x7f49880ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 >> 172.21.15.185:0/1070136692 conn(0x7f49880657c0 msgr2=0x7f49880b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 shutdown_connections 2020-11-16T22:21:54.735 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:54.742+0000 7f498f06c700 1 -- 172.21.15.185:0/1070136692 wait complete. 2020-11-16T22:21:54.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:21:55.755 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:55.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 Processor -- start 2020-11-16T22:21:55.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- start start 2020-11-16T22:21:55.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6af40602d0 0x7f6af40673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:55.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6af4148f90 0x7f6af4145d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:55.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af40b34f0 con 0x7f6af41458a0 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af40b7660 con 0x7f6af40602d0 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af40b7900 con 0x7f6af4148f90 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f6af4148f90 0x7f6af4145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af27fc700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f6af41458a0 0x7f6af4148880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56512/0 (socket says 172.21.15.185:56512) 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af2ffd700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6af40602d0 0x7f6af40673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:55.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af27fc700 1 -- 172.21.15.185:0/414449435 learned_addr learned my addr 172.21.15.185:0/414449435 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:55.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 -- 172.21.15.185:0/414449435 >> v2:172.21.15.96:3300/0 conn(0x7f6af40602d0 msgr2=0x7f6af40673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:55.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 --2- 172.21.15.185:0/414449435 >> v2:172.21.15.96:3300/0 conn(0x7f6af40602d0 0x7f6af40673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:55.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 -- 172.21.15.185:0/414449435 >> v1:172.21.15.185:6789/0 conn(0x7f6af41458a0 legacy=0x7f6af4148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:55.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af1ffb700 1 -- 172.21.15.185:0/414449435 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631395360 0 0) 0x7f6af40b34f0 con 0x7f6af41458a0 2020-11-16T22:21:55.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 -- 172.21.15.185:0/414449435 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40b7ba0 con 0x7f6af4148f90 2020-11-16T22:21:55.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af37fe700 1 --2- 172.21.15.185:0/414449435 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af4148f90 0x7f6af4145d00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f6af40b5040 tx=0x7f6adc006df0).ready entity=mon.1 client_cookie=305988da4aea138e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af1ffb700 1 -- 172.21.15.185:0/414449435 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0097b0 con 0x7f6af4148f90 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af1ffb700 1 -- 172.21.15.185:0/414449435 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc009910 con 0x7f6af4148f90 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af1ffb700 1 -- 172.21.15.185:0/414449435 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc009c90 con 0x7f6af4148f90 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- 172.21.15.185:0/414449435 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af4148f90 msgr2=0x7f6af4145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 --2- 172.21.15.185:0/414449435 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af4148f90 0x7f6af4145d00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f6af40b5040 tx=0x7f6adc006df0).stop 2020-11-16T22:21:55.890 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- 172.21.15.185:0/414449435 shutdown_connections 2020-11-16T22:21:55.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 --2- 172.21.15.185:0/414449435 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af4148f90 0x7f6af4145d00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:55.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 --2- 172.21.15.185:0/414449435 >> v2:172.21.15.96:3300/0 conn(0x7f6af40602d0 0x7f6af40673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:55.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- 172.21.15.185:0/414449435 >> 172.21.15.185:0/414449435 conn(0x7f6af405ba80 msgr2=0x7f6af405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:55.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- 172.21.15.185:0/414449435 shutdown_connections 2020-11-16T22:21:55.891 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.894+0000 7f6af3fff700 1 -- 172.21.15.185:0/414449435 wait complete. 2020-11-16T22:21:55.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.898+0000 7f6af3fff700 1 Processor -- start 2020-11-16T22:21:55.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.898+0000 7f6af3fff700 1 -- start start 2020-11-16T22:21:55.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:55.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6af41458a0 0x7f6af4144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:55.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 0x7f6af4144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:55.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af414ae30 con 0x7f6af41458a0 2020-11-16T22:21:55.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af414b0d0 con 0x7f6af40602d0 2020-11-16T22:21:55.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af414b370 con 0x7f6af4148f90 2020-11-16T22:21:55.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 0x7f6af4144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34846/0 (socket says 172.21.15.185:34846) 2020-11-16T22:21:56.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af2ffd700 1 -- 172.21.15.185:0/94178529 learned_addr learned my addr 172.21.15.185:0/94178529 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:56.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af27fc700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6af41458a0 0x7f6af4144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 -- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 msgr2=0x7f6af414d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.100 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 -- 172.21.15.185:0/94178529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6af41458a0 msgr2=0x7f6af4144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6af41458a0 0x7f6af4144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 -- 172.21.15.185:0/94178529 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af414b610 con 0x7f6af4148f90 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af2ffd700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af37fe700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 0x7f6af4144f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f6adc012000 tx=0x7f6adc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:56.101 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc009e40 con 0x7f6af4148f90 2020-11-16T22:21:56.102 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc01d5d0 con 0x7f6af4148f90 2020-11-16T22:21:56.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4145410 con 0x7f6af4148f90 2020-11-16T22:21:56.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc01d950 con 0x7f6af4148f90 2020-11-16T22:21:56.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af4149980 con 0x7f6af4148f90 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af414b7a0 con 0x7f6af4148f90 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.902+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6adc025030 con 0x7f6af4148f90 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.906+0000 7f6aeaffd700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 0x7f6ac4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.906+0000 7f6af2ffd700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 0x7f6ac4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.906+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6adc06f520 con 0x7f6af4148f90 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.906+0000 7f6af2ffd700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 0x7f6ac4057550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6ae4009ec0 tx=0x7f6ae40091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:56.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:55.910+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6adc045780 con 0x7f6af4148f90 2020-11-16T22:21:56.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.110+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af405d200 con 0x7f6af4148f90 2020-11-16T22:21:56.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.110+0000 7f6aeaffd700 1 -- 172.21.15.185:0/94178529 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+2526 (secure 0 0 0) 0x7f6adc0239f0 con 0x7f6af4148f90 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 msgr2=0x7f6ac4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 0x7f6ac4057550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f6ae4009ec0 tx=0x7f6ae40091d0).stop 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 msgr2=0x7f6af4144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 0x7f6af4144f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f6adc012000 tx=0x7f6adc015000).stop 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 shutdown_connections 2020-11-16T22:21:56.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6ac4055090 0x7f6ac4057550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6af40602d0 0x7f6af414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6af4148f90 0x7f6af4144f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 --2- 172.21.15.185:0/94178529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6af41458a0 0x7f6af4144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 >> 172.21.15.185:0/94178529 conn(0x7f6af405ba80 msgr2=0x7f6af4067390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 shutdown_connections 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.114+0000 7f6af3fff700 1 -- 172.21.15.185:0/94178529 wait complete. 2020-11-16T22:21:56.311 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 92 2020-11-16T22:21:56.312 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:56.312 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:54.261092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:56.312 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'a'] 2020-11-16T22:21:56.313 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:21:56.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.446+0000 7f97e0d83700 1 Processor -- start 2020-11-16T22:21:56.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.446+0000 7f97e0d83700 1 -- start start 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 0x7f97dc147970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97dc0b2a30 con 0x7f97dc0b0f70 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97dc0b24f0 con 0x7f97dc0680a0 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc148570 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97d3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 0x7f97dc147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97db7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f97dc0680a0 0x7f97dc14bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56526/0 (socket says 172.21.15.185:56526) 2020-11-16T22:21:56.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97db7fe700 1 -- 172.21.15.185:0/2358793424 learned_addr learned my addr 172.21.15.185:0/2358793424 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:21:56.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97daffd700 1 -- 172.21.15.185:0/2358793424 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509499939 0 0) 0x7f97dc0b24f0 con 0x7f97dc0680a0 2020-11-16T22:21:56.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97daffd700 1 -- 172.21.15.185:0/2358793424 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97bc003240 con 0x7f97dc0680a0 2020-11-16T22:21:56.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97d3fff700 1 -- 172.21.15.185:0/2358793424 >> v1:172.21.15.185:6789/0 conn(0x7f97dc0680a0 legacy=0x7f97dc14bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97d3fff700 1 -- 172.21.15.185:0/2358793424 >> v1:172.21.15.96:6789/0 conn(0x7f97dc0b0f70 legacy=0x7f97dc147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97d3fff700 1 -- 172.21.15.185:0/2358793424 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0b2790 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97d3fff700 1 --2- 172.21.15.185:0/2358793424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 0x7f97dc147970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f97cc00b9d0 tx=0x7f97cc006df0).ready entity=mon.2 client_cookie=695c648bbc2a2724 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97daffd700 1 -- 172.21.15.185:0/2358793424 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0097b0 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97daffd700 1 -- 172.21.15.185:0/2358793424 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc009910 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97daffd700 1 -- 172.21.15.185:0/2358793424 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc009c90 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- 172.21.15.185:0/2358793424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 msgr2=0x7f97dc147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 --2- 172.21.15.185:0/2358793424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 0x7f97dc147970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f97cc00b9d0 tx=0x7f97cc006df0).stop 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- 172.21.15.185:0/2358793424 shutdown_connections 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 --2- 172.21.15.185:0/2358793424 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b1fa0 0x7f97dc147970 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- 172.21.15.185:0/2358793424 >> 172.21.15.185:0/2358793424 conn(0x7f97dc0657c0 msgr2=0x7f97dc065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- 172.21.15.185:0/2358793424 shutdown_connections 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- 172.21.15.185:0/2358793424 wait complete. 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 Processor -- start 2020-11-16T22:21:56.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.450+0000 7f97e0d83700 1 -- start start 2020-11-16T22:21:56.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 0x7f97dc0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 0x7f97dc0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc0b06f0 con 0x7f97dc0b1fa0 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc158840 con 0x7f97dc0680a0 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97db7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc158560 con 0x7f97dc0b0f70 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 0x7f97dc0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97db7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34860/0 (socket says 172.21.15.185:34860) 2020-11-16T22:21:56.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97db7fe700 1 -- 172.21.15.185:0/1335380455 learned_addr learned my addr 172.21.15.185:0/1335380455 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97dbfff700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 0x7f97dc0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 -- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 msgr2=0x7f97dc0aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 -- 172.21.15.185:0/1335380455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 msgr2=0x7f97dc0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 0x7f97dc0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 -- 172.21.15.185:0/1335380455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0b0370 con 0x7f97dc0b0f70 2020-11-16T22:21:56.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97dbfff700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 0x7f97dc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97db7fe700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d3fff700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 0x7f97dc0ad1a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f97cc012000 tx=0x7f97cc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc006760 con 0x7f97dc0b0f70 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc011c80 con 0x7f97dc0b0f70 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc0ad690 con 0x7f97dc0b0f70 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc01e460 con 0x7f97dc0b0f70 2020-11-16T22:21:56.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc09d800 con 0x7f97dc0b0f70 2020-11-16T22:21:56.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc05d200 con 0x7f97dc0b0f70 2020-11-16T22:21:56.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f97cc01a040 con 0x7f97dc0b0f70 2020-11-16T22:21:56.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.454+0000 7f97d37fe700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 0x7f97b8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:56.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.458+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f97cc06efe0 con 0x7f97dc0b0f70 2020-11-16T22:21:56.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.458+0000 7f97db7fe700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 0x7f97b8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:56.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.458+0000 7f97db7fe700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 0x7f97b8057520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f97dc0aff30 tx=0x7f97c4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:56.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.466+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f97cc0452f0 con 0x7f97dc0b0f70 2020-11-16T22:21:56.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.674+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f97dc0aaae0 con 0x7f97dc0b0f70 2020-11-16T22:21:56.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.674+0000 7f97d37fe700 1 -- 172.21.15.185:0/1335380455 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1791 (secure 0 0 0) 0x7f97cc023710 con 0x7f97dc0b0f70 2020-11-16T22:21:56.666 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:56.667 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":110202,"bytes_used":57311232,"bytes_avail":773036802048,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7336,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":6545},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7080},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7083}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2020-11-16T22:21:55.190159+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:21:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.674+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 msgr2=0x7f97b8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 0x7f97b8057520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f97dc0aff30 tx=0x7f97c4009040).stop 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 msgr2=0x7f97dc0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 0x7f97dc0ad1a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f97cc012000 tx=0x7f97cc015000).stop 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 shutdown_connections 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f97b8055060 0x7f97b8057520 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97dc0680a0 0x7f97dc0aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97dc0b1fa0 0x7f97dc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 --2- 172.21.15.185:0/1335380455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97dc0b0f70 0x7f97dc0ad1a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 >> 172.21.15.185:0/1335380455 conn(0x7f97dc0657c0 msgr2=0x7f97dc0b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 shutdown_connections 2020-11-16T22:21:56.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:56.678+0000 7f97e0d83700 1 -- 172.21.15.185:0/1335380455 wait complete. 2020-11-16T22:21:56.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:21:57.683 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:57.816 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 Processor -- start 2020-11-16T22:21:57.817 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- start start 2020-11-16T22:21:57.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc7c0b0f90 0x7efc7c141960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7c066f80 con 0x7efc7c0b0f90 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7c1478d0 con 0x7efc7c148550 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c147b70 con 0x7efc7c0b37e0 2020-11-16T22:21:57.819 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7bfff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34870/0 (socket says 172.21.15.185:34870) 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56544/0 (socket says 172.21.15.185:56544) 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7bfff700 1 -- 172.21.15.185:0/2897571254 learned_addr learned my addr 172.21.15.185:0/2897571254 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc812d3700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.96:3300/0 conn(0x7efc7c0b0f90 0x7efc7c141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 -- 172.21.15.185:0/2897571254 >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 msgr2=0x7efc7c141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:57.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 -- 172.21.15.185:0/2897571254 >> v2:172.21.15.96:3300/0 conn(0x7efc7c0b0f90 msgr2=0x7efc7c141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.96:3300/0 conn(0x7efc7c0b0f90 0x7efc7c141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 -- 172.21.15.185:0/2897571254 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c147e10 con 0x7efc7c0b37e0 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7bfff700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc80ad2700 1 --2- 172.21.15.185:0/2897571254 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7efc7c0651a0 tx=0x7efc64006df0).ready entity=mon.2 client_cookie=bedef9844625bec7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7b7fe700 1 -- 172.21.15.185:0/2897571254 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64009640 con 0x7efc7c0b37e0 2020-11-16T22:21:57.821 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7b7fe700 1 -- 172.21.15.185:0/2897571254 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc640097a0 con 0x7efc7c0b37e0 2020-11-16T22:21:57.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc7b7fe700 1 -- 172.21.15.185:0/2897571254 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64009b20 con 0x7efc7c0b37e0 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- 172.21.15.185:0/2897571254 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 msgr2=0x7efc7c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- 172.21.15.185:0/2897571254 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7efc7c0651a0 tx=0x7efc64006df0).stop 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- 172.21.15.185:0/2897571254 shutdown_connections 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.96:3300/0 conn(0x7efc7c0b0f90 0x7efc7c141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- 172.21.15.185:0/2897571254 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b37e0 0x7efc7c0b0530 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 --2- 172.21.15.185:0/2897571254 >> v2:172.21.15.185:3300/0 conn(0x7efc7c148550 0x7efc7c141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- 172.21.15.185:0/2897571254 >> 172.21.15.185:0/2897571254 conn(0x7efc7c0604a0 msgr2=0x7efc7c0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:57.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- 172.21.15.185:0/2897571254 shutdown_connections 2020-11-16T22:21:57.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.826+0000 7efc82d36700 1 -- 172.21.15.185:0/2897571254 wait complete. 2020-11-16T22:21:57.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.830+0000 7efc82d36700 1 Processor -- start 2020-11-16T22:21:57.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.830+0000 7efc82d36700 1 -- start start 2020-11-16T22:21:57.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 0x7efc7c13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c099690 con 0x7efc7c148550 2020-11-16T22:21:57.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c099930 con 0x7efc7c0b37e0 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c099bd0 con 0x7efc7c0b0f90 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc80ad2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34874/0 (socket says 172.21.15.185:34874) 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 -- 172.21.15.185:0/3066831133 learned_addr learned my addr 172.21.15.185:0/3066831133 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc80ad2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56548/0 (socket says 172.21.15.185:56548) 2020-11-16T22:21:57.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc812d3700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 0x7efc7c13ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 -- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 msgr2=0x7efc7c1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 -- 172.21.15.185:0/3066831133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 msgr2=0x7efc7c13ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 0x7efc7c13ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 -- 172.21.15.185:0/3066831133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c099e70 con 0x7efc7c0b37e0 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc812d3700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 0x7efc7c13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:21:57.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc80ad2700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc7bfff700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7efc6c00ef20 tx=0x7efc6c006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00cbc0 con 0x7efc7c0b37e0 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c00cd20 con 0x7efc7c0b37e0 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c145d90 con 0x7efc7c0b37e0 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c012600 con 0x7efc7c0b37e0 2020-11-16T22:21:57.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c13f210 con 0x7efc7c0b37e0 2020-11-16T22:21:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c05d4e0 con 0x7efc7c0b37e0 2020-11-16T22:21:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.834+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efc6c022070 con 0x7efc7c0b37e0 2020-11-16T22:21:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.838+0000 7efc78ff9700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 0x7efc58057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:57.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.838+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc6c06e4a0 con 0x7efc7c0b37e0 2020-11-16T22:21:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.838+0000 7efc80ad2700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 0x7efc58057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:57.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.838+0000 7efc80ad2700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 0x7efc58057520 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7efc64006e30 tx=0x7efc64015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:57.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:57.842+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efc6c044700 con 0x7efc7c0b37e0 2020-11-16T22:21:58.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.046+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc7c0b1770 con 0x7efc7c0b37e0 2020-11-16T22:21:58.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.046+0000 7efc78ff9700 1 -- 172.21.15.185:0/3066831133 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7efc6c047d20 con 0x7efc7c0b37e0 2020-11-16T22:21:58.039 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:58.039 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:58.041 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 msgr2=0x7efc58057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 0x7efc58057520 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7efc64006e30 tx=0x7efc64015000).stop 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 msgr2=0x7efc7c13e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7efc6c00ef20 tx=0x7efc6c006eb0).stop 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 shutdown_connections 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc58055060 0x7efc58057520 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc7c148550 0x7efc7c13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc7c0b0f90 0x7efc7c1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 --2- 172.21.15.185:0/3066831133 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc7c0b37e0 0x7efc7c13e830 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 >> 172.21.15.185:0/3066831133 conn(0x7efc7c0604a0 msgr2=0x7efc7c062c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:58.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 shutdown_connections 2020-11-16T22:21:58.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.050+0000 7efc82d36700 1 -- 172.21.15.185:0/3066831133 wait complete. 2020-11-16T22:21:58.044 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:21:58.059 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'a'] 2020-11-16T22:21:58.059 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:21:58.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.178+0000 7fdefcbe5700 1 Processor -- start 2020-11-16T22:21:58.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- start start 2020-11-16T22:21:58.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdef80b5c70 0x7fdef80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdef80b6c30 0x7fdef8068250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef8148670 con 0x7fdef80b1ea0 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b4d70 con 0x7fdef80b5c70 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef81488a0 con 0x7fdef80b6c30 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdef80b6c30 0x7fdef8068250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdef80b6c30 0x7fdef8068250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34884/0 (socket says 172.21.15.185:34884) 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2045300498 learned_addr learned my addr 172.21.15.185:0/2045300498 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:58.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef659c700 1 --2- 172.21.15.185:0/2045300498 >> v2:172.21.15.96:3300/0 conn(0x7fdef80b5c70 0x7fdef80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2045300498 >> v2:172.21.15.96:3300/0 conn(0x7fdef80b5c70 msgr2=0x7fdef80b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:58.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2045300498 >> v2:172.21.15.96:3300/0 conn(0x7fdef80b5c70 0x7fdef80b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2045300498 >> v1:172.21.15.96:6789/0 conn(0x7fdef80b1ea0 legacy=0x7fdef81456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2045300498 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8148b40 con 0x7fdef80b6c30 2020-11-16T22:21:58.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2045300498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b6c30 0x7fdef8068250 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fdef8080e20 tx=0x7fdeec006eb0).ready entity=mon.1 client_cookie=63ef7172ea461159 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef559a700 1 -- 172.21.15.185:0/2045300498 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec009810 con 0x7fdef80b6c30 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef559a700 1 -- 172.21.15.185:0/2045300498 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec009970 con 0x7fdef80b6c30 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdef559a700 1 -- 172.21.15.185:0/2045300498 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec009cf0 con 0x7fdef80b6c30 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2045300498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b6c30 msgr2=0x7fdef8068250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2045300498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b6c30 0x7fdef8068250 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fdef8080e20 tx=0x7fdeec006eb0).stop 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2045300498 shutdown_connections 2020-11-16T22:21:58.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2045300498 >> v2:172.21.15.96:3300/0 conn(0x7fdef80b5c70 0x7fdef80b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2045300498 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b6c30 0x7fdef8068250 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2045300498 >> 172.21.15.185:0/2045300498 conn(0x7fdef8061b20 msgr2=0x7fdef8067860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:58.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2045300498 shutdown_connections 2020-11-16T22:21:58.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.182+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2045300498 wait complete. 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 Processor -- start 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- start start 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 0x7fdef8144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdef80b6c30 0x7fdef8145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80646c0 con 0x7fdef80b6c30 2020-11-16T22:21:58.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8145580 con 0x7fdef80b1ea0 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8064180 con 0x7fdef80b5c70 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34886/0 (socket says 172.21.15.185:34886) 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef659c700 1 -- 172.21.15.185:0/2573710628 learned_addr learned my addr 172.21.15.185:0/2573710628 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 0x7fdef8144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef6d9d700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdef80b6c30 0x7fdef8145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 msgr2=0x7fdef80b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2573710628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdef80b6c30 msgr2=0x7fdef8145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdef80b6c30 0x7fdef8145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 -- 172.21.15.185:0/2573710628 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8064420 con 0x7fdef80b5c70 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef659c700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdef5d9b700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 0x7fdef8144ba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdeec012000 tx=0x7fdeec015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:58.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec011d50 con 0x7fdef80b5c70 2020-11-16T22:21:58.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec01e460 con 0x7fdef80b5c70 2020-11-16T22:21:58.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8063520 con 0x7fdef80b5c70 2020-11-16T22:21:58.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec01e7e0 con 0x7fdef80b5c70 2020-11-16T22:21:58.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.186+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef8158560 con 0x7fdef80b5c70 2020-11-16T22:21:58.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef805d4e0 con 0x7fdef80b5c70 2020-11-16T22:21:58.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdeec025030 con 0x7fdef80b5c70 2020-11-16T22:21:58.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdee67fc700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 0x7fdecc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:58.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdeec06f600 con 0x7fdef80b5c70 2020-11-16T22:21:58.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdef659c700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 0x7fdecc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:58.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.190+0000 7fdef659c700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 0x7fdecc057520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fdef814c0a0 tx=0x7fdee8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:58.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.194+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdeec045790 con 0x7fdef80b5c70 2020-11-16T22:21:58.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.402+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdef80b82f0 con 0x7fdef80b5c70 2020-11-16T22:21:58.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.402+0000 7fdee67fc700 1 -- 172.21.15.185:0/2573710628 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1791 (secure 0 0 0) 0x7fdeec023710 con 0x7fdef80b5c70 2020-11-16T22:21:58.396 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:58.396 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":491,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":110202,"bytes_used":57311232,"bytes_avail":773036802048,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6141,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":93,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":6545},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7080},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7083}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2020-11-16T22:21:55.190159+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:21:58.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 msgr2=0x7fdecc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 0x7fdecc057520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fdef814c0a0 tx=0x7fdee8006df0).stop 2020-11-16T22:21:58.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 msgr2=0x7fdef8144ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:58.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 0x7fdef8144ba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fdeec012000 tx=0x7fdeec015000).stop 2020-11-16T22:21:58.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 shutdown_connections 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdecc055060 0x7fdecc057520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdef80b1ea0 0x7fdef80b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdef80b6c30 0x7fdef8145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 --2- 172.21.15.185:0/2573710628 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdef80b5c70 0x7fdef8144ba0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 >> 172.21.15.185:0/2573710628 conn(0x7fdef8061b20 msgr2=0x7fdef814a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 shutdown_connections 2020-11-16T22:21:58.399 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:58.406+0000 7fdefcbe5700 1 -- 172.21.15.185:0/2573710628 wait complete. 2020-11-16T22:21:58.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:21:59.411 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:21:59.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 Processor -- start 2020-11-16T22:21:59.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 -- start start 2020-11-16T22:21:59.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f37e80b3b80 0x7f37e81453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f37e81458a0 0x7f37e8145d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e8060ef0 con 0x7f37e81458a0 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e8062090 con 0x7f37e80b3b80 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37ec80a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80ad640 con 0x7f37e80b3090 2020-11-16T22:21:59.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37e6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f37e80b3b80 0x7f37e81453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56572/0 (socket says 172.21.15.185:56572) 2020-11-16T22:21:59.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37e77fe700 1 -- 172.21.15.185:0/2454090186 learned_addr learned my addr 172.21.15.185:0/2454090186 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:59.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.550+0000 7f37e7fff700 1 --2- 172.21.15.185:0/2454090186 >> v2:172.21.15.96:3300/0 conn(0x7f37e81458a0 0x7f37e8145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 -- 172.21.15.185:0/2454090186 >> v2:172.21.15.185:3300/0 conn(0x7f37e80b3b80 msgr2=0x7f37e81453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 --2- 172.21.15.185:0/2454090186 >> v2:172.21.15.185:3300/0 conn(0x7f37e80b3b80 0x7f37e81453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 -- 172.21.15.185:0/2454090186 >> v2:172.21.15.96:3300/0 conn(0x7f37e81458a0 msgr2=0x7f37e8145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 --2- 172.21.15.185:0/2454090186 >> v2:172.21.15.96:3300/0 conn(0x7f37e81458a0 0x7f37e8145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 -- 172.21.15.185:0/2454090186 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80ad8e0 con 0x7f37e80b3090 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e77fe700 1 --2- 172.21.15.185:0/2454090186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f37d8006fd0 tx=0x7f37d8006eb0).ready entity=mon.2 client_cookie=9cc2fc469777adde server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e67fc700 1 -- 172.21.15.185:0/2454090186 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d80096e0 con 0x7f37e80b3090 2020-11-16T22:21:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e67fc700 1 -- 172.21.15.185:0/2454090186 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8009840 con 0x7f37e80b3090 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37e67fc700 1 -- 172.21.15.185:0/2454090186 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8009bc0 con 0x7f37e80b3090 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 -- 172.21.15.185:0/2454090186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 msgr2=0x7f37e80b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 --2- 172.21.15.185:0/2454090186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f37d8006fd0 tx=0x7f37d8006eb0).stop 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 -- 172.21.15.185:0/2454090186 shutdown_connections 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 --2- 172.21.15.185:0/2454090186 >> v2:172.21.15.96:3300/0 conn(0x7f37e81458a0 0x7f37e8145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 --2- 172.21.15.185:0/2454090186 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e80b1840 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 --2- 172.21.15.185:0/2454090186 >> v2:172.21.15.185:3300/0 conn(0x7f37e80b3b80 0x7f37e81453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 -- 172.21.15.185:0/2454090186 >> 172.21.15.185:0/2454090186 conn(0x7f37e8060130 msgr2=0x7f37e8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:21:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 -- 172.21.15.185:0/2454090186 shutdown_connections 2020-11-16T22:21:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.554+0000 7f37ec80a700 1 -- 172.21.15.185:0/2454090186 wait complete. 2020-11-16T22:21:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 Processor -- start 2020-11-16T22:21:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- start start 2020-11-16T22:21:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f37e80b3b80 0x7f37e810fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37e81458a0 0x7f37e8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b5b00 con 0x7f37e81458a0 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b5d50 con 0x7f37e80b3b80 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b5f80 con 0x7f37e80b3090 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56576/0 (socket says 172.21.15.185:56576) 2020-11-16T22:21:59.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 -- 172.21.15.185:0/1640737551 learned_addr learned my addr 172.21.15.185:0/1640737551 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:21:59.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e6ffd700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f37e80b3b80 0x7f37e810fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e7fff700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37e81458a0 0x7f37e8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 -- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f37e80b3b80 msgr2=0x7f37e810fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:59.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f37e80b3b80 0x7f37e810fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 -- 172.21.15.185:0/1640737551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37e81458a0 msgr2=0x7f37e8154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:59.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37e81458a0 0x7f37e8154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 -- 172.21.15.185:0/1640737551 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80b6220 con 0x7f37e80b3090 2020-11-16T22:21:59.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37e77fe700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f37e80ad780 tx=0x7f37d8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:59.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8011b40 con 0x7f37e80b3090 2020-11-16T22:21:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8011ca0 con 0x7f37e80b3090 2020-11-16T22:21:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d801e690 con 0x7f37e80b3090 2020-11-16T22:21:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e8142810 con 0x7f37e80b3090 2020-11-16T22:21:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.558+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e80b6d60 con 0x7f37e80b3090 2020-11-16T22:21:59.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f37d8025030 con 0x7f37e80b3090 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e80ae7a0 con 0x7f37e80b3090 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37c77fe700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 0x7f37c0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f37d806f280 con 0x7f37e80b3090 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37e6ffd700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 0x7f37c0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.562+0000 7f37e6ffd700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 0x7f37c0057520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f37e8111280 tx=0x7f37dc006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:21:59.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.570+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f37e80ae7a0 con 0x7f37e80b3090 2020-11-16T22:21:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.770+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37e80ae7a0 con 0x7f37e80b3090 2020-11-16T22:21:59.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37c77fe700 1 -- 172.21.15.185:0/1640737551 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7f37e80ae7a0 con 0x7f37e80b3090 2020-11-16T22:21:59.764 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:21:59.764 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:21:59.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 msgr2=0x7f37c0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:59.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 0x7f37c0057520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f37e8111280 tx=0x7f37dc006df0).stop 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 msgr2=0x7f37e8142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f37e80ad780 tx=0x7f37d8015000).stop 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 shutdown_connections 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f37c0055060 0x7f37c0057520 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37e81458a0 0x7f37e8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f37e80b3090 0x7f37e8142320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:21:59.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 --2- 172.21.15.185:0/1640737551 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f37e80b3b80 0x7f37e810fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 >> 172.21.15.185:0/1640737551 conn(0x7f37e8060130 msgr2=0x7f37e80acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:00.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 shutdown_connections 2020-11-16T22:22:00.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:21:59.774+0000 7f37ec80a700 1 -- 172.21.15.185:0/1640737551 wait complete. 2020-11-16T22:22:00.171 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:00.171 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'a'] 2020-11-16T22:22:00.172 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:22:00.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 Processor -- start 2020-11-16T22:22:00.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- start start 2020-11-16T22:22:00.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 0x7fa0f8147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f8143e90 con 0x7fa0f80bb120 2020-11-16T22:22:00.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f80afd50 con 0x7fa0f80b58c0 2020-11-16T22:22:00.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80afff0 con 0x7fa0f80af7d0 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fd05c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 0x7fa0f8147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe05e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa0f80b58c0 0x7fa0f8142de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56582/0 (socket says 172.21.15.185:56582) 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe05e700 1 -- 172.21.15.185:0/2836764868 learned_addr learned my addr 172.21.15.185:0/2836764868 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fd05c700 1 -- 172.21.15.185:0/2836764868 >> v1:172.21.15.185:6789/0 conn(0x7fa0f80b58c0 legacy=0x7fa0f8142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fd05c700 1 -- 172.21.15.185:0/2836764868 >> v1:172.21.15.96:6789/0 conn(0x7fa0f80bb120 legacy=0x7fa0f8147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fd05c700 1 -- 172.21.15.185:0/2836764868 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80b0290 con 0x7fa0f80af7d0 2020-11-16T22:22:00.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fd05c700 1 --2- 172.21.15.185:0/2836764868 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 0x7fa0f8147ea0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fa0e800b9c0 tx=0x7fa0e8006d90).ready entity=mon.2 client_cookie=6ac96c09f0840eac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fc85b700 1 -- 172.21.15.185:0/2836764868 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8006030 con 0x7fa0f80af7d0 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fc85b700 1 -- 172.21.15.185:0/2836764868 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8006190 con 0x7fa0f80af7d0 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fc85b700 1 -- 172.21.15.185:0/2836764868 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8006510 con 0x7fa0f80af7d0 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- 172.21.15.185:0/2836764868 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 msgr2=0x7fa0f8147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/2836764868 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 0x7fa0f8147ea0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fa0e800b9c0 tx=0x7fa0e8006d90).stop 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- 172.21.15.185:0/2836764868 shutdown_connections 2020-11-16T22:22:00.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/2836764868 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80af7d0 0x7fa0f8147ea0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- 172.21.15.185:0/2836764868 >> 172.21.15.185:0/2836764868 conn(0x7fa0f8068330 msgr2=0x7fa0f80ba870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:00.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- 172.21.15.185:0/2836764868 shutdown_connections 2020-11-16T22:22:00.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.310+0000 7fa0fe85f700 1 -- 172.21.15.185:0/2836764868 wait complete. 2020-11-16T22:22:00.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 Processor -- start 2020-11-16T22:22:00.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 -- start start 2020-11-16T22:22:00.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0f80b58c0 0x7fa0f80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 0x7fa0f81488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8140900 con 0x7fa0f80b58c0 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80b4c60 con 0x7fa0f80af7d0 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe85f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f809cec0 con 0x7fa0f80bb120 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fd85d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 0x7fa0f81488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fd85d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34914/0 (socket says 172.21.15.185:34914) 2020-11-16T22:22:00.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fd85d700 1 -- 172.21.15.185:0/973917244 learned_addr learned my addr 172.21.15.185:0/973917244 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:00.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fd05c700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0f80b58c0 0x7fa0f80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 -- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 msgr2=0x7fa0f80b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 -- 172.21.15.185:0/973917244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0f80b58c0 msgr2=0x7fa0f80b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0f80b58c0 0x7fa0f80b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 -- 172.21.15.185:0/973917244 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f809d160 con 0x7fa0f80bb120 2020-11-16T22:22:00.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fd85d700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:00.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0fe05e700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 0x7fa0f81488b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fa0f400c710 tx=0x7fa0f4007aa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:00.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40072e0 con 0x7fa0f80bb120 2020-11-16T22:22:00.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.314+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f4007440 con 0x7fa0f80bb120 2020-11-16T22:22:00.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f809d460 con 0x7fa0f80bb120 2020-11-16T22:22:00.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4005800 con 0x7fa0f80bb120 2020-11-16T22:22:00.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f80b4240 con 0x7fa0f80bb120 2020-11-16T22:22:00.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f80b30f0 con 0x7fa0f80bb120 2020-11-16T22:22:00.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa0f401a060 con 0x7fa0f80bb120 2020-11-16T22:22:00.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0e5ffb700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 0x7fa0d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0fd85d700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 0x7fa0d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0f406cdb0 con 0x7fa0f80bb120 2020-11-16T22:22:00.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.318+0000 7fa0fd85d700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 0x7fa0d0057520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa0f814a6c0 tx=0x7fa0f0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:00.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.326+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa0f4043010 con 0x7fa0f80bb120 2020-11-16T22:22:00.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa0f805d200 con 0x7fa0f80bb120 2020-11-16T22:22:00.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0e5ffb700 1 -- 172.21.15.185:0/973917244 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1832 (secure 0 0 0) 0x7fa0f4018020 con 0x7fa0f80bb120 2020-11-16T22:22:00.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 msgr2=0x7fa0d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 0x7fa0d0057520 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa0f814a6c0 tx=0x7fa0f0009040).stop 2020-11-16T22:22:00.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 msgr2=0x7fa0f81488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:00.718 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 0x7fa0f81488b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fa0f400c710 tx=0x7fa0f4007aa0).stop 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 shutdown_connections 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa0d0055060 0x7fa0d0057520 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa0f80af7d0 0x7fa0f80b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa0f80bb120 0x7fa0f81488b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 --2- 172.21.15.185:0/973917244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0f80b58c0 0x7fa0f80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 >> 172.21.15.185:0/973917244 conn(0x7fa0f8068330 msgr2=0x7fa0f8146800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:00.719 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 shutdown_connections 2020-11-16T22:22:00.720 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.514+0000 7fa0fe85f700 1 -- 172.21.15.185:0/973917244 wait complete. 2020-11-16T22:22:00.720 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:00.720 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":110202,"bytes_used":57311232,"bytes_avail":773036802048,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6141,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":93,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":6545},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7080},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7083}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2020-11-16T22:21:59.191437+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2020-11-16T22:22:00.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2020-11-16T22:22:00.722 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:00.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 Processor -- start 2020-11-16T22:22:00.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 -- start start 2020-11-16T22:22:00.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 0x7fa5fc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa5fc099d80 0x7fa5fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:00.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc068060 con 0x7fa5fc0b2760 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc067260 con 0x7fa5fc099d80 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa6024ab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc067ce0 con 0x7fa5fc0b4140 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa5fb7fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa5fc099d80 0x7fa5fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa600a48700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fa5fc0b2760 0x7fa5fc063c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56596/0 (socket says 172.21.15.185:56596) 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa600a48700 1 -- 172.21.15.185:0/470918224 learned_addr learned my addr 172.21.15.185:0/470918224 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:00.853 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.858+0000 7fa5fbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 0x7fa5fc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:01.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5faffd700 1 -- 172.21.15.185:0/470918224 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366252048 0 0) 0x7fa5fc068060 con 0x7fa5fc0b2760 2020-11-16T22:22:01.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5faffd700 1 -- 172.21.15.185:0/470918224 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5e8003240 con 0x7fa5fc0b2760 2020-11-16T22:22:01.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5fbfff700 1 -- 172.21.15.185:0/470918224 >> v2:172.21.15.185:3300/0 conn(0x7fa5fc099d80 msgr2=0x7fa5fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:01.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5fbfff700 1 --2- 172.21.15.185:0/470918224 >> v2:172.21.15.185:3300/0 conn(0x7fa5fc099d80 0x7fa5fc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5fbfff700 1 -- 172.21.15.185:0/470918224 >> v1:172.21.15.185:6789/0 conn(0x7fa5fc0b2760 legacy=0x7fa5fc063c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5fbfff700 1 -- 172.21.15.185:0/470918224 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc066de0 con 0x7fa5fc0b4140 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5fbfff700 1 --2- 172.21.15.185:0/470918224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 0x7fa5fc0b4520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fa5ec009c90 tx=0x7fa5ec0091d0).ready entity=mon.2 client_cookie=3afb41379f04d92 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5faffd700 1 -- 172.21.15.185:0/470918224 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00b9c0 con 0x7fa5fc0b4140 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5faffd700 1 -- 172.21.15.185:0/470918224 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec00bb20 con 0x7fa5fc0b4140 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa5faffd700 1 -- 172.21.15.185:0/470918224 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00bea0 con 0x7fa5fc0b4140 2020-11-16T22:22:01.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- 172.21.15.185:0/470918224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 msgr2=0x7fa5fc0b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 --2- 172.21.15.185:0/470918224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 0x7fa5fc0b4520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fa5ec009c90 tx=0x7fa5ec0091d0).stop 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- 172.21.15.185:0/470918224 shutdown_connections 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 --2- 172.21.15.185:0/470918224 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc0b4140 0x7fa5fc0b4520 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 --2- 172.21.15.185:0/470918224 >> v2:172.21.15.185:3300/0 conn(0x7fa5fc099d80 0x7fa5fc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- 172.21.15.185:0/470918224 >> 172.21.15.185:0/470918224 conn(0x7fa5fc061240 msgr2=0x7fa5fc061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- 172.21.15.185:0/470918224 shutdown_connections 2020-11-16T22:22:01.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- 172.21.15.185:0/470918224 wait complete. 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 Processor -- start 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.862+0000 7fa6024ab700 1 -- start start 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 0x7fa5fc153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5fc0b4140 0x7fa5fc157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc1542e0 con 0x7fa5fc0b4140 2020-11-16T22:22:01.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc093600 con 0x7fa5fc0b2760 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc156920 con 0x7fa5fc099d80 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 0x7fa5fc153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34930/0 (socket says 172.21.15.185:34930) 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fbfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 0x7fa5fc153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56604/0 (socket says 172.21.15.185:56604) 2020-11-16T22:22:01.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 -- 172.21.15.185:0/1802599711 learned_addr learned my addr 172.21.15.185:0/1802599711 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:01.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa600a48700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5fc0b4140 0x7fa5fc157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:01.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 -- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 msgr2=0x7fa5fc153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 0x7fa5fc153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 -- 172.21.15.185:0/1802599711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5fc0b4140 msgr2=0x7fa5fc157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5fc0b4140 0x7fa5fc157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 -- 172.21.15.185:0/1802599711 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc08f5c0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fb7fe700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fa5f000eba0 tx=0x7fa5f0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000c8e0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000ca40 con 0x7fa5fc0b2760 2020-11-16T22:22:01.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc156be0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000cdc0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc08f240 con 0x7fa5fc0b2760 2020-11-16T22:22:01.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc05d4e0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa5f001a030 con 0x7fa5fc0b2760 2020-11-16T22:22:01.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 0x7fa5d4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:01.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5f006d8e0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.866+0000 7fa5fbfff700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 0x7fa5d4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:01.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.870+0000 7fa5fbfff700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 0x7fa5d4057520 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fa5ec006ee0 tx=0x7fa5ec016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:01.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:00.874+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa5f0043bf0 con 0x7fa5fc0b2760 2020-11-16T22:22:01.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5fc08f750 con 0x7fa5fc0b2760 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa5dbfff700 1 -- 172.21.15.185:0/1802599711 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7fa5f0047210 con 0x7fa5fc0b2760 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 msgr2=0x7fa5d4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 0x7fa5d4057520 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fa5ec006ee0 tx=0x7fa5ec016000).stop 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 msgr2=0x7fa5fc142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fa5f000eba0 tx=0x7fa5f0006df0).stop 2020-11-16T22:22:01.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 shutdown_connections 2020-11-16T22:22:01.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa5d4055060 0x7fa5d4057520 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5fc0b4140 0x7fa5fc157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa5fc099d80 0x7fa5fc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 --2- 172.21.15.185:0/1802599711 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa5fc0b2760 0x7fa5fc142fd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 >> 172.21.15.185:0/1802599711 conn(0x7fa5fc061240 msgr2=0x7fa5fc0b7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 shutdown_connections 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:01.074+0000 7fa6024ab700 1 -- 172.21.15.185:0/1802599711 wait complete. 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:01.675 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:01.878 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub resume 2020-11-16T22:22:02.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 Processor -- start 2020-11-16T22:22:02.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- start start 2020-11-16T22:22:02.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3bd00674d0 0x7f3bd00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.018 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3bd0068050 0x7f3bd013da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0141a50 con 0x7f3bd0066890 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00b8430 con 0x7f3bd00674d0 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd0141c80 con 0x7f3bd0068050 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bceffd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3bd0066890 0x7f3bd00b7d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56606/0 (socket says 172.21.15.185:56606) 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bceffd700 1 -- 172.21.15.185:0/3232280216 learned_addr learned my addr 172.21.15.185:0/3232280216 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/3232280216 >> v2:172.21.15.185:3300/0 conn(0x7f3bd0068050 0x7f3bd013da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/3232280216 >> v2:172.21.15.96:3300/0 conn(0x7f3bd00674d0 0x7f3bd00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcdffb700 1 -- 172.21.15.185:0/3232280216 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076333026 0 0) 0x7f3bd0141a50 con 0x7f3bd0066890 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcdffb700 1 -- 172.21.15.185:0/3232280216 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bbc003240 con 0x7f3bd0066890 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/3232280216 >> v2:172.21.15.96:3300/0 conn(0x7f3bd00674d0 msgr2=0x7f3bd00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/3232280216 >> v2:172.21.15.96:3300/0 conn(0x7f3bd00674d0 0x7f3bd00b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/3232280216 >> v1:172.21.15.185:6789/0 conn(0x7f3bd0066890 legacy=0x7f3bd00b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:02.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/3232280216 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0141f20 con 0x7f3bd0068050 2020-11-16T22:22:02.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/3232280216 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd0068050 0x7f3bd013da80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f3bc8009c90 tx=0x7f3bc80091d0).ready entity=mon.1 client_cookie=9922946975f34b9e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:02.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcdffb700 1 -- 172.21.15.185:0/3232280216 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800ba10 con 0x7f3bd0068050 2020-11-16T22:22:02.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcdffb700 1 -- 172.21.15.185:0/3232280216 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800bb70 con 0x7f3bd0068050 2020-11-16T22:22:02.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcdffb700 1 -- 172.21.15.185:0/3232280216 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8010470 con 0x7f3bd0068050 2020-11-16T22:22:02.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- 172.21.15.185:0/3232280216 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd0068050 msgr2=0x7f3bd013da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:02.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3232280216 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd0068050 0x7f3bd013da80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f3bc8009c90 tx=0x7f3bc80091d0).stop 2020-11-16T22:22:02.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- 172.21.15.185:0/3232280216 shutdown_connections 2020-11-16T22:22:02.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3232280216 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd0068050 0x7f3bd013da80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3232280216 >> v2:172.21.15.96:3300/0 conn(0x7f3bd00674d0 0x7f3bd00b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- 172.21.15.185:0/3232280216 >> 172.21.15.185:0/3232280216 conn(0x7f3bd005ba80 msgr2=0x7f3bd005be90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- 172.21.15.185:0/3232280216 shutdown_connections 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.026+0000 7f3bcffff700 1 -- 172.21.15.185:0/3232280216 wait complete. 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 Processor -- start 2020-11-16T22:22:02.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 -- start start 2020-11-16T22:22:02.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd0066890 0x7f3bd00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 0x7f3bd0144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd013ab80 con 0x7f3bd0066890 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd014fa70 con 0x7f3bd00674d0 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00ad630 con 0x7f3bd0068050 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 0x7f3bd0144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 0x7f3bd0144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56620/0 (socket says 172.21.15.185:56620) 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/3797815446 learned_addr learned my addr 172.21.15.185:0/3797815446 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:02.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34946/0 (socket says 172.21.15.185:34946) 2020-11-16T22:22:02.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bceffd700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd0066890 0x7f3bd00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 -- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 msgr2=0x7f3bd0144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:02.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 0x7f3bd0144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 -- 172.21.15.185:0/3797815446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd0066890 msgr2=0x7f3bd00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:02.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd0066890 0x7f3bd00b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 -- 172.21.15.185:0/3797815446 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00ae910 con 0x7f3bd00674d0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f3bd0067980 tx=0x7f3bc4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bb6ffd700 1 -- 172.21.15.185:0/3797815446 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400c820 con 0x7f3bd00674d0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bb6ffd700 1 -- 172.21.15.185:0/3797815446 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc400c980 con 0x7f3bd00674d0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00ad390 con 0x7f3bd00674d0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.030+0000 7f3bb6ffd700 1 -- 172.21.15.185:0/3797815446 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400cd00 con 0x7f3bd00674d0 2020-11-16T22:22:02.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd00b16b0 con 0x7f3bd00674d0 2020-11-16T22:22:02.241 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:22:02.065+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:22:02.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bb6ffd700 1 -- 172.21.15.185:0/3797815446 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3bc401a030 con 0x7f3bd00674d0 2020-11-16T22:22:02.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bb6ffd700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 0x7f3bac057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bceffd700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 0x7f3bac057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bb6ffd700 1 -- 172.21.15.185:0/3797815446 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bc406d790 con 0x7f3bd00674d0 2020-11-16T22:22:02.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.034+0000 7f3bceffd700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 0x7f3bac057550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f3bc00055a0 tx=0x7f3bc0009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:02.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.038+0000 7f3be3ee7740 1 Processor -- start 2020-11-16T22:22:02.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.038+0000 7f3be3ee7740 1 -- start start 2020-11-16T22:22:02.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e174b0 con 0x1cffbb0 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ecb930 con 0x1f8bda0 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d8b6f0 con 0x1d6d7b0 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56628/0 (socket says 172.21.15.185:56628) 2020-11-16T22:22:02.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34950/0 (socket says 172.21.15.185:34950) 2020-11-16T22:22:02.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bceffd700 1 -- 172.21.15.185:0/2089938410 learned_addr learned my addr 172.21.15.185:0/2089938410 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:02.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 msgr2=0x1f8b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:02.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/2089938410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 msgr2=0x1f89e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:02.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f89e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 -- 172.21.15.185:0/2089938410 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d865a0 con 0x1f8bda0 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bceffd700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f89e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f3bd00b7510 tx=0x7f3bc8018000).ready entity=mon.1 client_cookie=cdc11e03cc321cb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3ba7fff700 1 -- 172.21.15.185:0/2089938410 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc801f030 con 0x1f8bda0 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3ba7fff700 1 -- 172.21.15.185:0/2089938410 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8020040 con 0x1f8bda0 2020-11-16T22:22:02.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3ba7fff700 1 -- 172.21.15.185:0/2089938410 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8010470 con 0x1f8bda0 2020-11-16T22:22:02.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 msgr2=0x1f8a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:02.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f3bd00b7510 tx=0x7f3bc8018000).stop 2020-11-16T22:22:02.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- 172.21.15.185:0/2089938410 shutdown_connections 2020-11-16T22:22:02.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f8bda0 0x1f8a380 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1f8b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/2089938410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f89e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:02.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- 172.21.15.185:0/2089938410 >> 172.21.15.185:0/2089938410 conn(0x1f3be40 msgr2=0x1f88dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- 172.21.15.185:0/2089938410 shutdown_connections 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- 172.21.15.185:0/2089938410 wait complete. 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 -- start start 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.042+0000 7f3be3ee7740 1 Processor -- start 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1f8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 0x1fba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eb86e0 con 0x1f8bda0 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eb8820 con 0x1cffbb0 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eb9a90 con 0x1d6d7b0 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 10 client.0 ll_register_callbacks cb 0x1eaead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1f8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:02.857 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56634/0 (socket says 172.21.15.185:56634) 2020-11-16T22:22:03.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1f8a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34956/0 (socket says 172.21.15.185:34956) 2020-11-16T22:22:03.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 -- 172.21.15.185:0/214259236 learned_addr learned my addr 172.21.15.185:0/214259236 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:03.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 0x1fba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 -- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 msgr2=0x1f8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1f8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 -- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 msgr2=0x1fba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 0x1fba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 -- 172.21.15.185:0/214259236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ea9280 con 0x1d6d7b0 2020-11-16T22:22:03.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 0x1fba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:03.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3bc4047690 tx=0x7f3bc406d040).ready entity=mon.2 client_cookie=bde64d41be1deb0f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:03.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:03.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4043760 con 0x1d6d7b0 2020-11-16T22:22:03.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bc40466c0 con 0x1d6d7b0 2020-11-16T22:22:03.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f6c6c0 con 0x1d6d7b0 2020-11-16T22:22:03.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bc4046a90 con 0x1d6d7b0 2020-11-16T22:22:03.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 10 client.6947 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:03.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x1cdefd0 con 0x1d6d7b0 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f6c6c0 con 0x1d6d7b0 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3bc4045dc0 con 0x1d6d7b0 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 10 client.6947 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 10 client.6947 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 20 client.6947 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:03.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1fbf140 0x1fbf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 4 client.6947 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1da9c50 con 0x1fbf140 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.046+0000 7f3bceffd700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1fbf140 0x1fbf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3ba57fa700 10 client.6947 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3bceffd700 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1fbf140 0x1fbf520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fb3293fe7e5dd6de server_cookie=cb66286012342ce0 in_seq=0 out_seq=0 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3ba57fa700 1 -- 172.21.15.185:0/214259236 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1da9c50 con 0x1fbf140 2020-11-16T22:22:03.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3ba57fa700 10 client.6947 handle_command_reply: tid=0 2020-11-16T22:22:03.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 client.6947 shutdown 2020-11-16T22:22:03.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 msgr2=0x1fb9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:03.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3bc4047690 tx=0x7f3bc406d040).stop 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 shutdown_connections 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1fbf140 0x1fbf520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1f8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1f8bda0 0x1fba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3ba57fa700 0 client.6947 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:03.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/214259236 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fb9b60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 >> 172.21.15.185:0/214259236 conn(0x1f3be40 msgr2=0x1fbb310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 shutdown_connections 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 1 -- 172.21.15.185:0/214259236 wait complete. 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.050+0000 7f3be3ee7740 20 client.6947 trim_cache size 0 max 16384 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 Processor -- start 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 -- start start 2020-11-16T22:22:03.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 0x1fc03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d6d7b0 0x1fcfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e90e40 con 0x1d6d7b0 2020-11-16T22:22:03.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd09c0 con 0x1cffbb0 2020-11-16T22:22:03.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f8a550 con 0x1f3be40 2020-11-16T22:22:03.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34968/0 (socket says 172.21.15.185:34968) 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bce7fc700 1 -- 172.21.15.185:0/404215526 learned_addr learned my addr 172.21.15.185:0/404215526 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 0x1fc03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d6d7b0 0x1fcfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 -- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 msgr2=0x1fcf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:03.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 -- 172.21.15.185:0/404215526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d6d7b0 msgr2=0x1fcfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:03.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d6d7b0 0x1fcfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 -- 172.21.15.185:0/404215526 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c008160 con 0x1f3be40 2020-11-16T22:22:03.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:03.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.070+0000 7f3bceffd700 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 0x1fc03d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x1fc1e30 tx=0x7f3bc000a000).ready entity=mon.2 client_cookie=b0a78f354eba885c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:03.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bb4ff9700 1 -- 172.21.15.185:0/404215526 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc001a040 con 0x1f3be40 2020-11-16T22:22:03.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bb4ff9700 1 -- 172.21.15.185:0/404215526 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc00206d0 con 0x1f3be40 2020-11-16T22:22:03.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bb4ff9700 1 -- 172.21.15.185:0/404215526 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0020a50 con 0x1f3be40 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 msgr2=0x1fc03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 0x1fc03d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x1fc1e30 tx=0x7f3bc000a000).stop 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- 172.21.15.185:0/404215526 shutdown_connections 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1d6d7b0 0x1fcfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1f3be40 0x1fc03d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/404215526 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1cffbb0 0x1fcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- 172.21.15.185:0/404215526 >> 172.21.15.185:0/404215526 conn(0x7f3b9c002320 msgr2=0x1fd34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- 172.21.15.185:0/404215526 shutdown_connections 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- 172.21.15.185:0/404215526 wait complete. 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- start start 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 Processor -- start 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:03.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 0x20257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f888e0 con 0x1cffbb0 2020-11-16T22:22:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fbef30 con 0x1f3be40 2020-11-16T22:22:04.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f8ddb0 con 0x1d6d7b0 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3be3ee7740 10 client.0 ll_register_callbacks cb 0x1f895b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56646/0 (socket says 172.21.15.185:56646) 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 -- 172.21.15.185:0/1953186715 learned_addr learned my addr 172.21.15.185:0/1953186715 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 0x20257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:04.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bceffd700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 -- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 msgr2=0x20257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 0x20257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 -- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 msgr2=0x1f8b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f8b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 -- 172.21.15.185:0/1953186715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d1d720 con 0x1d6d7b0 2020-11-16T22:22:04.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bcf7fe700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 0x20257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:04.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bceffd700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:04.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3bce7fc700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3bc4047690 tx=0x7f3bc40471f0).ready entity=mon.2 client_cookie=d9e7988c00052c79 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:04.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.074+0000 7f3ba57fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4046a90 con 0x1d6d7b0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bc40466c0 con 0x1d6d7b0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bc8020040 con 0x1d6d7b0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bc4043fd0 con 0x1d6d7b0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 10 client.6953 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x1d28e00 con 0x1d6d7b0 2020-11-16T22:22:04.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8020040 con 0x1d6d7b0 2020-11-16T22:22:04.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f3bc4045dc0 con 0x1d6d7b0 2020-11-16T22:22:04.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 10 client.6953 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:04.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 10 client.6953 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:22:04.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 20 client.6953 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:04.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1f8bda0 0x2025cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:04.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 4 client.6953 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,r,e,s,u,m,e,",}] 2020-11-16T22:22:04.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " s c r u b r e s u m e " }) v1 -- 0x1db0f50 con 0x1f8bda0 2020-11-16T22:22:04.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3bceffd700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1f8bda0 0x2025cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 10 client.6953 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3bceffd700 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1f8bda0 0x2025cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3ee5f90b5ceba3a4 server_cookie=669f969ed43656d3 in_seq=0 out_seq=0 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 1 -- 172.21.15.185:0/1953186715 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x1db0f50 con 0x1f8bda0 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 10 client.6953 handle_command_reply: tid=0 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 client.6953 shutdown 2020-11-16T22:22:04.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 msgr2=0x1fc8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f3bc4047690 tx=0x7f3bc40471f0).stop 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 shutdown_connections 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1f8bda0 0x2025cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1f3be40 0x20257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3ba57fa700 0 client.6953 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1cffbb0 0x1f8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 --2- 172.21.15.185:0/1953186715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1d6d7b0 0x1fc8980 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 >> 172.21.15.185:0/1953186715 conn(0x7f3b9c002320 msgr2=0x1f85700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:04.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 shutdown_connections 2020-11-16T22:22:04.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 1 -- 172.21.15.185:0/1953186715 wait complete. 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3be3ee7740 20 client.6953 trim_cache size 0 max 16384 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.078+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 msgr2=0x7f3bac057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 0x7f3bac057550 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f3bc00055a0 tx=0x7f3bc0009040).stop 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 msgr2=0x7f3bd0144a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f3bd0067980 tx=0x7f3bc4006df0).stop 2020-11-16T22:22:04.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 shutdown_connections 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3bac055090 0x7f3bac057550 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3bd0068050 0x7f3bd0144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bd0066890 0x7f3bd00b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 --2- 172.21.15.185:0/3797815446 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3bd00674d0 0x7f3bd0144a20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 >> 172.21.15.185:0/3797815446 conn(0x7f3bd005ba80 msgr2=0x7f3bd00b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:04.696 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 shutdown_connections 2020-11-16T22:22:04.697 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:02.082+0000 7f3bcffff700 1 -- 172.21.15.185:0/3797815446 wait complete. 2020-11-16T22:22:04.697 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:22:04.697 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:22:04.898 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:22:04.899 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:05.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.038+0000 7ff804b04700 1 Processor -- start 2020-11-16T22:22:05.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.038+0000 7ff804b04700 1 -- start start 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.038+0000 7ff804b04700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff800143a70 0x7ff800143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff800069140 0x7ff80013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff800143640 con 0x7ff8000b7200 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff800068790 con 0x7ff800069140 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000b1ce0 con 0x7ff800143a70 2020-11-16T22:22:05.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff800143a70 0x7ff800143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7ff7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff8000b7200 0x7ff8000b15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56650/0 (socket says 172.21.15.185:56650) 2020-11-16T22:22:05.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7ff7fe700 1 -- 172.21.15.185:0/3473824395 learned_addr learned my addr 172.21.15.185:0/3473824395 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:05.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7fffff700 1 --2- 172.21.15.185:0/3473824395 >> v2:172.21.15.96:3300/0 conn(0x7ff800069140 0x7ff80013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 -- 172.21.15.185:0/3473824395 >> v2:172.21.15.96:3300/0 conn(0x7ff800069140 msgr2=0x7ff80013fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:05.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 --2- 172.21.15.185:0/3473824395 >> v2:172.21.15.96:3300/0 conn(0x7ff800069140 0x7ff80013fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 -- 172.21.15.185:0/3473824395 >> v1:172.21.15.185:6789/0 conn(0x7ff8000b7200 legacy=0x7ff8000b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7fe7fc700 1 -- 172.21.15.185:0/3473824395 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325492339 0 0) 0x7ff800143640 con 0x7ff8000b7200 2020-11-16T22:22:05.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 -- 172.21.15.185:0/3473824395 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000668b0 con 0x7ff800143a70 2020-11-16T22:22:05.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7feffd700 1 --2- 172.21.15.185:0/3473824395 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800143a70 0x7ff800143ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff8000666e0 tx=0x7ff7e8006e80).ready entity=mon.1 client_cookie=b95de930237c04da server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:05.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7fe7fc700 1 -- 172.21.15.185:0/3473824395 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8014070 con 0x7ff800143a70 2020-11-16T22:22:05.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7fe7fc700 1 -- 172.21.15.185:0/3473824395 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800a770 con 0x7ff800143a70 2020-11-16T22:22:05.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff7fe7fc700 1 -- 172.21.15.185:0/3473824395 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800aaf0 con 0x7ff800143a70 2020-11-16T22:22:05.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- 172.21.15.185:0/3473824395 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800143a70 msgr2=0x7ff800143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 --2- 172.21.15.185:0/3473824395 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800143a70 0x7ff800143ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff8000666e0 tx=0x7ff7e8006e80).stop 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- 172.21.15.185:0/3473824395 shutdown_connections 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 --2- 172.21.15.185:0/3473824395 >> v2:172.21.15.96:3300/0 conn(0x7ff800069140 0x7ff80013fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 --2- 172.21.15.185:0/3473824395 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800143a70 0x7ff800143ed0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- 172.21.15.185:0/3473824395 >> 172.21.15.185:0/3473824395 conn(0x7ff800060c00 msgr2=0x7ff8000b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- 172.21.15.185:0/3473824395 shutdown_connections 2020-11-16T22:22:05.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.042+0000 7ff804b04700 1 -- 172.21.15.185:0/3473824395 wait complete. 2020-11-16T22:22:05.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 Processor -- start 2020-11-16T22:22:05.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 -- start start 2020-11-16T22:22:05.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 0x7ff80013c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8000b7200 0x7ff800139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 0x7ff8001394f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000ada00 con 0x7ff8000b7200 2020-11-16T22:22:05.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000afe30 con 0x7ff800069140 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff804b04700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000af650 con 0x7ff800143a70 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff7ff7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 0x7ff80013c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff7ff7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 0x7ff80013c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34984/0 (socket says 172.21.15.185:34984) 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff7ff7fe700 1 -- 172.21.15.185:0/1453606241 learned_addr learned my addr 172.21.15.185:0/1453606241 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff7fffff700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 0x7ff8001394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.046+0000 7ff7feffd700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8000b7200 0x7ff800139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 -- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 msgr2=0x7ff80013c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 0x7ff80013c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 -- 172.21.15.185:0/1453606241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8000b7200 msgr2=0x7ff800139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8000b7200 0x7ff800139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 -- 172.21.15.185:0/1453606241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000af8f0 con 0x7ff800143a70 2020-11-16T22:22:05.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7fffff700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 0x7ff8001394f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7ff7f800ed50 tx=0x7ff7f80091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:05.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800cb00 con 0x7ff800143a70 2020-11-16T22:22:05.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800cc80 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8000afbf0 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80123f0 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8001399e0 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff7f8022070 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff800069e40 con 0x7ff800143a70 2020-11-16T22:22:05.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 0x7ff7d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:05.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7ff7fe700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 0x7ff7d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:05.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff7f806e270 con 0x7ff800143a70 2020-11-16T22:22:05.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.050+0000 7ff7ff7fe700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 0x7ff7d0057520 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7ff7f0007260 tx=0x7ff7f0007ac0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:05.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.058+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff800069e40 con 0x7ff800143a70 2020-11-16T22:22:05.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.254+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff800151400 con 0x7ff800143a70 2020-11-16T22:22:05.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.254+0000 7ff7f77fe700 1 -- 172.21.15.185:0/1453606241 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7ff800151400 con 0x7ff800143a70 2020-11-16T22:22:05.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 msgr2=0x7ff7d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.862 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 0x7ff7d0057520 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7ff7f0007260 tx=0x7ff7f0007ac0).stop 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 msgr2=0x7ff8001394f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 0x7ff8001394f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7ff7f800ed50 tx=0x7ff7f80091d0).stop 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 shutdown_connections 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff7d0055060 0x7ff7d0057520 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff800069140 0x7ff80013c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.863 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff800143a70 0x7ff8001394f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 --2- 172.21.15.185:0/1453606241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8000b7200 0x7ff800139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 >> 172.21.15.185:0/1453606241 conn(0x7ff800060c00 msgr2=0x7ff80013de80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 shutdown_connections 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:05.258+0000 7ff804b04700 1 -- 172.21.15.185:0/1453606241 wait complete. 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:05.864 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:05.865 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:06.066 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:22:06.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 Processor -- start 2020-11-16T22:22:06.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 -- start start 2020-11-16T22:22:06.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe5a00b3080 0x7fe5a00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:06.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:06.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a014c660 con 0x7fe5a00b3920 2020-11-16T22:22:06.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00ac120 con 0x7fe5a00b3080 2020-11-16T22:22:06.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59f59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00ac3c0 con 0x7fe5a00b13d0 2020-11-16T22:22:06.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:06.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59ed9d700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fe5a00b3920 0x7fe5a0147740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56666/0 (socket says 172.21.15.185:56666) 2020-11-16T22:22:06.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59ed9d700 1 -- 172.21.15.185:0/1503086885 learned_addr learned my addr 172.21.15.185:0/1503086885 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:06.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1503086885 >> v2:172.21.15.185:3300/0 conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:34992/0 (socket says 172.21.15.185:34992) 2020-11-16T22:22:06.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59e59c700 1 --2- 172.21.15.185:0/1503086885 >> v2:172.21.15.96:3300/0 conn(0x7fe5a00b3080 0x7fe5a00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 -- 172.21.15.185:0/1503086885 >> v2:172.21.15.96:3300/0 conn(0x7fe5a00b3080 msgr2=0x7fe5a00ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1503086885 >> v2:172.21.15.96:3300/0 conn(0x7fe5a00b3080 0x7fe5a00ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59d59a700 1 -- 172.21.15.185:0/1503086885 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634972191 0 0) 0x7fe5a014c660 con 0x7fe5a00b3920 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 -- 172.21.15.185:0/1503086885 >> v1:172.21.15.185:6789/0 conn(0x7fe5a00b3920 legacy=0x7fe5a0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 -- 172.21.15.185:0/1503086885 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00acd50 con 0x7fe5a00b13d0 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.210+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1503086885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe5a00b3ee0 tx=0x7fe590006df0).ready entity=mon.1 client_cookie=bb74fa730bfc0bb7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59d59a700 1 -- 172.21.15.185:0/1503086885 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900097b0 con 0x7fe5a00b13d0 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59d59a700 1 -- 172.21.15.185:0/1503086885 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590009910 con 0x7fe5a00b13d0 2020-11-16T22:22:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59d59a700 1 -- 172.21.15.185:0/1503086885 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590009c90 con 0x7fe5a00b13d0 2020-11-16T22:22:06.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- 172.21.15.185:0/1503086885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b13d0 msgr2=0x7fe5a00b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:06.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 --2- 172.21.15.185:0/1503086885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe5a00b3ee0 tx=0x7fe590006df0).stop 2020-11-16T22:22:06.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- 172.21.15.185:0/1503086885 shutdown_connections 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 --2- 172.21.15.185:0/1503086885 >> v2:172.21.15.96:3300/0 conn(0x7fe5a00b3080 0x7fe5a00ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 --2- 172.21.15.185:0/1503086885 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00b1bf0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- 172.21.15.185:0/1503086885 >> 172.21.15.185:0/1503086885 conn(0x7fe5a0060130 msgr2=0x7fe5a0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- 172.21.15.185:0/1503086885 shutdown_connections 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- 172.21.15.185:0/1503086885 wait complete. 2020-11-16T22:22:06.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 Processor -- start 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.214+0000 7fe59f59e700 1 -- start start 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 0x7fe5a010f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00b6990 con 0x7fe5a00b13d0 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00afcd0 con 0x7fe5a00b3920 2020-11-16T22:22:06.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0158520 con 0x7fe5a00b3080 2020-11-16T22:22:06.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:06.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 0x7fe5a010f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:06.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56676/0 (socket says 172.21.15.185:56676) 2020-11-16T22:22:06.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 -- 172.21.15.185:0/529093629 learned_addr learned my addr 172.21.15.185:0/529093629 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59e59c700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 -- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 msgr2=0x7fe5a010f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 0x7fe5a010f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 -- 172.21.15.185:0/529093629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a00b13d0 msgr2=0x7fe5a00ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:06.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 -- 172.21.15.185:0/529093629 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0157fe0 con 0x7fe5a00b3080 2020-11-16T22:22:06.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 0x7fe5a010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:06.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fe590000dd0 tx=0x7fe590015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:06.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe5967fc700 1 -- 172.21.15.185:0/529093629 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011c80 con 0x7fe5a00b3080 2020-11-16T22:22:06.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe5967fc700 1 -- 172.21.15.185:0/529093629 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590011de0 con 0x7fe5a00b3080 2020-11-16T22:22:06.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a0158280 con 0x7fe5a00b3080 2020-11-16T22:22:07.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe5967fc700 1 -- 172.21.15.185:0/529093629 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59001e670 con 0x7fe5a00b3080 2020-11-16T22:22:07.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a00b64a0 con 0x7fe5a00b3080 2020-11-16T22:22:07.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe5967fc700 1 -- 172.21.15.185:0/529093629 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fe590025030 con 0x7fe5a00b3080 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.218+0000 7fe5967fc700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 0x7fe570057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.222+0000 7fe5967fc700 1 -- 172.21.15.185:0/529093629 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe59006f250 con 0x7fe5a00b3080 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.222+0000 7fe59e59c700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 0x7fe570057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.222+0000 7fe59e59c700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 0x7fe570057520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fe5880099f0 tx=0x7fe5880091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.222+0000 7fe5b388a740 1 Processor -- start 2020-11-16T22:22:07.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- start start 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 0xe57380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xce44b0 con 0xe58da0 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xd98930 con 0xbccbb0 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xc586f0 con 0xc3a7b0 2020-11-16T22:22:07.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56682/0 (socket says 172.21.15.185:56682) 2020-11-16T22:22:07.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 -- 172.21.15.185:0/976860863 learned_addr learned my addr 172.21.15.185:0/976860863 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35008/0 (socket says 172.21.15.185:35008) 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 0xe57380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 -- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 msgr2=0xe56e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 -- 172.21.15.185:0/976860863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 msgr2=0xe57380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:07.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 0xe57380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 -- 172.21.15.185:0/976860863 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc535a0 con 0xc3a7b0 2020-11-16T22:22:07.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:07.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 0xe57380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:07.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe59e59c700 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fe588013490 tx=0x7fe588014000).ready entity=mon.2 client_cookie=bb730e218f8007a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:07.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe58f7fe700 1 -- 172.21.15.185:0/976860863 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800f7e0 con 0xc3a7b0 2020-11-16T22:22:07.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe58f7fe700 1 -- 172.21.15.185:0/976860863 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588013cb0 con 0xc3a7b0 2020-11-16T22:22:07.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe58f7fe700 1 -- 172.21.15.185:0/976860863 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58801e610 con 0xc3a7b0 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 msgr2=0xe588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fe588013490 tx=0x7fe588014000).stop 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- 172.21.15.185:0/976860863 shutdown_connections 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe58da0 0xe57380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc3a7b0 0xe588b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 --2- 172.21.15.185:0/976860863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe56e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- 172.21.15.185:0/976860863 >> 172.21.15.185:0/976860863 conn(0xe08e40 msgr2=0xe55dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- 172.21.15.185:0/976860863 shutdown_connections 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- 172.21.15.185:0/976860863 wait complete. 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 -- start start 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.226+0000 7fe5b388a740 1 Processor -- start 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xd856e0 con 0xc3a7b0 2020-11-16T22:22:07.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xd85820 con 0xbccbb0 2020-11-16T22:22:07.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xd86a90 con 0xe58da0 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 10 client.0 ll_register_callbacks cb 0xd7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56690/0 (socket says 172.21.15.185:56690) 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 -- 172.21.15.185:0/3898057929 learned_addr learned my addr 172.21.15.185:0/3898057929 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59e59c700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:07.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 -- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 msgr2=0xe9c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:07.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe9c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 -- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 msgr2=0xe552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:07.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:07.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 -- 172.21.15.185:0/3898057929 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd76280 con 0xe58da0 2020-11-16T22:22:07.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fe598007b50 tx=0x7fe598009040).ready entity=mon.2 client_cookie=c6e5efdb2837c2d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:07.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:07.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598007110 con 0xe58da0 2020-11-16T22:22:07.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe598016ca0 con 0xe58da0 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe396c0 con 0xe58da0 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe598016e60 con 0xe58da0 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 10 client.6965 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0xbabfd0 con 0xe58da0 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe396c0 con 0xe58da0 2020-11-16T22:22:07.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fe5980143d0 con 0xe58da0 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 10 client.6965 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 10 client.6965 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 20 client.6965 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xea0870 0xe870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 4 client.6965 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xc76c50 con 0xea0870 2020-11-16T22:22:07.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59e59c700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xea0870 0xe870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe58cff9700 10 client.6965 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:22:08.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.230+0000 7fe59e59c700 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xea0870 0xe870c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=c77392aa4b78c037 server_cookie=9b6b5e3cde44df99 in_seq=0 out_seq=0 2020-11-16T22:22:08.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe58cff9700 1 -- 172.21.15.185:0/3898057929 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xc76c50 con 0xea0870 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe58cff9700 10 client.6965 handle_command_reply: tid=0 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 client.6965 shutdown 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 msgr2=0xea0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fe598007b50 tx=0x7fe598009040).stop 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 shutdown_connections 2020-11-16T22:22:08.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xea0870 0xe870c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xbccbb0 0xe9c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xe58da0 0xea0380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 --2- 172.21.15.185:0/3898057929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe58cff9700 0 client.6965 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 >> 172.21.15.185:0/3898057929 conn(0xe08e40 msgr2=0xe56e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 shutdown_connections 2020-11-16T22:22:08.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 1 -- 172.21.15.185:0/3898057929 wait complete. 2020-11-16T22:22:08.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.234+0000 7fe5b388a740 20 client.6965 trim_cache size 0 max 16384 2020-11-16T22:22:08.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 Processor -- start 2020-11-16T22:22:08.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- start start 2020-11-16T22:22:08.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 0xe53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe8d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xd5de40 con 0xe08e40 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe56af0 con 0xc3a7b0 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe52950 con 0xbccbb0 2020-11-16T22:22:08.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe8d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35020/0 (socket says 172.21.15.185:35020) 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 -- 172.21.15.185:0/2812527315 learned_addr learned my addr 172.21.15.185:0/2812527315 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59dd9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe8d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56698/0 (socket says 172.21.15.185:56698) 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59e59c700 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 0xe53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 -- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 msgr2=0xe8d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:08.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe8d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 -- 172.21.15.185:0/2812527315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 msgr2=0xe53800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:08.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 0xe53800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 -- 172.21.15.185:0/2812527315 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe86a40 con 0xc3a7b0 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59e59c700 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 0xe53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe598020490 tx=0x7fe598015000).ready entity=mon.1 client_cookie=47f9444f5c037c42 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe58ffff700 1 -- 172.21.15.185:0/2812527315 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598007110 con 0xc3a7b0 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe58ffff700 1 -- 172.21.15.185:0/2812527315 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5980143d0 con 0xc3a7b0 2020-11-16T22:22:08.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe58ffff700 1 -- 172.21.15.185:0/2812527315 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598014540 con 0xc3a7b0 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 msgr2=0xe8dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe598020490 tx=0x7fe598015000).stop 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- 172.21.15.185:0/2812527315 shutdown_connections 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc3a7b0 0xe8dd80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe08e40 0xe53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 --2- 172.21.15.185:0/2812527315 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe8d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- 172.21.15.185:0/2812527315 >> 172.21.15.185:0/2812527315 conn(0x7fe568002320 msgr2=0xe88310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:08.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- 172.21.15.185:0/2812527315 shutdown_connections 2020-11-16T22:22:08.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- 172.21.15.185:0/2812527315 wait complete. 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.258+0000 7fe5b388a740 1 -- start start 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 Processor -- start 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:08.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xef3430 con 0xc3a7b0 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe8bb40 con 0xe08e40 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xef2f20 con 0xbccbb0 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 10 client.0 ll_register_callbacks cb 0xe8be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56702/0 (socket says 172.21.15.185:56702) 2020-11-16T22:22:08.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 -- 172.21.15.185:0/1645167347 learned_addr learned my addr 172.21.15.185:0/1645167347 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:08.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35030/0 (socket says 172.21.15.185:35030) 2020-11-16T22:22:08.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:08.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 -- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 msgr2=0xeef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:08.866 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 -- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 msgr2=0xe87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 -- 172.21.15.185:0/1645167347 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe568008160 con 0xbccbb0 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59ed9d700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59e59c700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fe588021040 tx=0x7fe58801e000).ready entity=mon.2 client_cookie=bcf9951c57757ec5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:08.867 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58801e610 con 0xbccbb0 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe58801ea40 con 0xbccbb0 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe588013cb0 con 0xbccbb0 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe58801ee10 con 0xbccbb0 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 10 client.6971 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:08.868 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0xbea720 con 0xbccbb0 2020-11-16T22:22:09.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588013cb0 con 0xbccbb0 2020-11-16T22:22:09.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fe588035030 con 0xbccbb0 2020-11-16T22:22:09.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 10 client.6971 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 10 client.6971 resolve_mds: resolved ID 'b' to GID 7080 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 20 client.6971 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xe58da0 0xeefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 4 client.6971 mds_command: new command op to 7080 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 --> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xc7df50 con 0xe58da0 2020-11-16T22:22:09.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.262+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xe58da0 0xeefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe58cff9700 10 client.6971 ms_handle_connect on v2:172.21.15.96:6832/862887203 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59dd9b700 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xe58da0 0xeefa90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=b4f24678c81b10c6 server_cookie=26e146bd0451d4d0 in_seq=0 out_seq=0 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe58cff9700 1 -- 172.21.15.185:0/1645167347 <== mds.1 v2:172.21.15.96:6832/862887203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xc7df50 con 0xe58da0 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe58cff9700 10 client.6971 handle_command_reply: tid=0 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 client.6971 shutdown 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 msgr2=0xe92650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:09.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fe588021040 tx=0x7fe58801e000).stop 2020-11-16T22:22:09.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 shutdown_connections 2020-11-16T22:22:09.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:6832/862887203,v1:172.21.15.96:6833/862887203] conn(0xe58da0 0xeefa90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe08e40 0xeef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe58cff9700 0 client.6971 ms_handle_reset on v2:172.21.15.96:6832/862887203 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xbccbb0 0xe92650 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 --2- 172.21.15.185:0/1645167347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc3a7b0 0xe87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 >> 172.21.15.185:0/1645167347 conn(0x7fe568002320 msgr2=0xe56d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 shutdown_connections 2020-11-16T22:22:09.275 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 1 -- 172.21.15.185:0/1645167347 wait complete. 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe5b388a740 20 client.6971 trim_cache size 0 max 16384 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 msgr2=0x7fe570057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 0x7fe570057520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fe5880099f0 tx=0x7fe5880091d0).stop 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 msgr2=0x7fe5a00b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fe590000dd0 tx=0x7fe590015000).stop 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 shutdown_connections 2020-11-16T22:22:09.276 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fe570055060 0x7fe570057520 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fe5a00b3920 0x7fe5a010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a00b13d0 0x7fe5a00ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 --2- 172.21.15.185:0/529093629 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fe5a00b3080 0x7fe5a00b5fb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 >> 172.21.15.185:0/529093629 conn(0x7fe5a0060130 msgr2=0x7fe5a00b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 shutdown_connections 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:06.266+0000 7fe59f59e700 1 -- 172.21.15.185:0/529093629 wait complete. 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:22:09.479 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:22:09.480 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:09.481 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:09.488 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:09.490 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:09.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 Processor -- start 2020-11-16T22:22:09.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 -- start start 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f13e80257d0 0x7f13e80dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e8025ce0 con 0x7f13e8041870 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80420e0 con 0x7f13e8046500 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13f4b74700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e8042380 con 0x7f13e80257d0 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ef7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f13e8046500 0x7f13e80dbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56704/0 (socket says 172.21.15.185:56704) 2020-11-16T22:22:09.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ef7fe700 1 -- 172.21.15.185:0/1638585712 learned_addr learned my addr 172.21.15.185:0/1638585712 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:09.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13eeffd700 1 --2- 172.21.15.185:0/1638585712 >> v2:172.21.15.96:3300/0 conn(0x7f13e80257d0 0x7f13e80dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:09.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209100480 0 0) 0x7f13e80420e0 con 0x7f13e8046500 2020-11-16T22:22:09.818 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d4003240 con 0x7f13e8046500 2020-11-16T22:22:09.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 800662970 0 0) 0x7f13d4003240 con 0x7f13e8046500 2020-11-16T22:22:09.822 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 >> v2:172.21.15.96:3300/0 conn(0x7f13e80257d0 msgr2=0x7f13e80dc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 --2- 172.21.15.185:0/1638585712 >> v2:172.21.15.96:3300/0 conn(0x7f13e80257d0 0x7f13e80dc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 >> v1:172.21.15.96:6789/0 conn(0x7f13e8041870 legacy=0x7f13e80daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8042620 con 0x7f13e8046500 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f13f00455c0 con 0x7f13e8046500 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 >> v1:172.21.15.185:6789/0 conn(0x7f13e8046500 legacy=0x7f13e80dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:09.823 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 0x7f13d4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:09.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4004060 0x7f13d4004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:09.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13d4004b40 0x7f13d4004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:09.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8042380 con 0x7f13d4003730 2020-11-16T22:22:09.824 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d400b950 con 0x7f13d4004060 2020-11-16T22:22:10.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.622+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d400bb10 con 0x7f13d4004b40 2020-11-16T22:22:10.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13eeffd700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4004060 0x7f13d4004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 0x7f13d4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13effff700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13d4004b40 0x7f13d4004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 -- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13d4004b40 msgr2=0x7f13d4004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:10.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13d4004b40 0x7f13d4004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 -- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4004060 msgr2=0x7f13d4004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4004060 0x7f13d4004540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 -- 172.21.15.185:0/1638585712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4003240 con 0x7f13d4003730 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13eeffd700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4004060 0x7f13d4004540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13effff700 1 -- 172.21.15.185:0/1638585712 reap_dead start 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 0x7f13d4003b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f13f005ff30 tx=0x7f13e8009670).ready entity=mon.0 client_cookie=a43e4a435a047607 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:10.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0069c00 con 0x7f13d4003730 2020-11-16T22:22:10.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f0069d60 con 0x7f13d4003730 2020-11-16T22:22:10.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13ee7fc700 1 -- 172.21.15.185:0/1638585712 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0066440 con 0x7f13d4003730 2020-11-16T22:22:10.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 -- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 msgr2=0x7f13d4003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:10.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 0x7f13d4003b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f13f005ff30 tx=0x7f13e8009670).stop 2020-11-16T22:22:10.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 -- 172.21.15.185:0/1638585712 shutdown_connections 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 --2- 172.21.15.185:0/1638585712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13d4003730 0x7f13d4003b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 -- 172.21.15.185:0/1638585712 >> 172.21.15.185:0/1638585712 conn(0x7f13e8006c00 msgr2=0x7f13e8007010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 -- 172.21.15.185:0/1638585712 shutdown_connections 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.626+0000 7f13f4b74700 1 -- 172.21.15.185:0/1638585712 wait complete. 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 Processor -- start 2020-11-16T22:22:10.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- start start 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4003730 0x7f13e803f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e803eb70 0x7f13e80296f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e803fff0 con 0x7f13e803eb70 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e803e430 con 0x7f13d4003730 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e803e6d0 con 0x7f13e8029be0 2020-11-16T22:22:10.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13ef7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4003730 0x7f13e803f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56726/0 (socket says 172.21.15.185:56726) 2020-11-16T22:22:10.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 -- 172.21.15.185:0/4251497651 learned_addr learned my addr 172.21.15.185:0/4251497651 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:10.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13eeffd700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e803eb70 0x7f13e80296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 -- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4003730 msgr2=0x7f13e803f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4003730 0x7f13e803f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 -- 172.21.15.185:0/4251497651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e803eb70 msgr2=0x7f13e80296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e803eb70 0x7f13e80296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 -- 172.21.15.185:0/4251497651 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e802a380 con 0x7f13e8029be0 2020-11-16T22:22:10.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13effff700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f13e8009560 tx=0x7f13e4007b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4007db0 con 0x7f13e8029be0 2020-11-16T22:22:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4007240 con 0x7f13e8029be0 2020-11-16T22:22:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40064f0 con 0x7f13e8029be0 2020-11-16T22:22:10.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e802a630 con 0x7f13e8029be0 2020-11-16T22:22:10.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.630+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e802b2e0 con 0x7f13e8029be0 2020-11-16T22:22:10.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e803e890 con 0x7f13e8029be0 2020-11-16T22:22:10.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f13e40068e0 con 0x7f13e8029be0 2020-11-16T22:22:10.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13d37fe700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 0x7f13c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:10.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f13e406deb0 con 0x7f13e8029be0 2020-11-16T22:22:10.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 0x7f13c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:10.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.634+0000 7f13ef7fe700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 0x7f13c8057520 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f13d400b920 tx=0x7f13f0088000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:10.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.642+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f13e4044110 con 0x7f13e8029be0 2020-11-16T22:22:10.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e8004fb0 con 0x7f13e8029be0 2020-11-16T22:22:10.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13d37fe700 1 -- 172.21.15.185:0/4251497651 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7f13e4047730 con 0x7f13e8029be0 2020-11-16T22:22:10.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 msgr2=0x7f13c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:10.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 0x7f13c8057520 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f13d400b920 tx=0x7f13f0088000).stop 2020-11-16T22:22:10.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 msgr2=0x7f13e802ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:10.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.834+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f13e8009560 tx=0x7f13e4007b00).stop 2020-11-16T22:22:10.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 shutdown_connections 2020-11-16T22:22:10.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f13c8055060 0x7f13c8057520 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f13d4003730 0x7f13e803f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f13e8029be0 0x7f13e802ac50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 --2- 172.21.15.185:0/4251497651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e803eb70 0x7f13e80296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 >> 172.21.15.185:0/4251497651 conn(0x7f13e8006c00 msgr2=0x7f13e803b330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 shutdown_connections 2020-11-16T22:22:10.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:09.838+0000 7f13f4b74700 1 -- 172.21.15.185:0/4251497651 wait complete. 2020-11-16T22:22:10.853 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:10.853 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:10.853 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:11.054 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub status 2020-11-16T22:22:11.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.202+0000 7ffb06240700 1 Processor -- start 2020-11-16T22:22:11.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.202+0000 7ffb06240700 1 -- start start 2020-11-16T22:22:11.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ffb000b5d70 0x7ffb000ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb00143470 0x7ffb00143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000b7290 con 0x7ffb000b4470 2020-11-16T22:22:11.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000b4bf0 con 0x7ffb000b5d70 2020-11-16T22:22:11.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000b74c0 con 0x7ffb00143470 2020-11-16T22:22:11.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb00143470 0x7ffb00143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ffb00143470 0x7ffb00143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35058/0 (socket says 172.21.15.185:35058) 2020-11-16T22:22:11.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3455269714 learned_addr learned my addr 172.21.15.185:0/3455269714 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb0523e700 1 --2- 172.21.15.185:0/3455269714 >> v2:172.21.15.96:3300/0 conn(0x7ffb000b5d70 0x7ffb000ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3455269714 >> v2:172.21.15.96:3300/0 conn(0x7ffb000b5d70 msgr2=0x7ffb000ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3455269714 >> v2:172.21.15.96:3300/0 conn(0x7ffb000b5d70 0x7ffb000ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3455269714 >> v1:172.21.15.96:6789/0 conn(0x7ffb000b4470 legacy=0x7ffb000b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3455269714 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000b7760 con 0x7ffb00143470 2020-11-16T22:22:11.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3455269714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb00143470 0x7ffb00143920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7ffb000b30c0 tx=0x7ffaf8006eb0).ready entity=mon.1 client_cookie=659de1b0aa223eff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffaeffff700 1 -- 172.21.15.185:0/3455269714 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8009810 con 0x7ffb00143470 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffaeffff700 1 -- 172.21.15.185:0/3455269714 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8009970 con 0x7ffb00143470 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffaeffff700 1 -- 172.21.15.185:0/3455269714 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8009cf0 con 0x7ffb00143470 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- 172.21.15.185:0/3455269714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb00143470 msgr2=0x7ffb00143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 --2- 172.21.15.185:0/3455269714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb00143470 0x7ffb00143920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7ffb000b30c0 tx=0x7ffaf8006eb0).stop 2020-11-16T22:22:11.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- 172.21.15.185:0/3455269714 shutdown_connections 2020-11-16T22:22:11.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 --2- 172.21.15.185:0/3455269714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb00143470 0x7ffb00143920 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 --2- 172.21.15.185:0/3455269714 >> v2:172.21.15.96:3300/0 conn(0x7ffb000b5d70 0x7ffb000ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- 172.21.15.185:0/3455269714 >> 172.21.15.185:0/3455269714 conn(0x7ffb0005ba80 msgr2=0x7ffb0005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- 172.21.15.185:0/3455269714 shutdown_connections 2020-11-16T22:22:11.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.206+0000 7ffb06240700 1 -- 172.21.15.185:0/3455269714 wait complete. 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 Processor -- start 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 -- start start 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb000b5d70 0x7ffb00156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 0x7ffb00156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000aed00 con 0x7ffb000b5d70 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb001144d0 con 0x7ffb000b4470 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb06240700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000af3a0 con 0x7ffb00143470 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb05a3f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 0x7ffb00156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35060/0 (socket says 172.21.15.185:35060) 2020-11-16T22:22:11.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb0523e700 1 -- 172.21.15.185:0/2541827110 learned_addr learned my addr 172.21.15.185:0/2541827110 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.210+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb000b5d70 0x7ffb00156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 -- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 msgr2=0x7ffb000b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 -- 172.21.15.185:0/2541827110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb000b5d70 msgr2=0x7ffb00156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb000b5d70 0x7ffb00156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 -- 172.21.15.185:0/2541827110 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000b0790 con 0x7ffb00143470 2020-11-16T22:22:11.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb0523e700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 0x7ffb00156f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7ffb00080bf0 tx=0x7ffaf8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 -- 172.21.15.185:0/2541827110 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8006be0 con 0x7ffb00143470 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 -- 172.21.15.185:0/2541827110 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf801e460 con 0x7ffb00143470 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb000af700 con 0x7ffb00143470 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 -- 172.21.15.185:0/2541827110 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf801e7e0 con 0x7ffb00143470 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb00113a90 con 0x7ffb00143470 2020-11-16T22:22:11.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 -- 172.21.15.185:0/2541827110 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ffaf8025030 con 0x7ffb00143470 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 0x7ffad8057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffaed7fa700 1 -- 172.21.15.185:0/2541827110 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffaf806f5b0 con 0x7ffb00143470 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.214+0000 7ffb0523e700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 0x7ffad8057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.218+0000 7ffb0523e700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 0x7ffad8057550 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ffb000af5a0 tx=0x7ffafc0091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.218+0000 7ffb15f33740 1 Processor -- start 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.218+0000 7ffb15f33740 1 -- start start 2020-11-16T22:22:11.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 0xe348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe32e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xcc04b0 con 0xc167b0 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xd74930 con 0xe34da0 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xc346f0 con 0xba8bb0 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb04a3d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe32e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35072/0 (socket says 172.21.15.185:35072) 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3315575309 learned_addr learned my addr 172.21.15.185:0/3315575309 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb0523e700 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 0xe348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 msgr2=0xe32e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe32e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3315575309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 msgr2=0xe348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 0xe348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3315575309 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc2f5a0 con 0xe34da0 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb0523e700 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 0xe348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7ffaf80068f0 tx=0x7ffaf8011000).ready entity=mon.1 client_cookie=f8b211b6028d11d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffade7fc700 1 -- 172.21.15.185:0/3315575309 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8047870 con 0xe34da0 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffade7fc700 1 -- 172.21.15.185:0/3315575309 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80489c0 con 0xe34da0 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffade7fc700 1 -- 172.21.15.185:0/3315575309 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf801ebd0 con 0xe34da0 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 msgr2=0xe33380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7ffaf80068f0 tx=0x7ffaf8011000).stop 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- 172.21.15.185:0/3315575309 shutdown_connections 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xe34da0 0xe33380 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xc167b0 0xe348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3315575309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe32e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- 172.21.15.185:0/3315575309 >> 172.21.15.185:0/3315575309 conn(0xde4e40 msgr2=0xe31dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- 172.21.15.185:0/3315575309 shutdown_connections 2020-11-16T22:22:11.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.222+0000 7ffb15f33740 1 -- 172.21.15.185:0/3315575309 wait complete. 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- start start 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 Processor -- start 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc167b0 0xe62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe34da0 0xe630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xd616e0 con 0xe34da0 2020-11-16T22:22:11.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xd61820 con 0xba8bb0 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xd62a90 con 0xc167b0 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 10 client.0 ll_register_callbacks cb 0xd57ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35074/0 (socket says 172.21.15.185:35074) 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 -- 172.21.15.185:0/134837559 learned_addr learned my addr 172.21.15.185:0/134837559 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc167b0 0xe62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe34da0 0xe630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 -- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc167b0 msgr2=0xe62bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc167b0 0xe62bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 -- 172.21.15.185:0/134837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe34da0 msgr2=0xe630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe34da0 0xe630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 -- 172.21.15.185:0/134837559 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xd52280 con 0xba8bb0 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb0523e700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7ffafc0127a0 tx=0x7ffafc015000).ready entity=mon.1 client_cookie=c3ae985868706aa5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffacf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc01e410 con 0xba8bb0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffafc018040 con 0xba8bb0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe156c0 con 0xba8bb0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffafc01e780 con 0xba8bb0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 10 client.7819 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.226+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0xb87fd0 con 0xba8bb0 2020-11-16T22:22:11.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe156c0 con 0xba8bb0 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7ffafc02d020 con 0xba8bb0 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 10 client.7819 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 10 client.7819 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 20 client.7819 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe7bc40 0xe7c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 4 client.7819 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0xc52c50 con 0xe7bc40 2020-11-16T22:22:11.629 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe7bc40 0xe7c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe7bc40 0xe7c020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=d58cfe5fe0f0134a server_cookie=db9daa8029d67c68 in_seq=0 out_seq=0 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 10 client.7819 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 1 -- 172.21.15.185:0/134837559 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xc52c50 con 0xe7bc40 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 10 client.7819 handle_command_reply: tid=0 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 client.7819 shutdown 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 msgr2=0xe37e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7ffafc0127a0 tx=0x7ffafc015000).stop 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 shutdown_connections 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe7bc40 0xe7c020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xba8bb0 0xe37e40 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xe34da0 0xe630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 --2- 172.21.15.185:0/134837559 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xc167b0 0xe62bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffacf7fe700 0 client.7819 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:22:11.631 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 >> 172.21.15.185:0/134837559 conn(0xde4e40 msgr2=0xe32e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 shutdown_connections 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 1 -- 172.21.15.185:0/134837559 wait complete. 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.230+0000 7ffb15f33740 20 client.7819 trim_cache size 0 max 16384 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 Processor -- start 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 -- start start 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xe32d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.632 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xd39e40 con 0xde4e40 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe37c10 con 0xc167b0 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe35f10 con 0xba8bb0 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb05a3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb05a3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35084/0 (socket says 172.21.15.185:35084) 2020-11-16T22:22:11.633 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3604669136 learned_addr learned my addr 172.21.15.185:0/3604669136 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe31820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.250+0000 7ffb0523e700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xe32d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 msgr2=0xe31820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe31820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3604669136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 msgr2=0xe32d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xe32d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 -- 172.21.15.185:0/3604669136 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4008160 con 0xc167b0 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb0523e700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xe32d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7ffaf80459d0 tx=0x7ffaf8029000).ready entity=mon.1 client_cookie=2c6b25e7ab62f51 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffadeffd700 1 -- 172.21.15.185:0/3604669136 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf801ebd0 con 0xc167b0 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffadeffd700 1 -- 172.21.15.185:0/3604669136 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80452c0 con 0xc167b0 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffadeffd700 1 -- 172.21.15.185:0/3604669136 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8045660 con 0xc167b0 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 -- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 msgr2=0xe31d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7ffaf80459d0 tx=0x7ffaf8029000).stop 2020-11-16T22:22:11.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 -- 172.21.15.185:0/3604669136 shutdown_connections 2020-11-16T22:22:11.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xe31d10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xe32d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 --2- 172.21.15.185:0/3604669136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe31820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 -- 172.21.15.185:0/3604669136 >> 172.21.15.185:0/3604669136 conn(0x7ffad4002320 msgr2=0xe30eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 -- 172.21.15.185:0/3604669136 shutdown_connections 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.254+0000 7ffb15f33740 1 -- 172.21.15.185:0/3604669136 wait complete. 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 -- start start 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 Processor -- start 2020-11-16T22:22:11.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xece260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xece750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0xe72930 con 0xde4e40 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0xe677a0 con 0xc167b0 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0xe6b410 con 0xba8bb0 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 10 client.0 ll_register_callbacks cb 0xe32740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb04a3d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xece260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56764/0 (socket says 172.21.15.185:56764) 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 -- 172.21.15.185:0/1914161187 learned_addr learned my addr 172.21.15.185:0/1914161187 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xece750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 -- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 msgr2=0xece260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xece260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 -- 172.21.15.185:0/1914161187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 msgr2=0xece750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:11.841 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xece750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 -- 172.21.15.185:0/1914161187 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0xbc6720 con 0xba8bb0 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb05a3f700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xece750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb0523e700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7ffafc013860 tx=0x7ffafc023000).ready entity=mon.2 client_cookie=8a2b10d17a106a54 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffacf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc037070 con 0xba8bb0 2020-11-16T22:22:11.842 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffafc026720 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.258+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaf80489c0 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffafc013de0 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 10 client.6989 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0xbd1e00 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80489c0 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7ffafc02d040 con 0xba8bb0 2020-11-16T22:22:11.843 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 10 client.6989 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 10 client.6989 resolve_mds: resolved ID 'a' to GID 6545 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 20 client.6989 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe34da0 0xecec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 4 client.6989 mds_command: new command op to 6545 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 --> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0xc59f50 con 0xe34da0 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe34da0 0xecec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb04a3d700 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe34da0 0xecec40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=bf8a92dac73755f8 server_cookie=898496141921f662 in_seq=0 out_seq=0 2020-11-16T22:22:11.844 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 10 client.6989 ms_handle_connect on v2:172.21.15.185:6834/2224996985 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 1 -- 172.21.15.185:0/1914161187 <== mds.2 v2:172.21.15.185:6834/2224996985 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xc59f50 con 0xe34da0 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 10 client.6989 handle_command_reply: tid=0 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 client.6989 shutdown 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 msgr2=0xe696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7ffafc013860 tx=0x7ffafc023000).stop 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 shutdown_connections 2020-11-16T22:22:11.845 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:6834/2224996985,v1:172.21.15.185:6835/2224996985] conn(0xe34da0 0xecec40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0xde4e40 0xece750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0xba8bb0 0xe696f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 --2- 172.21.15.185:0/1914161187 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0xc167b0 0xece260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffacf7fe700 0 client.6989 ms_handle_reset on v2:172.21.15.185:6834/2224996985 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 >> 172.21.15.185:0/1914161187 conn(0x7ffad4002320 msgr2=0xe30200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 shutdown_connections 2020-11-16T22:22:11.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 1 -- 172.21.15.185:0/1914161187 wait complete. 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb15f33740 20 client.6989 trim_cache size 0 max 16384 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 msgr2=0x7ffad8057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 0x7ffad8057550 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7ffb000af5a0 tx=0x7ffafc0091d0).stop 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 msgr2=0x7ffb00156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 0x7ffb00156f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7ffb00080bf0 tx=0x7ffaf8015000).stop 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 shutdown_connections 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ffad8055090 0x7ffad8057550 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ffb000b4470 0x7ffb000b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ffb00143470 0x7ffb00156f80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 --2- 172.21.15.185:0/2541827110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb000b5d70 0x7ffb00156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 >> 172.21.15.185:0/2541827110 conn(0x7ffb0005ba80 msgr2=0x7ffb000b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 shutdown_connections 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.262+0000 7ffb06240700 1 -- 172.21.15.185:0/2541827110 wait complete. 2020-11-16T22:22:11.848 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:22:11.849 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:22:11.849 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:22:11.849 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:22:11.849 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:11.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 Processor -- start 2020-11-16T22:22:11.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 -- start start 2020-11-16T22:22:11.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f96f00b3b80 0x7f96f00b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f0060ef0 con 0x7f96f00b3090 2020-11-16T22:22:11.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f01461d0 con 0x7f96f01421d0 2020-11-16T22:22:11.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f7540700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f0062090 con 0x7f96f00b3b80 2020-11-16T22:22:11.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f6d3f700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f96f01421d0 0x7f96f0145ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56766/0 (socket says 172.21.15.185:56766) 2020-11-16T22:22:11.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f6d3f700 1 -- 172.21.15.185:0/2076530262 learned_addr learned my addr 172.21.15.185:0/2076530262 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:11.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f5d3d700 1 --2- 172.21.15.185:0/2076530262 >> v2:172.21.15.96:3300/0 conn(0x7f96f00b3b80 0x7f96f00b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.990+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157109682 0 0) 0x7f96f01461d0 con 0x7f96f01421d0 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d8003240 con 0x7f96f01421d0 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657086984 0 0) 0x7f96f0060ef0 con 0x7f96f00b3090 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f01461d0 con 0x7f96f00b3090 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f5d3d700 1 -- 172.21.15.185:0/2076530262 >> v1:172.21.15.185:6789/0 conn(0x7f96f01421d0 legacy=0x7f96f0145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f5d3d700 1 -- 172.21.15.185:0/2076530262 >> v1:172.21.15.96:6789/0 conn(0x7f96f00b3090 legacy=0x7f96f01453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f5d3d700 1 -- 172.21.15.185:0/2076530262 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00ad640 con 0x7f96f00b3b80 2020-11-16T22:22:11.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f5d3d700 1 --2- 172.21.15.185:0/2076530262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f00b3b80 0x7f96f00b1840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f96e400b9a0 tx=0x7f96e4006df0).ready entity=mon.0 client_cookie=2e7c66a5187ccdad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4009680 con 0x7f96f00b3b80 2020-11-16T22:22:11.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e40097e0 con 0x7f96f00b3b80 2020-11-16T22:22:11.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f553c700 1 -- 172.21.15.185:0/2076530262 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4009b60 con 0x7f96f00b3b80 2020-11-16T22:22:11.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- 172.21.15.185:0/2076530262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f00b3b80 msgr2=0x7f96f00b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 --2- 172.21.15.185:0/2076530262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f00b3b80 0x7f96f00b1840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f96e400b9a0 tx=0x7f96e4006df0).stop 2020-11-16T22:22:11.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- 172.21.15.185:0/2076530262 shutdown_connections 2020-11-16T22:22:11.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 --2- 172.21.15.185:0/2076530262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f00b3b80 0x7f96f00b1840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- 172.21.15.185:0/2076530262 >> 172.21.15.185:0/2076530262 conn(0x7f96f0060130 msgr2=0x7f96f0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:11.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- 172.21.15.185:0/2076530262 shutdown_connections 2020-11-16T22:22:11.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- 172.21.15.185:0/2076530262 wait complete. 2020-11-16T22:22:11.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 Processor -- start 2020-11-16T22:22:11.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.994+0000 7f96f7540700 1 -- start start 2020-11-16T22:22:11.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f00b3090 0x7f96f0151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f01421d0 0x7f96f013dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01105d0 con 0x7f96f01421d0 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0151ff0 con 0x7f96f013e090 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f010fdf0 con 0x7f96f00b3090 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f653e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f00b3090 0x7f96f0151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35104/0 (socket says 172.21.15.185:35104) 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 -- 172.21.15.185:0/1915350397 learned_addr learned my addr 172.21.15.185:0/1915350397 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f5d3d700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f01421d0 0x7f96f013dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 -- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f00b3090 msgr2=0x7f96f0151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f00b3090 0x7f96f0151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 -- 172.21.15.185:0/1915350397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f01421d0 msgr2=0x7f96f013dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f01421d0 0x7f96f013dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:11.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 -- 172.21.15.185:0/1915350397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0110090 con 0x7f96f013e090 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f6d3f700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f96e800ca60 tx=0x7f96e8007d80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8005b80 con 0x7f96f013e090 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8005ce0 con 0x7f96f013e090 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f0110390 con 0x7f96f013e090 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8007360 con 0x7f96f013e090 2020-11-16T22:22:11.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f013d1c0 con 0x7f96f013e090 2020-11-16T22:22:11.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f013eb00 con 0x7f96f013e090 2020-11-16T22:22:11.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f96e8010030 con 0x7f96f013e090 2020-11-16T22:22:11.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:11.998+0000 7f96e27fc700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 0x7f96c8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:11.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.002+0000 7f96f653e700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 0x7f96c8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:11.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.002+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96e806db30 con 0x7f96f013e090 2020-11-16T22:22:11.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.002+0000 7f96f653e700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 0x7f96c8057520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f96ec0033d0 tx=0x7f96ec007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:11.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.006+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f96f013eb00 con 0x7f96f013e090 2020-11-16T22:22:12.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.206+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96f013eb00 con 0x7f96f013e090 2020-11-16T22:22:12.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.206+0000 7f96e27fc700 1 -- 172.21.15.185:0/1915350397 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7f96f013eb00 con 0x7f96f013e090 2020-11-16T22:22:12.199 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:12.199 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 msgr2=0x7f96c8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 0x7f96c8057520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f96ec0033d0 tx=0x7f96ec007a20).stop 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 msgr2=0x7f96f013ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f96e800ca60 tx=0x7f96e8007d80).stop 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 shutdown_connections 2020-11-16T22:22:12.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f96c8055060 0x7f96c8057520 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f96f013e090 0x7f96f013ccd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f96f00b3090 0x7f96f0151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 --2- 172.21.15.185:0/1915350397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f01421d0 0x7f96f013dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 >> 172.21.15.185:0/1915350397 conn(0x7f96f0060130 msgr2=0x7f96f00b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 shutdown_connections 2020-11-16T22:22:12.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.210+0000 7f96f7540700 1 -- 172.21.15.185:0/1915350397 wait complete. 2020-11-16T22:22:12.203 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:12.213 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:22:12.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.342+0000 7fac0afc2700 1 Processor -- start 2020-11-16T22:22:12.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.342+0000 7fac0afc2700 1 -- start start 2020-11-16T22:22:12.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fac040ba800 0x7fac04144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac041446e0 con 0x7fac0413dd30 2020-11-16T22:22:12.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac041444f0 con 0x7fac040ba800 2020-11-16T22:22:12.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac04144980 con 0x7fac040b5c40 2020-11-16T22:22:12.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56786/0 (socket says 172.21.15.185:56786) 2020-11-16T22:22:12.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 -- 172.21.15.185:0/4136347117 learned_addr learned my addr 172.21.15.185:0/4136347117 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac097bf700 1 --2- 172.21.15.185:0/4136347117 >> v2:172.21.15.96:3300/0 conn(0x7fac040ba800 0x7fac04144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 -- 172.21.15.185:0/4136347117 >> v2:172.21.15.96:3300/0 conn(0x7fac040ba800 msgr2=0x7fac04144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 --2- 172.21.15.185:0/4136347117 >> v2:172.21.15.96:3300/0 conn(0x7fac040ba800 0x7fac04144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 -- 172.21.15.185:0/4136347117 >> v1:172.21.15.96:6789/0 conn(0x7fac0413dd30 legacy=0x7fac040b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 -- 172.21.15.185:0/4136347117 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04144c20 con 0x7fac040b5c40 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac09fc0700 1 --2- 172.21.15.185:0/4136347117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fac00006fd0 tx=0x7fac00006eb0).ready entity=mon.2 client_cookie=7e1530ef4a2c7d4e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac08fbe700 1 -- 172.21.15.185:0/4136347117 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00009810 con 0x7fac040b5c40 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac08fbe700 1 -- 172.21.15.185:0/4136347117 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00009970 con 0x7fac040b5c40 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac08fbe700 1 -- 172.21.15.185:0/4136347117 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00009cf0 con 0x7fac040b5c40 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- 172.21.15.185:0/4136347117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 msgr2=0x7fac040b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.339 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 --2- 172.21.15.185:0/4136347117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fac00006fd0 tx=0x7fac00006eb0).stop 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- 172.21.15.185:0/4136347117 shutdown_connections 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 --2- 172.21.15.185:0/4136347117 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac040b6020 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 --2- 172.21.15.185:0/4136347117 >> v2:172.21.15.96:3300/0 conn(0x7fac040ba800 0x7fac04144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- 172.21.15.185:0/4136347117 >> 172.21.15.185:0/4136347117 conn(0x7fac0405ba80 msgr2=0x7fac0405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- 172.21.15.185:0/4136347117 shutdown_connections 2020-11-16T22:22:12.340 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 -- 172.21.15.185:0/4136347117 wait complete. 2020-11-16T22:22:12.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.346+0000 7fac0afc2700 1 Processor -- start 2020-11-16T22:22:12.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- start start 2020-11-16T22:22:12.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac0413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 0x7fac040ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040b0ce0 con 0x7fac0413dd30 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04146160 con 0x7fac040ba800 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040aed40 con 0x7fac040b5c40 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35118/0 (socket says 172.21.15.185:35118) 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 -- 172.21.15.185:0/1588328863 learned_addr learned my addr 172.21.15.185:0/1588328863 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac0413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 0x7fac040ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 -- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 msgr2=0x7fac0413cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac0413cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 -- 172.21.15.185:0/1588328863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 msgr2=0x7fac040ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 0x7fac040ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 -- 172.21.15.185:0/1588328863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040b1140 con 0x7fac040ba800 2020-11-16T22:22:12.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 0x7fac040ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac097bf700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fabf800eba0 tx=0x7fabf8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fabf5ffb700 1 -- 172.21.15.185:0/1588328863 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800c8e0 con 0x7fac040ba800 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fabf5ffb700 1 -- 172.21.15.185:0/1588328863 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800ca40 con 0x7fac040ba800 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fabf5ffb700 1 -- 172.21.15.185:0/1588328863 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800cdc0 con 0x7fac040ba800 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac04145e90 con 0x7fac040ba800 2020-11-16T22:22:12.345 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.350+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040af3d0 con 0x7fac040ba800 2020-11-16T22:22:12.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.354+0000 7fabf5ffb700 1 -- 172.21.15.185:0/1588328863 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fabf801a030 con 0x7fac040ba800 2020-11-16T22:22:12.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.354+0000 7fabf5ffb700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 0x7fabdc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.354+0000 7fabf5ffb700 1 -- 172.21.15.185:0/1588328863 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabf806d9d0 con 0x7fac040ba800 2020-11-16T22:22:12.346 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.354+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 0x7fabdc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.347 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.354+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 0x7fabdc057520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fac0000b1b0 tx=0x7fac00015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 Processor -- start 2020-11-16T22:22:12.350 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 -- start start 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x17f2bb0 0x1a7ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 0x1a7d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x190a4b0 con 0x1a7eda0 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19be930 con 0x18607b0 2020-11-16T22:22:12.352 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x187e6f0 con 0x17f2bb0 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac09fc0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x17f2bb0 0x1a7ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac09fc0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35124/0 (socket says 172.21.15.185:35124) 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac09fc0700 1 -- 172.21.15.185:0/2062844397 learned_addr learned my addr 172.21.15.185:0/2062844397 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.358+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 0x1a7d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 -- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x17f2bb0 msgr2=0x1a7ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.353 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x17f2bb0 0x1a7ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 -- 172.21.15.185:0/2062844397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 msgr2=0x1a7d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 0x1a7d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 -- 172.21.15.185:0/2062844397 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18795a0 con 0x18607b0 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 0x1a7d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7fac04144ac0 tx=0x7fac0001f000).ready entity=mon.1 client_cookie=3ae405003479735a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fabe2ffd700 1 -- 172.21.15.185:0/2062844397 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00011c80 con 0x18607b0 2020-11-16T22:22:12.354 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fabe2ffd700 1 -- 172.21.15.185:0/2062844397 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00011de0 con 0x18607b0 2020-11-16T22:22:12.355 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fabe2ffd700 1 -- 172.21.15.185:0/2062844397 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0002a660 con 0x18607b0 2020-11-16T22:22:12.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 msgr2=0x1a7e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.357 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7fac04144ac0 tx=0x7fac0001f000).stop 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- 172.21.15.185:0/2062844397 shutdown_connections 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18607b0 0x1a7e8b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1a7eda0 0x1a7d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2062844397 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x17f2bb0 0x1a7ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- 172.21.15.185:0/2062844397 >> 172.21.15.185:0/2062844397 conn(0x1a2ee40 msgr2=0x1a7bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- 172.21.15.185:0/2062844397 shutdown_connections 2020-11-16T22:22:12.358 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- 172.21.15.185:0/2062844397 wait complete. 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- start start 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 Processor -- start 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1aacb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1ab2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ab6e0 con 0x17f2bb0 2020-11-16T22:22:12.359 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ab820 con 0x1a7eda0 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19aca90 con 0x18607b0 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac1acb5740 10 client.0 ll_register_callbacks cb 0x19a1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1ab2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56808/0 (socket says 172.21.15.185:56808) 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac097bf700 1 -- 172.21.15.185:0/1502749153 learned_addr learned my addr 172.21.15.185:0/1502749153 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.360 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1aacb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 -- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 msgr2=0x1aad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 -- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 msgr2=0x1aacb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1aacb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac0a7c1700 1 -- 172.21.15.185:0/1502749153 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x199c280 con 0x1a7eda0 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac097bf700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:12.361 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.362+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1aacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1ab2480 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fabfc00f850 tx=0x7fabfc009040).ready entity=mon.1 client_cookie=9f1f2c017289ebb2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00d610 con 0x1a7eda0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabfc00d770 con 0x1a7eda0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a5f6c0 con 0x1a7eda0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabfc00db40 con 0x1a7eda0 2020-11-16T22:22:12.362 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 10 client.7840 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x17d1fd0 con 0x1a7eda0 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a5f6c0 con 0x1a7eda0 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fabfc00dca0 con 0x1a7eda0 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 10 client.7840 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 10 client.7840 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 20 client.7840 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1ab3dc0 0x1ac64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.363 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 4 client.7840 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:22:12.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x189cc50 con 0x1ab3dc0 2020-11-16T22:22:12.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1ab3dc0 0x1ac64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 10 client.7840 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:12.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac09fc0700 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1ab3dc0 0x1ac64d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ef374584ee90eef5 server_cookie=111cce2fafd63b6f in_seq=0 out_seq=0 2020-11-16T22:22:12.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 1 -- 172.21.15.185:0/1502749153 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x189cc50 con 0x1ab3dc0 2020-11-16T22:22:12.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 10 client.7840 handle_command_reply: tid=0 2020-11-16T22:22:12.365 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 client.7840 shutdown 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 msgr2=0x1ab2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1ab2480 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7fabfc00f850 tx=0x7fabfc009040).stop 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 shutdown_connections 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1ab3dc0 0x1ac64d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1ab2480 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1aacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fabd7fff700 0 client.7840 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 --2- 172.21.15.185:0/1502749153 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1aad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 >> 172.21.15.185:0/1502749153 conn(0x1a2ee40 msgr2=0x1aae770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 shutdown_connections 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 1 -- 172.21.15.185:0/1502749153 wait complete. 2020-11-16T22:22:12.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.366+0000 7fac1acb5740 20 client.7840 trim_cache size 0 max 16384 2020-11-16T22:22:12.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 Processor -- start 2020-11-16T22:22:12.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- start start 2020-11-16T22:22:12.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 0x1aac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1983e40 con 0x17f2bb0 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ab4ba0 con 0x1a2ee40 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a80ff0 con 0x18607b0 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac0a7c1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 0x1aac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac0a7c1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56816/0 (socket says 172.21.15.185:56816) 2020-11-16T22:22:12.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac0a7c1700 1 -- 172.21.15.185:0/2952414342 learned_addr learned my addr 172.21.15.185:0/2952414342 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac097bf700 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 -- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 msgr2=0x1a7e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 -- 172.21.15.185:0/2952414342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 msgr2=0x1a7def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 -- 172.21.15.185:0/2952414342 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00081d0 con 0x1a2ee40 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac09fc0700 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 0x1aac960 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fac0002c000 tx=0x7fac00013000).ready entity=mon.1 client_cookie=70062bde8a3d44f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fabe37fe700 1 -- 172.21.15.185:0/2952414342 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00011c80 con 0x1a2ee40 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fabe37fe700 1 -- 172.21.15.185:0/2952414342 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0000b730 con 0x1a2ee40 2020-11-16T22:22:12.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fabe37fe700 1 -- 172.21.15.185:0/2952414342 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000370e0 con 0x1a2ee40 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 msgr2=0x1aac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 0x1aac960 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fac0002c000 tx=0x7fac00013000).stop 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- 172.21.15.185:0/2952414342 shutdown_connections 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a2ee40 0x1aac960 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1a7e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 --2- 172.21.15.185:0/2952414342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- 172.21.15.185:0/2952414342 >> 172.21.15.185:0/2952414342 conn(0x7fabd0002320 msgr2=0x1aae770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- 172.21.15.185:0/2952414342 shutdown_connections 2020-11-16T22:22:12.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- 172.21.15.185:0/2952414342 wait complete. 2020-11-16T22:22:12.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 -- start start 2020-11-16T22:22:12.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.390+0000 7fac1acb5740 1 Processor -- start 2020-11-16T22:22:12.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1b141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a80500 con 0x17f2bb0 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aacd20 con 0x1a7eda0 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a80260 con 0x18607b0 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 10 client.0 ll_register_callbacks cb 0x1a78a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:22:12.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac0a7c1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1b141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac0a7c1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35148/0 (socket says 172.21.15.185:35148) 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac0a7c1700 1 -- 172.21.15.185:0/3449964896 learned_addr learned my addr 172.21.15.185:0/3449964896 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac09fc0700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 -- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 msgr2=0x1b146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 -- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 msgr2=0x1a7e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 -- 172.21.15.185:0/3449964896 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1810720 con 0x18607b0 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac0a7c1700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac09fc0700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac097bf700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1b141b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x1a7d320 tx=0x7fabf8013000).ready entity=mon.2 client_cookie=53b8d2bec74604b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:22:12.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8046c40 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18a3f50 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabf806d210 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabf806d5e0 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 10 client.7010 fetch_fsmap learned FSMap version 93 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x181be00 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18a3f50 con 0x18607b0 2020-11-16T22:22:12.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 93) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fabf8031030 con 0x18607b0 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 10 client.7010 fetch_fsmap finished waiting for FSMap version 93 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 10 client.7010 resolve_mds: resolved ID 'c' to GID 7083 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 20 client.7010 populate_metadata read hostname 'smithi185' 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1b18310 0x1b186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 4 client.7010 mds_command: new command op to 7083 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 --> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x18811d0 con 0x1b18310 2020-11-16T22:22:12.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac09fc0700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1b18310 0x1b186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fabd7fff700 10 client.7010 ms_handle_connect on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.394+0000 7fac09fc0700 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1b18310 0x1b186f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ed32beb408450179 server_cookie=d312f9dfb95ecdb in_seq=0 out_seq=0 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fabd7fff700 1 -- 172.21.15.185:0/3449964896 <== mds.0 v2:172.21.15.96:6834/3098481638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x18811d0 con 0x1b18310 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fabd7fff700 10 client.7010 handle_command_reply: tid=0 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 client.7010 shutdown 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 msgr2=0x1b141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1b141b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x1a7d320 tx=0x7fabf8013000).stop 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 shutdown_connections 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:6834/3098481638,v1:172.21.15.96:6835/3098481638] conn(0x1b18310 0x1b186f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1a7eda0 0x1b146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fabd7fff700 0 client.7010 ms_handle_reset on v2:172.21.15.96:6834/3098481638 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17f2bb0 0x1a7e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 --2- 172.21.15.185:0/3449964896 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18607b0 0x1b141b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 >> 172.21.15.185:0/3449964896 conn(0x7fabd0002320 msgr2=0x1a73a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 shutdown_connections 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 1 -- 172.21.15.185:0/3449964896 wait complete. 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac1acb5740 20 client.7010 trim_cache size 0 max 16384 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 msgr2=0x7fabdc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 0x7fabdc057520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fac0000b1b0 tx=0x7fac00015000).stop 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 msgr2=0x7fac040ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fabf800eba0 tx=0x7fabf8006df0).stop 2020-11-16T22:22:12.397 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 shutdown_connections 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fabdc055060 0x7fabdc057520 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0413dd30 0x7fac040ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fac040b5c40 0x7fac0413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 --2- 172.21.15.185:0/1588328863 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fac040ba800 0x7fac040ad570 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 >> 172.21.15.185:0/1588328863 conn(0x7fac0405ba80 msgr2=0x7fac040b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 shutdown_connections 2020-11-16T22:22:12.398 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.398+0000 7fac0afc2700 1 -- 172.21.15.185:0/1588328863 wait complete. 2020-11-16T22:22:12.399 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:22:12.399 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "no active scrubs running", 2020-11-16T22:22:12.399 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": {} 2020-11-16T22:22:12.399 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:22:12.405 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s 2020-11-16T22:22:12.405 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:12.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.546+0000 7f8ea4812700 1 Processor -- start 2020-11-16T22:22:12.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.546+0000 7f8ea4812700 1 -- start start 2020-11-16T22:22:12.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea0067510 con 0x7f8ea00b7470 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00677b0 con 0x7f8ea0067070 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00b17a0 con 0x7f8ea00682b0 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9effd700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f8ea0067070 0x7f8ea00ac030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56824/0 (socket says 172.21.15.185:56824) 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9effd700 1 -- 172.21.15.185:0/3086769787 learned_addr learned my addr 172.21.15.185:0/3086769787 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:12.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9ffff700 1 --2- 172.21.15.185:0/3086769787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746311245 0 0) 0x7f8ea00677b0 con 0x7f8ea0067070 2020-11-16T22:22:12.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c003240 con 0x7f8ea0067070 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9ffff700 1 -- 172.21.15.185:0/3086769787 >> v1:172.21.15.185:6789/0 conn(0x7f8ea0067070 legacy=0x7f8ea00ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9ffff700 1 -- 172.21.15.185:0/3086769787 >> v1:172.21.15.96:6789/0 conn(0x7f8ea00b7470 legacy=0x7f8ea00b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875207953 0 0) 0x7f8ea0067510 con 0x7f8ea00b7470 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9ffff700 1 -- 172.21.15.185:0/3086769787 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea00b1a40 con 0x7f8ea00682b0 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9ffff700 1 --2- 172.21.15.185:0/3086769787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b7850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f8e9800ae60 tx=0x7f8e9800a370).ready entity=mon.2 client_cookie=beb22146468782ad server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800cd90 con 0x7f8ea00682b0 2020-11-16T22:22:12.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98010410 con 0x7f8ea00682b0 2020-11-16T22:22:12.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8e9e7fc700 1 -- 172.21.15.185:0/3086769787 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98010790 con 0x7f8ea00682b0 2020-11-16T22:22:12.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- 172.21.15.185:0/3086769787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 msgr2=0x7f8ea00b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 --2- 172.21.15.185:0/3086769787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b7850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f8e9800ae60 tx=0x7f8e9800a370).stop 2020-11-16T22:22:12.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- 172.21.15.185:0/3086769787 shutdown_connections 2020-11-16T22:22:12.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 --2- 172.21.15.185:0/3086769787 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b7850 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- 172.21.15.185:0/3086769787 >> 172.21.15.185:0/3086769787 conn(0x7f8ea00637b0 msgr2=0x7f8ea00b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- 172.21.15.185:0/3086769787 shutdown_connections 2020-11-16T22:22:12.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.550+0000 7f8ea4812700 1 -- 172.21.15.185:0/3086769787 wait complete. 2020-11-16T22:22:12.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 Processor -- start 2020-11-16T22:22:12.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 -- start start 2020-11-16T22:22:12.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 0x7f8ea0151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 0x7f8ea00accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea013a110 con 0x7f8ea0067070 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00af1f0 con 0x7f8ea00b7470 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8ea4812700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0150e20 con 0x7f8ea00682b0 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 0x7f8ea00accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 0x7f8ea0151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 0x7f8ea0151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.185:39452/0 (socket says 172.21.15.185:39452) 2020-11-16T22:22:12.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56836/0 (socket says 172.21.15.185:56836) 2020-11-16T22:22:12.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.554+0000 7f8e9f7fe700 1 -- 172.21.15.185:0/913985798 learned_addr learned my addr 172.21.15.185:0/913985798 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 -- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 msgr2=0x7f8ea00b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 -- 172.21.15.185:0/913985798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 msgr2=0x7f8ea0151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 0x7f8ea0151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 -- 172.21.15.185:0/913985798 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea01510c0 con 0x7f8ea00b7470 2020-11-16T22:22:12.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9ffff700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 0x7f8ea00accd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f8e9800ae60 tx=0x7f8ea00676a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801a850 con 0x7f8ea00b7470 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9801a9b0 con 0x7f8ea00b7470 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801ad30 con 0x7f8ea00b7470 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea0151360 con 0x7f8ea00b7470 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea00ad1c0 con 0x7f8ea00b7470 2020-11-16T22:22:12.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea01514f0 con 0x7f8ea00b7470 2020-11-16T22:22:12.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f8e9802a070 con 0x7f8ea00b7470 2020-11-16T22:22:12.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 0x7f8e78057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e9806f8f0 con 0x7f8ea00b7470 2020-11-16T22:22:12.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.558+0000 7f8e9f7fe700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 0x7f8e78057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.562+0000 7f8e9f7fe700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 0x7f8e78057520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f8e90003400 tx=0x7f8e90007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.557 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.566+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f8e98045b50 con 0x7f8ea00b7470 2020-11-16T22:22:12.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ea005d200 con 0x7f8ea00b7470 2020-11-16T22:22:12.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8e7f7fe700 1 -- 172.21.15.185:0/913985798 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7f8e98014070 con 0x7f8ea00b7470 2020-11-16T22:22:12.758 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:12.758 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:12.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 msgr2=0x7f8e78057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 0x7f8e78057520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f8e90003400 tx=0x7f8e90007a20).stop 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 msgr2=0x7f8ea00accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 0x7f8ea00accd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f8e9800ae60 tx=0x7f8ea00676a0).stop 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 shutdown_connections 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.766+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8e78055060 0x7f8e78057520 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f8ea00b7470 0x7f8ea00accd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ea0067070 0x7f8ea0151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 --2- 172.21.15.185:0/913985798 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f8ea00682b0 0x7f8ea00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 >> 172.21.15.185:0/913985798 conn(0x7f8ea00637b0 msgr2=0x7f8ea0144170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 shutdown_connections 2020-11-16T22:22:12.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.770+0000 7f8ea4812700 1 -- 172.21.15.185:0/913985798 wait complete. 2020-11-16T22:22:12.762 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:12.772 DEBUG:tasks.cephfs.test_multimds_misc:current active=['c', 'b', 'a'] 2020-11-16T22:22:12.772 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2020-11-16T22:22:12.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 Processor -- start 2020-11-16T22:22:12.892 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- start start 2020-11-16T22:22:12.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 0x7fc55c147970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c0b2a30 con 0x7fc55c0680a0 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c0b24f0 con 0x7fc55c0b0f70 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c148570 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561a91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 0x7fc55c147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc562a93700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc55c0b0f70 0x7fc55c147e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56840/0 (socket says 172.21.15.185:56840) 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc562a93700 1 -- 172.21.15.185:0/3493246654 learned_addr learned my addr 172.21.15.185:0/3493246654 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:12.894 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561290700 1 -- 172.21.15.185:0/3493246654 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870151642 0 0) 0x7fc55c0b24f0 con 0x7fc55c0b0f70 2020-11-16T22:22:12.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561290700 1 -- 172.21.15.185:0/3493246654 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc544003240 con 0x7fc55c0b0f70 2020-11-16T22:22:12.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561a91700 1 -- 172.21.15.185:0/3493246654 >> v1:172.21.15.185:6789/0 conn(0x7fc55c0b0f70 legacy=0x7fc55c147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561a91700 1 -- 172.21.15.185:0/3493246654 >> v1:172.21.15.96:6789/0 conn(0x7fc55c0680a0 legacy=0x7fc55c14bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561a91700 1 -- 172.21.15.185:0/3493246654 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c0b2790 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.895 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561a91700 1 --2- 172.21.15.185:0/3493246654 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 0x7fc55c147970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fc54c00b9d0 tx=0x7fc54c006df0).ready entity=mon.2 client_cookie=1a27de16762c127c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561290700 1 -- 172.21.15.185:0/3493246654 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0097b0 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.896 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561290700 1 -- 172.21.15.185:0/3493246654 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c009910 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc561290700 1 -- 172.21.15.185:0/3493246654 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c009c90 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- 172.21.15.185:0/3493246654 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 msgr2=0x7fc55c147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 --2- 172.21.15.185:0/3493246654 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 0x7fc55c147970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fc54c00b9d0 tx=0x7fc54c006df0).stop 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- 172.21.15.185:0/3493246654 shutdown_connections 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 --2- 172.21.15.185:0/3493246654 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b1fa0 0x7fc55c147970 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- 172.21.15.185:0/3493246654 >> 172.21.15.185:0/3493246654 conn(0x7fc55c0657c0 msgr2=0x7fc55c065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:12.897 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- 172.21.15.185:0/3493246654 shutdown_connections 2020-11-16T22:22:12.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.902+0000 7fc563294700 1 -- 172.21.15.185:0/3493246654 wait complete. 2020-11-16T22:22:12.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 Processor -- start 2020-11-16T22:22:12.898 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- start start 2020-11-16T22:22:12.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 0x7fc55c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.899 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc55c0b1fa0 0x7fc55c0a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c15bf30 con 0x7fc55c0b1fa0 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0b0630 con 0x7fc55c0680a0 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c1585e0 con 0x7fc55c0b0f70 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.900 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc562292700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 0x7fc55c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56852/0 (socket says 172.21.15.185:56852) 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc562292700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 0x7fc55c0b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35172/0 (socket says 172.21.15.185:35172) 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 -- 172.21.15.185:0/1984097191 learned_addr learned my addr 172.21.15.185:0/1984097191 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc562a93700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc55c0b1fa0 0x7fc55c0a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 -- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 msgr2=0x7fc55c0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 0x7fc55c0b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.901 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 -- 172.21.15.185:0/1984097191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc55c0b1fa0 msgr2=0x7fc55c0a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc55c0b1fa0 0x7fc55c0a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 -- 172.21.15.185:0/1984097191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c158880 con 0x7fc55c0b0f70 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc561a91700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fc55c0aa0b0 tx=0x7fc54c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c006760 con 0x7fc55c0b0f70 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c011c80 con 0x7fc55c0b0f70 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c0b5140 con 0x7fc55c0b0f70 2020-11-16T22:22:12.902 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c01e460 con 0x7fc55c0b0f70 2020-11-16T22:22:12.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.906+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c0ad060 con 0x7fc55c0b0f70 2020-11-16T22:22:12.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc54c025030 con 0x7fc55c0b0f70 2020-11-16T22:22:12.903 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c0b07a0 con 0x7fc55c0b0f70 2020-11-16T22:22:12.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc54a7fc700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 0x7fc534057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:12.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc54c06f1a0 con 0x7fc55c0b0f70 2020-11-16T22:22:12.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc562292700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 0x7fc534057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:12.904 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.910+0000 7fc562292700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 0x7fc534057520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fc55c158510 tx=0x7fc554009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:12.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:12.918+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc55c0b07a0 con 0x7fc55c0b0f70 2020-11-16T22:22:13.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.122+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc55c0b07a0 con 0x7fc55c0b0f70 2020-11-16T22:22:13.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.122+0000 7fc54a7fc700 1 -- 172.21.15.185:0/1984097191 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1689 (secure 0 0 0) 0x7fc55c0b07a0 con 0x7fc55c0b0f70 2020-11-16T22:22:13.115 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:13.115 INFO:teuthology.orchestra.run.smithi185.stdout:{"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_age":505,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1605564834,"num_in_osds":8,"osd_in_since":1605564834,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":25}],"num_pgs":25,"num_pools":4,"num_objects":70,"data_bytes":113672,"bytes_used":57384960,"bytes_avail":773036728320,"bytes_total":773094113280,"write_bytes_sec":426,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":93,"id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"a","status":"up:active","gid":6545},{"filesystem_id":6,"rank":1,"name":"b","status":"up:active","gid":7080},{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7083}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2020-11-16T22:22:09.195048+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 msgr2=0x7fc534057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 0x7fc534057520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fc55c158510 tx=0x7fc554009040).stop 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 msgr2=0x7fc55c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fc55c0aa0b0 tx=0x7fc54c015000).stop 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 shutdown_connections 2020-11-16T22:22:13.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc534055060 0x7fc534057520 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc55c0680a0 0x7fc55c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc55c0b1fa0 0x7fc55c0a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 --2- 172.21.15.185:0/1984097191 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc55c0b0f70 0x7fc55c0abf90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 >> 172.21.15.185:0/1984097191 conn(0x7fc55c0657c0 msgr2=0x7fc55c14b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 shutdown_connections 2020-11-16T22:22:13.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.126+0000 7fc563294700 1 -- 172.21.15.185:0/1984097191 wait complete. 2020-11-16T22:22:13.130 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2020-11-16T22:22:13.130 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:22:13.130 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:13.168 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:13.185 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:13.200 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:22:13.201 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-16T22:22:13.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:13.201 INFO:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-16T22:22:13.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2020-11-16T22:22:13.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:13.292 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-16T22:22:13.311 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-16T22:22:13.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 Processor -- start 2020-11-16T22:22:13.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 -- start start 2020-11-16T22:22:13.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6e0c0b3070 0x7f6e0c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c1483e0 con 0x7f6e0c0b4d00 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c061870 con 0x7f6e0c0b3070 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e10814700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c13e4e0 con 0x7f6e0c0b4920 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.454+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0b59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6e0c0b3070 0x7f6e0c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56858/0 (socket says 172.21.15.185:56858) 2020-11-16T22:22:13.448 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/223698366 learned_addr learned my addr 172.21.15.185:0/223698366 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:13.449 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/223698366 >> v2:172.21.15.96:3300/0 conn(0x7f6e0c0b3070 msgr2=0x7f6e0c143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 --2- 172.21.15.185:0/223698366 >> v2:172.21.15.96:3300/0 conn(0x7f6e0c0b3070 0x7f6e0c143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/223698366 >> v1:172.21.15.96:6789/0 conn(0x7f6e0c0b4d00 legacy=0x7f6e0c0677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/223698366 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c067ee0 con 0x7f6e0c0b4920 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e0ad9d700 1 --2- 172.21.15.185:0/223698366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f6dfc006fd0 tx=0x7f6dfc006e50).ready entity=mon.2 client_cookie=3d01da09151ca520 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e09d9b700 1 -- 172.21.15.185:0/223698366 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc006030 con 0x7f6e0c0b4920 2020-11-16T22:22:13.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e09d9b700 1 -- 172.21.15.185:0/223698366 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc006190 con 0x7f6e0c0b4920 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e09d9b700 1 -- 172.21.15.185:0/223698366 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc006510 con 0x7f6e0c0b4920 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 -- 172.21.15.185:0/223698366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 msgr2=0x7f6e0c147ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 --2- 172.21.15.185:0/223698366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f6dfc006fd0 tx=0x7f6dfc006e50).stop 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 -- 172.21.15.185:0/223698366 shutdown_connections 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 --2- 172.21.15.185:0/223698366 >> v2:172.21.15.96:3300/0 conn(0x7f6e0c0b3070 0x7f6e0c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 --2- 172.21.15.185:0/223698366 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b4920 0x7f6e0c147ef0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 -- 172.21.15.185:0/223698366 >> 172.21.15.185:0/223698366 conn(0x7f6e0c063ce0 msgr2=0x7f6e0c0640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:13.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 -- 172.21.15.185:0/223698366 shutdown_connections 2020-11-16T22:22:13.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.458+0000 7f6e10814700 1 -- 172.21.15.185:0/223698366 wait complete. 2020-11-16T22:22:13.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 Processor -- start 2020-11-16T22:22:13.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- start start 2020-11-16T22:22:13.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 0x7f6e0c149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e0c0b4d00 0x7f6e0c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0b1460 con 0x7f6e0c0b4d00 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0b26d0 con 0x7f6e0c0b4920 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0b16c0 con 0x7f6e0c0b3070 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56862/0 (socket says 172.21.15.185:56862) 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/2443634227 learned_addr learned my addr 172.21.15.185:0/2443634227 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0a59c700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 0x7f6e0c149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0b59e700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e0c0b4d00 0x7f6e0c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 msgr2=0x7f6e0c149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 0x7f6e0c149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/2443634227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e0c0b4d00 msgr2=0x7f6e0c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e0c0b4d00 0x7f6e0c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 -- 172.21.15.185:0/2443634227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0b2970 con 0x7f6e0c0b3070 2020-11-16T22:22:13.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0a59c700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 0x7f6e0c149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:13.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e0ad9d700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f6dfc006160 tx=0x7f6dfc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc005a20 con 0x7f6e0c0b3070 2020-11-16T22:22:13.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc005ba0 con 0x7f6e0c0b3070 2020-11-16T22:22:13.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c149d70 con 0x7f6e0c0b3070 2020-11-16T22:22:13.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc01e460 con 0x7f6e0c0b3070 2020-11-16T22:22:13.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c0b2350 con 0x7f6e0c0b3070 2020-11-16T22:22:13.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.462+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c0b2540 con 0x7f6e0c0b3070 2020-11-16T22:22:13.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.466+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6dfc025030 con 0x7f6e0c0b3070 2020-11-16T22:22:13.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.466+0000 7f6df2ffd700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 0x7f6de8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.466+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6dfc06fa00 con 0x7f6e0c0b3070 2020-11-16T22:22:13.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.466+0000 7f6e0a59c700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 0x7f6de8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.466+0000 7f6e0a59c700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 0x7f6de8057520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f6e000011a0 tx=0x7f6e00009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.470+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6dfc045ba0 con 0x7f6e0c0b3070 2020-11-16T22:22:13.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.622+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6e0c05d200 con 0x7f6e0c0b3070 2020-11-16T22:22:13.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6df2ffd700 1 -- 172.21.15.185:0/2443634227 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6dfc023070 con 0x7f6e0c0b3070 2020-11-16T22:22:13.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 msgr2=0x7f6de8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 0x7f6de8057520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f6e000011a0 tx=0x7f6e00009040).stop 2020-11-16T22:22:13.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 msgr2=0x7f6e0c0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f6dfc006160 tx=0x7f6dfc015000).stop 2020-11-16T22:22:13.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 shutdown_connections 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6de8055060 0x7f6de8057520 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e0c0b4d00 0x7f6e0c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6e0c0b3070 0x7f6e0c0b1b60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 --2- 172.21.15.185:0/2443634227 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6e0c0b4920 0x7f6e0c149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 >> 172.21.15.185:0/2443634227 conn(0x7f6e0c063ce0 msgr2=0x7f6e0c0b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 shutdown_connections 2020-11-16T22:22:13.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.634+0000 7f6e10814700 1 -- 172.21.15.185:0/2443634227 wait complete. 2020-11-16T22:22:13.640 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2020-11-16T22:22:13.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 Processor -- start 2020-11-16T22:22:13.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 -- start start 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713009d410 con 0x7f7130144840 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713009b010 con 0x7f713009b270 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f7134ae2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7130144cf0 con 0x7f713009c900 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56870/0 (socket says 172.21.15.185:56870) 2020-11-16T22:22:13.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 -- 172.21.15.185:0/313237532 learned_addr learned my addr 172.21.15.185:0/313237532 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:13.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 -- 172.21.15.185:0/313237532 >> v1:172.21.15.185:6789/0 conn(0x7f713009b270 legacy=0x7f71301476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 -- 172.21.15.185:0/313237532 >> v1:172.21.15.96:6789/0 conn(0x7f7130144840 legacy=0x7f7130147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 -- 172.21.15.185:0/313237532 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130148500 con 0x7f713009c900 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712d59a700 1 -- 172.21.15.185:0/313237532 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436663700 0 0) 0x7f713009b010 con 0x7f713009b270 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712e59c700 1 --2- 172.21.15.185:0/313237532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f7120009d80 tx=0x7f7120009290).ready entity=mon.2 client_cookie=e8b521877487c881 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712d59a700 1 -- 172.21.15.185:0/313237532 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000bc80 con 0x7f713009c900 2020-11-16T22:22:13.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712d59a700 1 -- 172.21.15.185:0/313237532 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000bde0 con 0x7f713009c900 2020-11-16T22:22:13.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.770+0000 7f712d59a700 1 -- 172.21.15.185:0/313237532 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011610 con 0x7f713009c900 2020-11-16T22:22:13.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- 172.21.15.185:0/313237532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 msgr2=0x7f713009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 --2- 172.21.15.185:0/313237532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f7120009d80 tx=0x7f7120009290).stop 2020-11-16T22:22:13.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- 172.21.15.185:0/313237532 shutdown_connections 2020-11-16T22:22:13.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 --2- 172.21.15.185:0/313237532 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f713009cce0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- 172.21.15.185:0/313237532 >> 172.21.15.185:0/313237532 conn(0x7f7130068330 msgr2=0x7f71300ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:13.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- 172.21.15.185:0/313237532 shutdown_connections 2020-11-16T22:22:13.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- 172.21.15.185:0/313237532 wait complete. 2020-11-16T22:22:13.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 Processor -- start 2020-11-16T22:22:13.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.774+0000 7f7134ae2700 1 -- start start 2020-11-16T22:22:13.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f71300b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7130144840 0x7f7130153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713009f0a0 con 0x7f7130144840 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300a0890 con 0x7f713009b270 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7130154370 con 0x7f713009c900 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35200/0 (socket says 172.21.15.185:35200) 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 -- 172.21.15.185:0/1577077231 learned_addr learned my addr 172.21.15.185:0/1577077231 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:13.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712dd9b700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f71300b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712ed9d700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7130144840 0x7f7130153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 -- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 msgr2=0x7f71300b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f71300b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 -- 172.21.15.185:0/1577077231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7130144840 msgr2=0x7f7130153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7130144840 0x7f7130153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 -- 172.21.15.185:0/1577077231 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300a0580 con 0x7f713009b270 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f712e59c700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f71200090f0 tx=0x7f7120016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.772 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712001c680 con 0x7f713009b270 2020-11-16T22:22:13.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712001c7e0 con 0x7f713009b270 2020-11-16T22:22:13.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300afcb0 con 0x7f713009b270 2020-11-16T22:22:13.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712001cba0 con 0x7f713009b270 2020-11-16T22:22:13.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.778+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300b1830 con 0x7f713009b270 2020-11-16T22:22:13.773 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71300afa10 con 0x7f713009b270 2020-11-16T22:22:13.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7120017070 con 0x7f713009b270 2020-11-16T22:22:13.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f711e7fc700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 0x7f7104057570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:13.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f712006fdc0 con 0x7f713009b270 2020-11-16T22:22:13.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f712dd9b700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 0x7f7104057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:13.774 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.782+0000 7f712dd9b700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 0x7f7104057570 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7130142890 tx=0x7f71240079b0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:13.780 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.790+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f71300afa10 con 0x7f713009b270 2020-11-16T22:22:13.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:13.994+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) v1 -- 0x7f71300af9c0 con 0x7f713009b270 2020-11-16T22:22:14.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.278+0000 7f711e7fc700 1 -- 172.21.15.185:0/1577077231 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v384) v1 ==== 154+0+0 (secure 0 0 0) 0x7f7120015070 con 0x7f713009b270 2020-11-16T22:22:14.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 msgr2=0x7f7104057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 0x7f7104057570 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7130142890 tx=0x7f71240079b0).stop 2020-11-16T22:22:14.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 msgr2=0x7f71300b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f71200090f0 tx=0x7f7120016000).stop 2020-11-16T22:22:14.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 shutdown_connections 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f71040550b0 0x7f7104057570 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f713009b270 0x7f71300b0150 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7130144840 0x7f7130153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 --2- 172.21.15.185:0/1577077231 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f713009c900 0x7f71300b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 >> 172.21.15.185:0/1577077231 conn(0x7f7130068330 msgr2=0x7f7130144010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:14.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 shutdown_connections 2020-11-16T22:22:14.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.282+0000 7f7134ae2700 1 -- 172.21.15.185:0/1577077231 wait complete. 2020-11-16T22:22:14.288 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2020-11-16T22:22:14.289 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2020-11-16T22:22:14.289 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2020-11-16T22:22:14.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 Processor -- start 2020-11-16T22:22:14.425 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 -- start start 2020-11-16T22:22:14.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff24c0b3200 0x7ff24c099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:14.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c09b330 con 0x7ff24c142830 2020-11-16T22:22:14.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c09b090 con 0x7ff24c099cf0 2020-11-16T22:22:14.426 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff25102f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c146430 con 0x7ff24c0b3200 2020-11-16T22:22:14.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff24b59e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7ff24c099cf0 0x7ff24c141c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56882/0 (socket says 172.21.15.185:56882) 2020-11-16T22:22:14.427 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff24b59e700 1 -- 172.21.15.185:0/1327883532 learned_addr learned my addr 172.21.15.185:0/1327883532 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff24a59c700 1 --2- 172.21.15.185:0/1327883532 >> v2:172.21.15.96:3300/0 conn(0x7ff24c0b3200 0x7ff24c099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090101050 0 0) 0x7ff24c09b090 con 0x7ff24c099cf0 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff238003240 con 0x7ff24c099cf0 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff24a59c700 1 -- 172.21.15.185:0/1327883532 >> v1:172.21.15.185:6789/0 conn(0x7ff24c099cf0 legacy=0x7ff24c141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.434+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 623373847 0 0) 0x7ff238003240 con 0x7ff24c099cf0 2020-11-16T22:22:14.428 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff24a59c700 1 -- 172.21.15.185:0/1327883532 >> v1:172.21.15.96:6789/0 conn(0x7ff24c142830 legacy=0x7ff24c145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff24a59c700 1 -- 172.21.15.185:0/1327883532 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c1455e0 con 0x7ff24c0b3200 2020-11-16T22:22:14.429 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff24a59c700 1 --2- 172.21.15.185:0/1327883532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c0b3200 0x7ff24c099800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff24000b920 tx=0x7ff240006df0).ready entity=mon.0 client_cookie=9700df8898a5dd64 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244001520 con 0x7ff24c0b3200 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff240012460 con 0x7ff24c0b3200 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff249d9b700 1 -- 172.21.15.185:0/1327883532 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400127e0 con 0x7ff24c0b3200 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- 172.21.15.185:0/1327883532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c0b3200 msgr2=0x7ff24c099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 --2- 172.21.15.185:0/1327883532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c0b3200 0x7ff24c099800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7ff24000b920 tx=0x7ff240006df0).stop 2020-11-16T22:22:14.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- 172.21.15.185:0/1327883532 shutdown_connections 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 --2- 172.21.15.185:0/1327883532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c0b3200 0x7ff24c099800 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- 172.21.15.185:0/1327883532 >> 172.21.15.185:0/1327883532 conn(0x7ff24c060130 msgr2=0x7ff24c060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- 172.21.15.185:0/1327883532 shutdown_connections 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- 172.21.15.185:0/1327883532 wait complete. 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 Processor -- start 2020-11-16T22:22:14.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 -- start start 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.438+0000 7ff25102f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff24c142830 0x7ff24c09de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c09e380 0x7ff24c13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1525f0 con 0x7ff24c09e380 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c09d4b0 con 0x7ff24c0b3200 2020-11-16T22:22:14.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c09d750 con 0x7ff24c142830 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24a59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff24c142830 0x7ff24c09de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35214/0 (socket says 172.21.15.185:35214) 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 -- 172.21.15.185:0/3126444580 learned_addr learned my addr 172.21.15.185:0/3126444580 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24b59e700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c09e380 0x7ff24c13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 -- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff24c142830 msgr2=0x7ff24c09de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff24c142830 0x7ff24c09de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 -- 172.21.15.185:0/3126444580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c09e380 msgr2=0x7ff24c13eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c09e380 0x7ff24c13eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 -- 172.21.15.185:0/3126444580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c09d9f0 con 0x7ff24c0b3200 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff24ad9d700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7ff23c00cac0 tx=0x7ff23c00cdd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00a830 con 0x7ff24c0b3200 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00a990 con 0x7ff24c0b3200 2020-11-16T22:22:14.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c140730 con 0x7ff24c0b3200 2020-11-16T22:22:14.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00ad30 con 0x7ff24c0b3200 2020-11-16T22:22:14.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c1523b0 con 0x7ff24c0b3200 2020-11-16T22:22:14.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c05d4e0 con 0x7ff24c0b3200 2020-11-16T22:22:14.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.442+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff23c010070 con 0x7ff24c0b3200 2020-11-16T22:22:14.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.446+0000 7ff232ffd700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 0x7ff228057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:14.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.446+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff23c06daa0 con 0x7ff24c0b3200 2020-11-16T22:22:14.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.446+0000 7ff24a59c700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 0x7ff228057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:14.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.446+0000 7ff24a59c700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 0x7ff228057520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7ff24c061010 tx=0x7ff24c09b220).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:14.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.450+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff23c043d00 con 0x7ff24c0b3200 2020-11-16T22:22:14.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:14.658+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) v1 -- 0x7ff24c09abd0 con 0x7ff24c0b3200 2020-11-16T22:22:15.280 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff232ffd700 1 -- 172.21.15.185:0/3126444580 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v385) v1 ==== 168+0+0 (secure 0 0 0) 0x7ff23c047320 con 0x7ff24c0b3200 2020-11-16T22:22:15.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 msgr2=0x7ff228057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 0x7ff228057520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7ff24c061010 tx=0x7ff24c09b220).stop 2020-11-16T22:22:15.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 msgr2=0x7ff24c143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7ff23c00cac0 tx=0x7ff23c00cdd0).stop 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 shutdown_connections 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff228055060 0x7ff228057520 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff24c0b3200 0x7ff24c143be0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff24c09e380 0x7ff24c13eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 --2- 172.21.15.185:0/3126444580 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff24c142830 0x7ff24c09de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 >> 172.21.15.185:0/3126444580 conn(0x7ff24c060130 msgr2=0x7ff24c141c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:15.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 shutdown_connections 2020-11-16T22:22:15.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.290+0000 7ff25102f700 1 -- 172.21.15.185:0/3126444580 wait complete. 2020-11-16T22:22:15.297 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-16T22:22:15.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 Processor -- start 2020-11-16T22:22:15.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 -- start start 2020-11-16T22:22:15.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 0x7fb8c00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:15.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c00ad490 con 0x7fb8c00b7440 2020-11-16T22:22:15.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c00ad730 con 0x7fb8c00afc60 2020-11-16T22:22:15.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c807a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00ad9d0 con 0x7fb8c00ae580 2020-11-16T22:22:15.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c5615700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb8c00afc60 0x7fb8c00ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56894/0 (socket says 172.21.15.185:56894) 2020-11-16T22:22:15.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c5615700 1 -- 172.21.15.185:0/1817197707 learned_addr learned my addr 172.21.15.185:0/1817197707 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:15.434 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c6617700 1 --2- 172.21.15.185:0/1817197707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 0x7fb8c00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:15.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c6617700 1 -- 172.21.15.185:0/1817197707 >> v1:172.21.15.185:6789/0 conn(0x7fb8c00afc60 legacy=0x7fb8c00ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c6617700 1 -- 172.21.15.185:0/1817197707 >> v1:172.21.15.96:6789/0 conn(0x7fb8c00b7440 legacy=0x7fb8c013dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c4e14700 1 -- 172.21.15.185:0/1817197707 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306173686 0 0) 0x7fb8c00ad730 con 0x7fb8c00afc60 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c6617700 1 -- 172.21.15.185:0/1817197707 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c013e5b0 con 0x7fb8c00ae580 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c6617700 1 --2- 172.21.15.185:0/1817197707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 0x7fb8c00b7820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fb8b8009cc0 tx=0x7fb8b80091d0).ready entity=mon.2 client_cookie=232a7656e4d51fdf server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c4e14700 1 -- 172.21.15.185:0/1817197707 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800bbc0 con 0x7fb8c00ae580 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c4e14700 1 -- 172.21.15.185:0/1817197707 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b800bd20 con 0x7fb8c00ae580 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.442+0000 7fb8c4e14700 1 -- 172.21.15.185:0/1817197707 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8011610 con 0x7fb8c00ae580 2020-11-16T22:22:15.436 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- 172.21.15.185:0/1817197707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 msgr2=0x7fb8c00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1817197707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 0x7fb8c00b7820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7fb8b8009cc0 tx=0x7fb8b80091d0).stop 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- 172.21.15.185:0/1817197707 shutdown_connections 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1817197707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00ae580 0x7fb8c00b7820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- 172.21.15.185:0/1817197707 >> 172.21.15.185:0/1817197707 conn(0x7fb8c0067860 msgr2=0x7fb8c0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- 172.21.15.185:0/1817197707 shutdown_connections 2020-11-16T22:22:15.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- 172.21.15.185:0/1817197707 wait complete. 2020-11-16T22:22:15.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 Processor -- start 2020-11-16T22:22:15.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- start start 2020-11-16T22:22:15.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 0x7fb8c00b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:15.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 0x7fb8c00b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:15.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 0x7fb8c0146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:15.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00b2440 con 0x7fb8c00afc60 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c00b21a0 con 0x7fb8c00ae580 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c807a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0156850 con 0x7fb8c00b7440 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c5e16700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 0x7fb8c00b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c5e16700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 0x7fb8c00b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35228/0 (socket says 172.21.15.185:35228) 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c6617700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 0x7fb8c0146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c5e16700 1 -- 172.21.15.185:0/1193422718 learned_addr learned my addr 172.21.15.185:0/1193422718 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:15.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.446+0000 7fb8c5615700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 0x7fb8c00b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:15.441 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 -- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 msgr2=0x7fb8c00b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 0x7fb8c00b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 -- 172.21.15.185:0/1193422718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 msgr2=0x7fb8c00b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 0x7fb8c00b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 -- 172.21.15.185:0/1193422718 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0156af0 con 0x7fb8c00b7440 2020-11-16T22:22:15.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c5615700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 0x7fb8c00b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:15.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c6617700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 0x7fb8c0146ed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fb8b8013000 tx=0x7fb8b8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8006ea0 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b8011e30 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8010cb0 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c01473c0 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c01479f0 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8c00b7440 2020-11-16T22:22:15.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c005d4e0 con 0x7fb8c00b7440 2020-11-16T22:22:15.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 0x7fb898057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:15.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c5e16700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 0x7fb898057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:15.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb8b806f840 con 0x7fb8c00b7440 2020-11-16T22:22:15.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.450+0000 7fb8c5e16700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 0x7fb898057520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fb8bc0033c0 tx=0x7fb8bc007c50).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:15.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.454+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb8b8045a40 con 0x7fb8c00b7440 2020-11-16T22:22:15.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.610+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fb8c00605a0 con 0x7fb8c00b7440 2020-11-16T22:22:15.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8adffb700 1 -- 172.21.15.185:0/1193422718 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8b8022020 con 0x7fb8c00b7440 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 msgr2=0x7fb898057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 0x7fb898057520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fb8bc0033c0 tx=0x7fb8bc007c50).stop 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 msgr2=0x7fb8c0146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 0x7fb8c0146ed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fb8b8013000 tx=0x7fb8b8016000).stop 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 shutdown_connections 2020-11-16T22:22:15.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb898055060 0x7fb898057520 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8c00ae580 0x7fb8c00b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb8c00b7440 0x7fb8c0146ed0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 --2- 172.21.15.185:0/1193422718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8c00afc60 0x7fb8c00b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:15.856 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 >> 172.21.15.185:0/1193422718 conn(0x7fb8c0067860 msgr2=0x7fb8c00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:16.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 shutdown_connections 2020-11-16T22:22:16.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:15.622+0000 7fb8c807a700 1 -- 172.21.15.185:0/1193422718 wait complete. 2020-11-16T22:22:16.059 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-16T22:22:16.059 INFO:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:16.098 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:16.116 INFO:teuthology.orchestra.run.smithi185:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-16T22:22:16.134 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-16T22:22:16.135 INFO:tasks.cephfs.cephfs_test_case:[, ] 2020-11-16T22:22:16.135 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:16.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 Processor -- start 2020-11-16T22:22:16.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 -- start start 2020-11-16T22:22:16.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb3141448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb3140b34f0 0x7fb3140b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb314068770 con 0x7fb31413de70 2020-11-16T22:22:16.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140b27a0 con 0x7fb3140b34f0 2020-11-16T22:22:16.315 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb314144e70 con 0x7fb314060530 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb3140b34f0 0x7fb3140b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb311d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb3141448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb3140b34f0 0x7fb3140b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35240/0 (socket says 172.21.15.185:35240) 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 -- 172.21.15.185:0/671244309 learned_addr learned my addr 172.21.15.185:0/671244309 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb311d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb3141448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56914/0 (socket says 172.21.15.185:56914) 2020-11-16T22:22:16.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 -- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 msgr2=0x7fb3141448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 --2- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb3141448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 -- 172.21.15.185:0/671244309 >> v1:172.21.15.96:6789/0 conn(0x7fb31413de70 legacy=0x7fb314068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 -- 172.21.15.185:0/671244309 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314067680 con 0x7fb3140b34f0 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb312d9d700 1 --2- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb3140b34f0 0x7fb3140b74f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fb30c009d50 tx=0x7fb30c009290).ready entity=mon.1 client_cookie=f197ce01044fd93d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.322+0000 7fb31159a700 1 -- 172.21.15.185:0/671244309 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00bbc0 con 0x7fb3140b34f0 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31159a700 1 -- 172.21.15.185:0/671244309 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00bd20 con 0x7fb3140b34f0 2020-11-16T22:22:16.317 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31159a700 1 -- 172.21.15.185:0/671244309 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c011610 con 0x7fb3140b34f0 2020-11-16T22:22:16.319 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb3140b34f0 msgr2=0x7fb3140b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb3140b34f0 0x7fb3140b74f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7fb30c009d50 tx=0x7fb30c009290).stop 2020-11-16T22:22:16.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- 172.21.15.185:0/671244309 shutdown_connections 2020-11-16T22:22:16.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb3140b34f0 0x7fb3140b74f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- 172.21.15.185:0/671244309 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb3141448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- 172.21.15.185:0/671244309 >> 172.21.15.185:0/671244309 conn(0x7fb31405ba80 msgr2=0x7fb31405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- 172.21.15.185:0/671244309 shutdown_connections 2020-11-16T22:22:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- 172.21.15.185:0/671244309 wait complete. 2020-11-16T22:22:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 Processor -- start 2020-11-16T22:22:16.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- start start 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb31413f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3140b34f0 0x7fb31413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314140260 con 0x7fb3140b34f0 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314153600 con 0x7fb31413de70 2020-11-16T22:22:16.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3141538a0 con 0x7fb314060530 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb312d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb31259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb31413f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb312d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35246/0 (socket says 172.21.15.185:35246) 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb312d9d700 1 -- 172.21.15.185:0/1292362617 learned_addr learned my addr 172.21.15.185:0/1292362617 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.326+0000 7fb311d9b700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3140b34f0 0x7fb31413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 -- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 msgr2=0x7fb31413f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb31413f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 -- 172.21.15.185:0/1292362617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3140b34f0 msgr2=0x7fb31413fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3140b34f0 0x7fb31413fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 -- 172.21.15.185:0/1292362617 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314153b40 con 0x7fb31413de70 2020-11-16T22:22:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31259c700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb31413f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:16.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb312d9d700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fb30c0092d0 tx=0x7fb30c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00bbc0 con 0x7fb31413de70 2020-11-16T22:22:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00be90 con 0x7fb31413de70 2020-11-16T22:22:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb314153de0 con 0x7fb31413de70 2020-11-16T22:22:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c010cb0 con 0x7fb31413de70 2020-11-16T22:22:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3140af6a0 con 0x7fb31413de70 2020-11-16T22:22:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb314153f70 con 0x7fb31413de70 2020-11-16T22:22:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb30c017070 con 0x7fb31413de70 2020-11-16T22:22:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 0x7fb2e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb30c06fe90 con 0x7fb31413de70 2020-11-16T22:22:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31259c700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 0x7fb2e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.330+0000 7fb31259c700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 0x7fb2e8057520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fb3040016e0 tx=0x7fb304009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.338+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb30c046170 con 0x7fb31413de70 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.538+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31405d200 con 0x7fb31413de70 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb3027fc700 1 -- 172.21.15.185:0/1292362617 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7fb30c015070 con 0x7fb31413de70 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 msgr2=0x7fb2e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 0x7fb2e8057520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fb3040016e0 tx=0x7fb304009040).stop 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 msgr2=0x7fb3140af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7fb30c0092d0 tx=0x7fb30c016000).stop 2020-11-16T22:22:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 shutdown_connections 2020-11-16T22:22:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb2e8055060 0x7fb2e8057520 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb31413de70 0x7fb3140af1b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb314060530 0x7fb31413f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 --2- 172.21.15.185:0/1292362617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3140b34f0 0x7fb31413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 >> 172.21.15.185:0/1292362617 conn(0x7fb31405ba80 msgr2=0x7fb31413d740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:17.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 shutdown_connections 2020-11-16T22:22:17.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:16.542+0000 7fb31359e700 1 -- 172.21.15.185:0/1292362617 wait complete. 2020-11-16T22:22:17.139 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:17.139 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:17.139 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:17.342 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:17.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 Processor -- start 2020-11-16T22:22:17.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 -- start start 2020-11-16T22:22:17.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11581444d0 con 0x7f11580b6b30 2020-11-16T22:22:17.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1158144770 con 0x7f11580b5b30 2020-11-16T22:22:17.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f115dde4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158144a10 con 0x7f115809d260 2020-11-16T22:22:17.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f1156ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f11577fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f11580b5b30 0x7f11581437f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56920/0 (socket says 172.21.15.185:56920) 2020-11-16T22:22:17.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f1156ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56928/0 (socket says 172.21.15.185:56928) 2020-11-16T22:22:17.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f11577fe700 1 -- 172.21.15.185:0/1673368080 learned_addr learned my addr 172.21.15.185:0/1673368080 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:17.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f1156ffd700 1 -- 172.21.15.185:0/1673368080 >> v1:172.21.15.185:6789/0 conn(0x7f11580b5b30 legacy=0x7f11581437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f1156ffd700 1 -- 172.21.15.185:0/1673368080 >> v1:172.21.15.96:6789/0 conn(0x7f11580b6b30 legacy=0x7f115809a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f1156ffd700 1 -- 172.21.15.185:0/1673368080 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158144cb0 con 0x7f115809d260 2020-11-16T22:22:17.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.482+0000 7f11567fc700 1 -- 172.21.15.185:0/1673368080 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301844233 0 0) 0x7f1158144770 con 0x7f11580b5b30 2020-11-16T22:22:17.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f1156ffd700 1 --2- 172.21.15.185:0/1673368080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f115000b9d0 tx=0x7f1150006df0).ready entity=mon.2 client_cookie=7b6a47798f4d9c75 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f11567fc700 1 -- 172.21.15.185:0/1673368080 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150009770 con 0x7f115809d260 2020-11-16T22:22:17.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f11567fc700 1 -- 172.21.15.185:0/1673368080 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11500098d0 con 0x7f115809d260 2020-11-16T22:22:17.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f11567fc700 1 -- 172.21.15.185:0/1673368080 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150009c50 con 0x7f115809d260 2020-11-16T22:22:17.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- 172.21.15.185:0/1673368080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 msgr2=0x7f11580b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 --2- 172.21.15.185:0/1673368080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f115000b9d0 tx=0x7f1150006df0).stop 2020-11-16T22:22:17.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- 172.21.15.185:0/1673368080 shutdown_connections 2020-11-16T22:22:17.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 --2- 172.21.15.185:0/1673368080 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f115809d260 0x7f11580b5f10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- 172.21.15.185:0/1673368080 >> 172.21.15.185:0/1673368080 conn(0x7f1158068330 msgr2=0x7f11580ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:17.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- 172.21.15.185:0/1673368080 shutdown_connections 2020-11-16T22:22:17.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- 172.21.15.185:0/1673368080 wait complete. 2020-11-16T22:22:17.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 Processor -- start 2020-11-16T22:22:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.486+0000 7f115dde4700 1 -- start start 2020-11-16T22:22:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11580b6b30 0x7f11581498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 0x7f1158157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158156a90 con 0x7f1158156ef0 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158147680 con 0x7f11580b5b30 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158147920 con 0x7f11580b6b30 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35258/0 (socket says 172.21.15.185:35258) 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 -- 172.21.15.185:0/3662860218 learned_addr learned my addr 172.21.15.185:0/3662860218 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f1156ffd700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11580b6b30 0x7f11581498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f1157fff700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 0x7f1158157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 -- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11580b6b30 msgr2=0x7f11581498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11580b6b30 0x7f11581498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 -- 172.21.15.185:0/3662860218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 msgr2=0x7f1158157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 0x7f1158157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 -- 172.21.15.185:0/3662860218 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158147bc0 con 0x7f11580b5b30 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f1157fff700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 0x7f1158157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:22:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f11577fe700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f114000c710 tx=0x7f1140007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11400072e0 con 0x7f11580b5b30 2020-11-16T22:22:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140007440 con 0x7f11580b5b30 2020-11-16T22:22:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1158149d90 con 0x7f11580b5b30 2020-11-16T22:22:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140005800 con 0x7f11580b5b30 2020-11-16T22:22:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1158157860 con 0x7f11580b5b30 2020-11-16T22:22:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115805d4e0 con 0x7f11580b5b30 2020-11-16T22:22:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.490+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1140010070 con 0x7f11580b5b30 2020-11-16T22:22:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.494+0000 7f114f7fe700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 0x7f1128057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.494+0000 7f1156ffd700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 0x7f1128057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.494+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f114006d4f0 con 0x7f11580b5b30 2020-11-16T22:22:17.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.494+0000 7f1156ffd700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 0x7f1128057520 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f115000b920 tx=0x7f1150015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.498+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f11400436d0 con 0x7f11580b5b30 2020-11-16T22:22:17.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.654+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1158060530 con 0x7f11580b5b30 2020-11-16T22:22:17.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.658+0000 7f114f7fe700 1 -- 172.21.15.185:0/3662860218 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f1140007c00 con 0x7f11580b5b30 2020-11-16T22:22:17.649 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:17.649 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:15.557108+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:17.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.658+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 msgr2=0x7f1128057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 0x7f1128057520 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f115000b920 tx=0x7f1150015000).stop 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 msgr2=0x7f11580b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f114000c710 tx=0x7f1140007ac0).stop 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 shutdown_connections 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1128055060 0x7f1128057520 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f11580b5b30 0x7f11580b4340 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1158156ef0 0x7f1158157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 --2- 172.21.15.185:0/3662860218 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f11580b6b30 0x7f11581498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 >> 172.21.15.185:0/3662860218 conn(0x7f1158068330 msgr2=0x7f115809ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 shutdown_connections 2020-11-16T22:22:17.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.662+0000 7f115dde4700 1 -- 172.21.15.185:0/3662860218 wait complete. 2020-11-16T22:22:17.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:21:11.777337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:21:12.541132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:22:17.670 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:22:17.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.806+0000 7f4e2f59e700 1 Processor -- start 2020-11-16T22:22:17.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2f59e700 1 -- start start 2020-11-16T22:22:17.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 0x7f4e30143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2f59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300b7290 con 0x7f4e300ba800 2020-11-16T22:22:17.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300b4bf0 con 0x7f4e300b5d70 2020-11-16T22:22:17.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b74c0 con 0x7f4e300b4470 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 0x7f4e30143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4e300b5d70 0x7f4e300b7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56934/0 (socket says 172.21.15.185:56934) 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2e59c700 1 -- 172.21.15.185:0/2744772622 learned_addr learned my addr 172.21.15.185:0/2744772622 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2d59a700 1 -- 172.21.15.185:0/2744772622 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196528292 0 0) 0x7f4e300b4bf0 con 0x7f4e300b5d70 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2d59a700 1 -- 172.21.15.185:0/2744772622 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e14003240 con 0x7f4e300b5d70 2020-11-16T22:22:17.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2ed9d700 1 -- 172.21.15.185:0/2744772622 >> v1:172.21.15.185:6789/0 conn(0x7f4e300b5d70 legacy=0x7f4e300b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2ed9d700 1 -- 172.21.15.185:0/2744772622 >> v1:172.21.15.96:6789/0 conn(0x7f4e300ba800 legacy=0x7f4e30143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2ed9d700 1 -- 172.21.15.185:0/2744772622 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300b7760 con 0x7f4e300b4470 2020-11-16T22:22:17.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2ed9d700 1 --2- 172.21.15.185:0/2744772622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 0x7f4e30143b80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f4e28009cc0 tx=0x7f4e280091d0).ready entity=mon.2 client_cookie=4c9578f0a0195c2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2d59a700 1 -- 172.21.15.185:0/2744772622 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800bad0 con 0x7f4e300b4470 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2d59a700 1 -- 172.21.15.185:0/2744772622 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800bc30 con 0x7f4e300b4470 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.810+0000 7f4e2d59a700 1 -- 172.21.15.185:0/2744772622 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28011400 con 0x7f4e300b4470 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- 172.21.15.185:0/2744772622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 msgr2=0x7f4e30143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/2744772622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 0x7f4e30143b80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f4e28009cc0 tx=0x7f4e280091d0).stop 2020-11-16T22:22:17.805 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- 172.21.15.185:0/2744772622 shutdown_connections 2020-11-16T22:22:17.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/2744772622 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300b4470 0x7f4e30143b80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- 172.21.15.185:0/2744772622 >> 172.21.15.185:0/2744772622 conn(0x7f4e3005ba80 msgr2=0x7f4e3005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:17.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- 172.21.15.185:0/2744772622 shutdown_connections 2020-11-16T22:22:17.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- 172.21.15.185:0/2744772622 wait complete. 2020-11-16T22:22:17.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 Processor -- start 2020-11-16T22:22:17.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.814+0000 7f4e2f59e700 1 -- start start 2020-11-16T22:22:17.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 0x7f4e30153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300ba800 0x7f4e30153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300af690 con 0x7f4e300b5d70 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30154060 con 0x7f4e300b4470 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301571e0 con 0x7f4e300ba800 2020-11-16T22:22:17.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35272/0 (socket says 172.21.15.185:35272) 2020-11-16T22:22:17.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 -- 172.21.15.185:0/1760811256 learned_addr learned my addr 172.21.15.185:0/1760811256 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:17.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2ed9d700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300ba800 0x7f4e30153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.809 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2dd9b700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 0x7f4e30153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 -- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300ba800 msgr2=0x7f4e30153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300ba800 0x7f4e30153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 -- 172.21.15.185:0/1760811256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 msgr2=0x7f4e30153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 0x7f4e30153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 -- 172.21.15.185:0/1760811256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e30157480 con 0x7f4e300b4470 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2dd9b700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 0x7f4e30153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2e59c700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4e2000c720 tx=0x7f4e20007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200072e0 con 0x7f4e300b4470 2020-11-16T22:22:17.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3014b100 con 0x7f4e300b4470 2020-11-16T22:22:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20007440 con 0x7f4e300b4470 2020-11-16T22:22:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20005800 con 0x7f4e300b4470 2020-11-16T22:22:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e300afe20 con 0x7f4e300b4470 2020-11-16T22:22:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3005d200 con 0x7f4e300b4470 2020-11-16T22:22:17.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4e20010070 con 0x7f4e300b4470 2020-11-16T22:22:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.818+0000 7f4e1e7fc700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.822+0000 7f4e2dd9b700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.822+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e2006d630 con 0x7f4e300b4470 2020-11-16T22:22:17.813 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.822+0000 7f4e2dd9b700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f4e30147910 tx=0x7f4e24009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:17.820 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.830+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4e20043890 con 0x7f4e300b4470 2020-11-16T22:22:17.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.978+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4e300b4920 con 0x7f4e300b4470 2020-11-16T22:22:17.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e1e7fc700 1 -- 172.21.15.185:0/1760811256 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4e20046eb0 con 0x7f4e300b4470 2020-11-16T22:22:17.972 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:22:17.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 msgr2=0x7f4e04057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f4e30147910 tx=0x7f4e24009040).stop 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 msgr2=0x7f4e3014ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f4e2000c720 tx=0x7f4e20007a00).stop 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 shutdown_connections 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e300b4470 0x7f4e3014ac10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e300ba800 0x7f4e30153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 --2- 172.21.15.185:0/1760811256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e300b5d70 0x7f4e30153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:17.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 >> 172.21.15.185:0/1760811256 conn(0x7f4e3005ba80 msgr2=0x7f4e30147a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:17.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 shutdown_connections 2020-11-16T22:22:17.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:17.982+0000 7f4e2f59e700 1 -- 172.21.15.185:0/1760811256 wait complete. 2020-11-16T22:22:17.986 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:22:18.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 Processor -- start 2020-11-16T22:22:18.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 -- start start 2020-11-16T22:22:18.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87d4143470 0x7f87d4143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40b7290 con 0x7f87d40b4470 2020-11-16T22:22:18.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d40b4bf0 con 0x7f87d4143470 2020-11-16T22:22:18.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40b74c0 con 0x7f87d40b5d70 2020-11-16T22:22:18.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d259c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56954/0 (socket says 172.21.15.185:56954) 2020-11-16T22:22:18.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d2d9d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87d4143470 0x7f87d4143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.126+0000 7f87d259c700 1 -- 172.21.15.185:0/426329081 learned_addr learned my addr 172.21.15.185:0/426329081 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:18.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d259c700 1 -- 172.21.15.185:0/426329081 >> v2:172.21.15.96:3300/0 conn(0x7f87d4143470 msgr2=0x7f87d4143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:18.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d259c700 1 --2- 172.21.15.185:0/426329081 >> v2:172.21.15.96:3300/0 conn(0x7f87d4143470 0x7f87d4143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d259c700 1 -- 172.21.15.185:0/426329081 >> v1:172.21.15.96:6789/0 conn(0x7f87d40b4470 legacy=0x7f87d40b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d259c700 1 -- 172.21.15.185:0/426329081 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40b7760 con 0x7f87d40b5d70 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d259c700 1 --2- 172.21.15.185:0/426329081 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f87c4006fd0 tx=0x7f87c4006eb0).ready entity=mon.2 client_cookie=2dcb12aca7425da2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d159a700 1 -- 172.21.15.185:0/426329081 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009810 con 0x7f87d40b5d70 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d159a700 1 -- 172.21.15.185:0/426329081 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4009970 con 0x7f87d40b5d70 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d159a700 1 -- 172.21.15.185:0/426329081 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009cf0 con 0x7f87d40b5d70 2020-11-16T22:22:18.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- 172.21.15.185:0/426329081 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 msgr2=0x7f87d40ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 --2- 172.21.15.185:0/426329081 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f87c4006fd0 tx=0x7f87c4006eb0).stop 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- 172.21.15.185:0/426329081 shutdown_connections 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 --2- 172.21.15.185:0/426329081 >> v2:172.21.15.96:3300/0 conn(0x7f87d4143470 0x7f87d4143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 --2- 172.21.15.185:0/426329081 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d40b5d70 0x7f87d40ba800 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- 172.21.15.185:0/426329081 >> 172.21.15.185:0/426329081 conn(0x7f87d405ba80 msgr2=0x7f87d405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- 172.21.15.185:0/426329081 shutdown_connections 2020-11-16T22:22:18.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- 172.21.15.185:0/426329081 wait complete. 2020-11-16T22:22:18.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 Processor -- start 2020-11-16T22:22:18.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.130+0000 7f87d359e700 1 -- start start 2020-11-16T22:22:18.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 0x7f87d41533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4156a50 con 0x7f87d40b4470 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40aeb60 con 0x7f87d40b5d70 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4156cb0 con 0x7f87d4143470 2020-11-16T22:22:18.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d2d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56962/0 (socket says 172.21.15.185:56962) 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d2d9d700 1 -- 172.21.15.185:0/2788419213 learned_addr learned my addr 172.21.15.185:0/2788419213 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35288/0 (socket says 172.21.15.185:35288) 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d259c700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 0x7f87d41533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 -- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 msgr2=0x7f87d4157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 -- 172.21.15.185:0/2788419213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 msgr2=0x7f87d41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 0x7f87d41533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 -- 172.21.15.185:0/2788419213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40aee00 con 0x7f87d40b5d70 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d259c700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 0x7f87d41533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d2d9d700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d1d9b700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f87bc00c710 tx=0x7f87bc007ac0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc013410 con 0x7f87d40b5d70 2020-11-16T22:22:18.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc013570 con 0x7f87d40b5d70 2020-11-16T22:22:18.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d4114d60 con 0x7f87d40b5d70 2020-11-16T22:22:18.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0138f0 con 0x7f87d40b5d70 2020-11-16T22:22:18.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d40ae450 con 0x7f87d40b5d70 2020-11-16T22:22:18.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.134+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d41538d0 con 0x7f87d40b5d70 2020-11-16T22:22:18.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.138+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f87bc010030 con 0x7f87d40b5d70 2020-11-16T22:22:18.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.138+0000 7f87ca7fc700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 0x7f87b0057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.138+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f87bc06da40 con 0x7f87d40b5d70 2020-11-16T22:22:18.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.138+0000 7f87d259c700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 0x7f87b0057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.138+0000 7f87d259c700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 0x7f87b0057550 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f87c400b1b0 tx=0x7f87c4015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.146+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f87d41538d0 con 0x7f87d40b5d70 2020-11-16T22:22:18.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.298+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87d41538d0 con 0x7f87d40b5d70 2020-11-16T22:22:18.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.298+0000 7f87ca7fc700 1 -- 172.21.15.185:0/2788419213 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87d41538d0 con 0x7f87d40b5d70 2020-11-16T22:22:18.291 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:18.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 msgr2=0x7f87b0057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 0x7f87b0057550 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f87c400b1b0 tx=0x7f87c4015000).stop 2020-11-16T22:22:18.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 msgr2=0x7f87d41496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f87bc00c710 tx=0x7f87bc007ac0).stop 2020-11-16T22:22:18.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 shutdown_connections 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f87b0055090 0x7f87b0057550 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f87d4143470 0x7f87d4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d40b4470 0x7f87d41533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 --2- 172.21.15.185:0/2788419213 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f87d40b5d70 0x7f87d41496e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 >> 172.21.15.185:0/2788419213 conn(0x7f87d405ba80 msgr2=0x7f87d4143920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 shutdown_connections 2020-11-16T22:22:18.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.302+0000 7f87d359e700 1 -- 172.21.15.185:0/2788419213 wait complete. 2020-11-16T22:22:18.305 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:22:18.432 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 Processor -- start 2020-11-16T22:22:18.433 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 -- start start 2020-11-16T22:22:18.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f88f4063550 0x7f88f4063930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f88f40b3e10 0x7f88f4144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b26a0 con 0x7f88f40b2070 2020-11-16T22:22:18.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f41452b0 con 0x7f88f4063550 2020-11-16T22:22:18.435 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.442+0000 7f88fac53700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40b2940 con 0x7f88f40b3e10 2020-11-16T22:22:18.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f3fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f88f40b2070 0x7f88f40b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56962/0 (socket says 172.21.15.185:56962) 2020-11-16T22:22:18.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f88f40b3e10 0x7f88f4144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f3fff700 1 -- 172.21.15.185:0/3200730644 learned_addr learned my addr 172.21.15.185:0/3200730644 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:18.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f89ef700 1 --2- 172.21.15.185:0/3200730644 >> v2:172.21.15.96:3300/0 conn(0x7f88f4063550 0x7f88f4063930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.437 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f37fe700 1 -- 172.21.15.185:0/3200730644 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873406828 0 0) 0x7f88f40b26a0 con 0x7f88f40b2070 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f37fe700 1 -- 172.21.15.185:0/3200730644 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e0003240 con 0x7f88f40b2070 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 -- 172.21.15.185:0/3200730644 >> v2:172.21.15.96:3300/0 conn(0x7f88f4063550 msgr2=0x7f88f4063930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 --2- 172.21.15.185:0/3200730644 >> v2:172.21.15.96:3300/0 conn(0x7f88f4063550 0x7f88f4063930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 -- 172.21.15.185:0/3200730644 >> v1:172.21.15.185:6789/0 conn(0x7f88f40b2070 legacy=0x7f88f40b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 -- 172.21.15.185:0/3200730644 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40b2be0 con 0x7f88f40b3e10 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 --2- 172.21.15.185:0/3200730644 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f40b3e10 0x7f88f4144dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f88f40b1f90 tx=0x7f88ec0091d0).ready entity=mon.1 client_cookie=e5330a22def452b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f37fe700 1 -- 172.21.15.185:0/3200730644 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00b9c0 con 0x7f88f40b3e10 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f37fe700 1 -- 172.21.15.185:0/3200730644 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec00bb20 con 0x7f88f40b3e10 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f37fe700 1 -- 172.21.15.185:0/3200730644 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00bea0 con 0x7f88f40b3e10 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- 172.21.15.185:0/3200730644 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f40b3e10 msgr2=0x7f88f4144dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- 172.21.15.185:0/3200730644 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f40b3e10 0x7f88f4144dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f88f40b1f90 tx=0x7f88ec0091d0).stop 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- 172.21.15.185:0/3200730644 shutdown_connections 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- 172.21.15.185:0/3200730644 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f40b3e10 0x7f88f4144dc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- 172.21.15.185:0/3200730644 >> v2:172.21.15.96:3300/0 conn(0x7f88f4063550 0x7f88f4063930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- 172.21.15.185:0/3200730644 >> 172.21.15.185:0/3200730644 conn(0x7f88f40649a0 msgr2=0x7f88f4062e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- 172.21.15.185:0/3200730644 shutdown_connections 2020-11-16T22:22:18.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- 172.21.15.185:0/3200730644 wait complete. 2020-11-16T22:22:18.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 Processor -- start 2020-11-16T22:22:18.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- start start 2020-11-16T22:22:18.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 0x7f88f4147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 0x7f88f413c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f413ee10 con 0x7f88f4063550 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4153eb0 con 0x7f88f4140490 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88fac53700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f413f9c0 con 0x7f88f40b2070 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 0x7f88f413c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.446+0000 7f88f91f0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f89ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 0x7f88f4147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 0x7f88f413c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56976/0 (socket says 172.21.15.185:56976) 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35300/0 (socket says 172.21.15.185:35300) 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f3fff700 1 -- 172.21.15.185:0/1066687152 learned_addr learned my addr 172.21.15.185:0/1066687152 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 -- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 msgr2=0x7f88f413c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 0x7f88f413c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 -- 172.21.15.185:0/1066687152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 msgr2=0x7f88f4147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 0x7f88f4147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 -- 172.21.15.185:0/1066687152 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f413fc60 con 0x7f88f4140490 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f89ef700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 0x7f88f4147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f91f0700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f88f40b1f90 tx=0x7f88ec018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec01d070 con 0x7f88f4140490 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec018610 con 0x7f88f4140490 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f4153bd0 con 0x7f88f4140490 2020-11-16T22:22:18.445 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec018990 con 0x7f88f4140490 2020-11-16T22:22:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f413d340 con 0x7f88f4140490 2020-11-16T22:22:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f4154070 con 0x7f88f4140490 2020-11-16T22:22:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f88ec00f3d0 con 0x7f88f4140490 2020-11-16T22:22:18.446 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 0x7f88cc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f89ef700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 0x7f88cc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.450+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f88ec06fb50 con 0x7f88f4140490 2020-11-16T22:22:18.447 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.454+0000 7f88f89ef700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 0x7f88cc057520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f88e8006fd0 tx=0x7f88e8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.458+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f88ec045dd0 con 0x7f88f4140490 2020-11-16T22:22:18.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.614+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88f413fdf0 con 0x7f88f4140490 2020-11-16T22:22:18.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88f0ff9700 1 -- 172.21.15.185:0/1066687152 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f88ec027070 con 0x7f88f4140490 2020-11-16T22:22:18.608 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:18.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 msgr2=0x7f88cc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 0x7f88cc057520 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f88e8006fd0 tx=0x7f88e8006df0).stop 2020-11-16T22:22:18.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 msgr2=0x7f88f413ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f88f40b1f90 tx=0x7f88ec018000).stop 2020-11-16T22:22:18.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 shutdown_connections 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f88cc055060 0x7f88cc057520 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f88f4140490 0x7f88f413ce50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88f4063550 0x7f88f4147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 --2- 172.21.15.185:0/1066687152 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f88f40b2070 0x7f88f413c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 >> 172.21.15.185:0/1066687152 conn(0x7f88f40649a0 msgr2=0x7f88f40b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 shutdown_connections 2020-11-16T22:22:18.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.618+0000 7f88fac53700 1 -- 172.21.15.185:0/1066687152 wait complete. 2020-11-16T22:22:18.623 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:22:18.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.758+0000 7fbdad9a5700 1 Processor -- start 2020-11-16T22:22:18.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.758+0000 7fbdad9a5700 1 -- start start 2020-11-16T22:22:18.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbda80ac820 0x7fbda813f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fbda8060130 0x7fbda813fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda8060970 con 0x7fbda80b3830 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80b4610 con 0x7fbda80ac820 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda8060ba0 con 0x7fbda8060130 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdac9a3700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fbda80b3830 0x7fbda8143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56974/0 (socket says 172.21.15.185:56974) 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdac9a3700 1 -- 172.21.15.185:0/547563645 learned_addr learned my addr 172.21.15.185:0/547563645 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 --2- 172.21.15.185:0/547563645 >> v2:172.21.15.185:3300/0 conn(0x7fbda8060130 0x7fbda813fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda7fff700 1 --2- 172.21.15.185:0/547563645 >> v2:172.21.15.96:3300/0 conn(0x7fbda80ac820 0x7fbda813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752192968 0 0) 0x7fbda8060970 con 0x7fbda80b3830 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd94003240 con 0x7fbda80b3830 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 -- 172.21.15.185:0/547563645 >> v2:172.21.15.96:3300/0 conn(0x7fbda80ac820 msgr2=0x7fbda813f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 --2- 172.21.15.185:0/547563645 >> v2:172.21.15.96:3300/0 conn(0x7fbda80ac820 0x7fbda813f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 -- 172.21.15.185:0/547563645 >> v1:172.21.15.185:6789/0 conn(0x7fbda80b3830 legacy=0x7fbda8143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1890682837 0 0) 0x7fbd94003240 con 0x7fbda80b3830 2020-11-16T22:22:18.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 -- 172.21.15.185:0/547563645 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8060e40 con 0x7fbda8060130 2020-11-16T22:22:18.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad1a4700 1 --2- 172.21.15.185:0/547563645 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda8060130 0x7fbda813fd30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fbda80b6c70 tx=0x7fbda0006e80).ready entity=mon.1 client_cookie=9813d3639ae96ccc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0014070 con 0x7fbda8060130 2020-11-16T22:22:18.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda000a5b0 con 0x7fbda8060130 2020-11-16T22:22:18.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbda77fe700 1 -- 172.21.15.185:0/547563645 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98001860 con 0x7fbda8060130 2020-11-16T22:22:18.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- 172.21.15.185:0/547563645 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda8060130 msgr2=0x7fbda813fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/547563645 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda8060130 0x7fbda813fd30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fbda80b6c70 tx=0x7fbda0006e80).stop 2020-11-16T22:22:18.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- 172.21.15.185:0/547563645 shutdown_connections 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/547563645 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda8060130 0x7fbda813fd30 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/547563645 >> v2:172.21.15.96:3300/0 conn(0x7fbda80ac820 0x7fbda813f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- 172.21.15.185:0/547563645 >> 172.21.15.185:0/547563645 conn(0x7fbda8063df0 msgr2=0x7fbda80641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- 172.21.15.185:0/547563645 shutdown_connections 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.762+0000 7fbdad9a5700 1 -- 172.21.15.185:0/547563645 wait complete. 2020-11-16T22:22:18.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 Processor -- start 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- start start 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 0x7fbda81114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda80b3830 0x7fbda8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8114730 con 0x7fbda80ac820 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80b0ef0 con 0x7fbda80b3830 2020-11-16T22:22:18.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8133400 con 0x7fbda8060130 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad1a4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda80b3830 0x7fbda8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:56986/0 (socket says 172.21.15.185:56986) 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 -- 172.21.15.185:0/1914054648 learned_addr learned my addr 172.21.15.185:0/1914054648 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbda7fff700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 0x7fbda81114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 -- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda80b3830 msgr2=0x7fbda8154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda80b3830 0x7fbda8154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 -- 172.21.15.185:0/1914054648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 msgr2=0x7fbda81114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 0x7fbda81114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 -- 172.21.15.185:0/1914054648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda81336a0 con 0x7fbda8060130 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbda7fff700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 0x7fbda81114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdac9a3700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fbd9800ce30 tx=0x7fbd98001600).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800aac0 con 0x7fbda8060130 2020-11-16T22:22:18.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800ac20 con 0x7fbda8060130 2020-11-16T22:22:18.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda81159f0 con 0x7fbda8060130 2020-11-16T22:22:18.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98007170 con 0x7fbda8060130 2020-11-16T22:22:18.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.766+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda81138d0 con 0x7fbda8060130 2020-11-16T22:22:18.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbd9801b060 con 0x7fbda8060130 2020-11-16T22:22:18.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda805d200 con 0x7fbda8060130 2020-11-16T22:22:18.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbda4ff9700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 0x7fbd80057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:18.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbda7fff700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 0x7fbd80057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:18.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbd9806de80 con 0x7fbda8060130 2020-11-16T22:22:18.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.770+0000 7fbda7fff700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 0x7fbd80057520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fbda8140b50 tx=0x7fbd9c008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:18.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.774+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbd98043fd0 con 0x7fbda8060130 2020-11-16T22:22:18.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.922+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbda8061e00 con 0x7fbda8060130 2020-11-16T22:22:18.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbda4ff9700 1 -- 172.21.15.185:0/1914054648 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbd98019070 con 0x7fbda8060130 2020-11-16T22:22:18.917 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:18.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 msgr2=0x7fbd80057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 0x7fbd80057520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fbda8140b50 tx=0x7fbd9c008040).stop 2020-11-16T22:22:18.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 msgr2=0x7fbda80b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:18.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fbd9800ce30 tx=0x7fbd98001600).stop 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 shutdown_connections 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbd80055060 0x7fbd80057520 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbda80b3830 0x7fbda8154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbda8060130 0x7fbda80b0850 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 --2- 172.21.15.185:0/1914054648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda80ac820 0x7fbda81114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:18.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 >> 172.21.15.185:0/1914054648 conn(0x7fbda8063df0 msgr2=0x7fbda81402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:18.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 shutdown_connections 2020-11-16T22:22:18.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:18.926+0000 7fbdad9a5700 1 -- 172.21.15.185:0/1914054648 wait complete. 2020-11-16T22:22:18.930 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:19.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 Processor -- start 2020-11-16T22:22:19.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- start start 2020-11-16T22:22:19.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 0x7f5510144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55100b3020 con 0x7f55100b7610 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55100b3280 con 0x7f5510142f30 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b3520 con 0x7f5510067910 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 0x7f5510144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f5514dfb700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5510142f30 0x7f5510143380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:56992/0 (socket says 172.21.15.185:56992) 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f5514dfb700 1 -- 172.21.15.185:0/2449627221 learned_addr learned my addr 172.21.15.185:0/2449627221 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:19.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550effd700 1 -- 172.21.15.185:0/2449627221 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556642854 0 0) 0x7f55100b3280 con 0x7f5510142f30 2020-11-16T22:22:19.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550effd700 1 -- 172.21.15.185:0/2449627221 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003240 con 0x7f5510142f30 2020-11-16T22:22:19.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550f7fe700 1 -- 172.21.15.185:0/2449627221 >> v1:172.21.15.185:6789/0 conn(0x7f5510142f30 legacy=0x7f5510143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550f7fe700 1 -- 172.21.15.185:0/2449627221 >> v1:172.21.15.96:6789/0 conn(0x7f55100b7610 legacy=0x7f55101441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550f7fe700 1 -- 172.21.15.185:0/2449627221 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510067f40 con 0x7f5510067910 2020-11-16T22:22:19.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550f7fe700 1 --2- 172.21.15.185:0/2449627221 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 0x7f5510144900 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f550400b9d0 tx=0x7f5504006df0).ready entity=mon.2 client_cookie=b34fae291b2a17f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:19.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550effd700 1 -- 172.21.15.185:0/2449627221 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040097b0 con 0x7f5510067910 2020-11-16T22:22:19.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550effd700 1 -- 172.21.15.185:0/2449627221 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504009910 con 0x7f5510067910 2020-11-16T22:22:19.057 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f550effd700 1 -- 172.21.15.185:0/2449627221 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504009c90 con 0x7f5510067910 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- 172.21.15.185:0/2449627221 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 msgr2=0x7f5510144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 --2- 172.21.15.185:0/2449627221 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 0x7f5510144900 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f550400b9d0 tx=0x7f5504006df0).stop 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- 172.21.15.185:0/2449627221 shutdown_connections 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 --2- 172.21.15.185:0/2449627221 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510067910 0x7f5510144900 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- 172.21.15.185:0/2449627221 >> 172.21.15.185:0/2449627221 conn(0x7f55100637b0 msgr2=0x7f55100b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:19.058 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- 172.21.15.185:0/2449627221 shutdown_connections 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.062+0000 7f55155fc700 1 -- 172.21.15.185:0/2449627221 wait complete. 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 Processor -- start 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- start start 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100b7610 0x7f55100ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510142f30 0x7f551014aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.059 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b12f0 con 0x7f55100b7610 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b1590 con 0x7f5510146ab0 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100abf90 con 0x7f5510142f30 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f550f7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510142f30 0x7f551014aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35326/0 (socket says 172.21.15.185:35326) 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 -- 172.21.15.185:0/675626055 learned_addr learned my addr 172.21.15.185:0/675626055 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:19.060 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f550ffff700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100b7610 0x7f55100ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 -- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510142f30 msgr2=0x7f551014aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510142f30 0x7f551014aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 -- 172.21.15.185:0/675626055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100b7610 msgr2=0x7f55100ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100b7610 0x7f55100ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 -- 172.21.15.185:0/675626055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100ac920 con 0x7f5510146ab0 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f5514dfb700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f550000ca90 tx=0x7f5500007d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:19.061 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500005b80 con 0x7f5510146ab0 2020-11-16T22:22:19.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500005ce0 con 0x7f5510146ab0 2020-11-16T22:22:19.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551014af90 con 0x7f5510146ab0 2020-11-16T22:22:19.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500007320 con 0x7f5510146ab0 2020-11-16T22:22:19.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55100ae1e0 con 0x7f5510146ab0 2020-11-16T22:22:19.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.066+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100ac150 con 0x7f5510146ab0 2020-11-16T22:22:19.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.070+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5500010030 con 0x7f5510146ab0 2020-11-16T22:22:19.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.070+0000 7f54effff700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 0x7f54e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.070+0000 7f550ffff700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 0x7f54e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.070+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f550006dd50 con 0x7f5510146ab0 2020-11-16T22:22:19.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.070+0000 7f550ffff700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 0x7f54e8057520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f5508007fa0 tx=0x7f5508009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:19.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.074+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5500043fb0 con 0x7f5510146ab0 2020-11-16T22:22:19.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.278+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55100b0e50 con 0x7f5510146ab0 2020-11-16T22:22:19.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.278+0000 7f54effff700 1 -- 172.21.15.185:0/675626055 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+2527 (secure 0 0 0) 0x7f55000475d0 con 0x7f5510146ab0 2020-11-16T22:22:19.269 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:19.269 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:21:13.541645+0000","modified":"2020-11-16T22:21:57.277223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7083,"mds_1":7080,"mds_2":6545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6545":{"gid":6545,"name":"a","rank":2,"incarnation":87,"state":"up:active","state_seq":13,"addr":"172.21.15.185:6835/2224996985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":2224996985},{"type":"v1","addr":"172.21.15.185:6835","nonce":2224996985}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7080":{"gid":7080,"name":"b","rank":1,"incarnation":85,"state":"up:active","state_seq":14,"addr":"172.21.15.96:6833/862887203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":862887203},{"type":"v1","addr":"172.21.15.96:6833","nonce":862887203}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7083":{"gid":7083,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":13,"addr":"172.21.15.96:6835/3098481638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3098481638},{"type":"v1","addr":"172.21.15.96:6835","nonce":3098481638}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-16T22:22:19.271 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.278+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 msgr2=0x7f54e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 0x7f54e8057520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f5508007fa0 tx=0x7f5508009040).stop 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 msgr2=0x7f55100adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f550000ca90 tx=0x7f5500007d60).stop 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 shutdown_connections 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f54e8055060 0x7f54e8057520 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.272 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5510146ab0 0x7f55100adcf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100b7610 0x7f55100ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 --2- 172.21.15.185:0/675626055 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5510142f30 0x7f551014aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 >> 172.21.15.185:0/675626055 conn(0x7f55100637b0 msgr2=0x7f55101401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:19.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 shutdown_connections 2020-11-16T22:22:19.273 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.282+0000 7f55155fc700 1 -- 172.21.15.185:0/675626055 wait complete. 2020-11-16T22:22:19.274 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 93 2020-11-16T22:22:19.284 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:19.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.414+0000 7f57a2d7c700 1 Processor -- start 2020-11-16T22:22:19.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.414+0000 7f57a2d7c700 1 -- start start 2020-11-16T22:22:19.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f579c0b1900 0x7f579c0ac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.409 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c0670a0 con 0x7f579c0abef0 2020-11-16T22:22:19.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c067560 con 0x7f579c0b1900 2020-11-16T22:22:19.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c067800 con 0x7f579c066c10 2020-11-16T22:22:19.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1579700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f579c0abef0 0x7f579c0681b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57002/0 (socket says 172.21.15.185:57002) 2020-11-16T22:22:19.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1579700 1 -- 172.21.15.185:0/3335631819 learned_addr learned my addr 172.21.15.185:0/3335631819 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:19.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/3335631819 >> v2:172.21.15.185:3300/0 conn(0x7f579c0b1900 0x7f579c0ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a257b700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064032968 0 0) 0x7f579c0670a0 con 0x7f579c0abef0 2020-11-16T22:22:19.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5784003240 con 0x7f579c0abef0 2020-11-16T22:22:19.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 -- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 msgr2=0x7f579c13f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:19.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c13f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.412 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 -- 172.21.15.185:0/3335631819 >> v1:172.21.15.185:6789/0 conn(0x7f579c0abef0 legacy=0x7f579c0681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2928543067 0 0) 0x7f5784003240 con 0x7f579c0abef0 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 -- 172.21.15.185:0/3335631819 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c144560 con 0x7f579c0b1900 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0b1900 0x7f579c0ac570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f579000dc30 tx=0x7f5790006d80).ready entity=mon.1 client_cookie=4b72e8d59c5de234 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980019e0 con 0x7f579c0b1900 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000b940 con 0x7f579c0b1900 2020-11-16T22:22:19.413 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a0d78700 1 -- 172.21.15.185:0/3335631819 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000bcc0 con 0x7f579c0b1900 2020-11-16T22:22:19.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 -- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0b1900 msgr2=0x7f579c0ac570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0b1900 0x7f579c0ac570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f579000dc30 tx=0x7f5790006d80).stop 2020-11-16T22:22:19.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 -- 172.21.15.185:0/3335631819 shutdown_connections 2020-11-16T22:22:19.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.418+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c13f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/3335631819 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0b1900 0x7f579c0ac570 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- 172.21.15.185:0/3335631819 >> 172.21.15.185:0/3335631819 conn(0x7f579c065e30 msgr2=0x7f579c066240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:19.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- 172.21.15.185:0/3335631819 shutdown_connections 2020-11-16T22:22:19.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- 172.21.15.185:0/3335631819 wait complete. 2020-11-16T22:22:19.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 Processor -- start 2020-11-16T22:22:19.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- start start 2020-11-16T22:22:19.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0abef0 0x7f579c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 0x7f579c140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b1240 con 0x7f579c0b1900 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c153390 con 0x7f579c0abef0 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a2d7c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c153630 con 0x7f579c066c10 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1579700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0abef0 0x7f579c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57012/0 (socket says 172.21.15.185:57012) 2020-11-16T22:22:19.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 -- 172.21.15.185:0/1639488958 learned_addr learned my addr 172.21.15.185:0/1639488958 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:19.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a257b700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 0x7f579c140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:19.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 -- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0abef0 msgr2=0x7f579c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0abef0 0x7f579c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.826 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 -- 172.21.15.185:0/1639488958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 msgr2=0x7f579c140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 0x7f579c140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 -- 172.21.15.185:0/1639488958 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c1538d0 con 0x7f579c066c10 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a257b700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 0x7f579c140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.422+0000 7f57a1d7a700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f579c065990 tx=0x7f5790018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790015e40 con 0x7f579c066c10 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579001d420 con 0x7f579c066c10 2020-11-16T22:22:19.827 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c141040 con 0x7f579c066c10 2020-11-16T22:22:19.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579001d7c0 con 0x7f579c066c10 2020-11-16T22:22:19.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c143f50 con 0x7f579c066c10 2020-11-16T22:22:19.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c0b1090 con 0x7f579c066c10 2020-11-16T22:22:19.828 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f579001dbf0 con 0x7f579c066c10 2020-11-16T22:22:20.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 0x7f5774057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f57a1579700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 0x7f5774057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f579006f760 con 0x7f579c066c10 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.426+0000 7f57a1579700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 0x7f5774057520 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f579c0b0880 tx=0x7f57980016f0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.434+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f57900458b0 con 0x7f579c066c10 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f579c0b1090 con 0x7f579c066c10 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f578dffb700 1 -- 172.21.15.185:0/1639488958 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f5790048ed0 con 0x7f579c066c10 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 msgr2=0x7f5774057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 0x7f5774057520 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f579c0b0880 tx=0x7f57980016f0).stop 2020-11-16T22:22:20.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 msgr2=0x7f579c153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f579c065990 tx=0x7f5790018000).stop 2020-11-16T22:22:20.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 shutdown_connections 2020-11-16T22:22:20.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5774055060 0x7f5774057520 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f579c0b1900 0x7f579c140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f579c066c10 0x7f579c153d70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 --2- 172.21.15.185:0/1639488958 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f579c0abef0 0x7f579c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 >> 172.21.15.185:0/1639488958 conn(0x7f579c065e30 msgr2=0x7f579c13eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:20.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 shutdown_connections 2020-11-16T22:22:20.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:19.582+0000 7f57a2d7c700 1 -- 172.21.15.185:0/1639488958 wait complete. 2020-11-16T22:22:20.451 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:20.451 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":74,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:21:15.557108+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:21:11.777337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2020-11-16T22:21:12.541132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:20.654 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-16T22:22:20.655 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-16T22:22:20.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 Processor -- start 2020-11-16T22:22:20.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- start start 2020-11-16T22:22:20.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f31c013f9b0 0x7f31c013fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c00b3b70 con 0x7f31c00b2450 2020-11-16T22:22:20.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c0067990 con 0x7f31c013f9b0 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c0144500 con 0x7f31c00670c0 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f31c013f9b0 0x7f31c013fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f31c013f9b0 0x7f31c013fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35348/0 (socket says 172.21.15.185:35348) 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 -- 172.21.15.185:0/3462447583 learned_addr learned my addr 172.21.15.185:0/3462447583 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57022/0 (socket says 172.21.15.185:57022) 2020-11-16T22:22:20.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 -- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 msgr2=0x7f31c00674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 -- 172.21.15.185:0/3462447583 >> v1:172.21.15.96:6789/0 conn(0x7f31c00b2450 legacy=0x7f31c0068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 -- 172.21.15.185:0/3462447583 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c01447a0 con 0x7f31c013f9b0 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bffff700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c49a6700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c013f9b0 0x7f31c013fe00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f31b8009d50 tx=0x7f31b8009290).ready entity=mon.1 client_cookie=195f677b278877e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bf7fe700 1 -- 172.21.15.185:0/3462447583 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b800bbc0 con 0x7f31c013f9b0 2020-11-16T22:22:20.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bf7fe700 1 -- 172.21.15.185:0/3462447583 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b800bd20 con 0x7f31c013f9b0 2020-11-16T22:22:20.796 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31bf7fe700 1 -- 172.21.15.185:0/3462447583 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8011610 con 0x7f31c013f9b0 2020-11-16T22:22:20.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c013f9b0 msgr2=0x7f31c013fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c013f9b0 0x7f31c013fe00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f31b8009d50 tx=0x7f31b8009290).stop 2020-11-16T22:22:20.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- 172.21.15.185:0/3462447583 shutdown_connections 2020-11-16T22:22:20.798 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c013f9b0 0x7f31c013fe00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3462447583 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c00670c0 0x7f31c00674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- 172.21.15.185:0/3462447583 >> 172.21.15.185:0/3462447583 conn(0x7f31c0060130 msgr2=0x7f31c0060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- 172.21.15.185:0/3462447583 shutdown_connections 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.802+0000 7f31c51a7700 1 -- 172.21.15.185:0/3462447583 wait complete. 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 Processor -- start 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- start start 2020-11-16T22:22:20.799 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c00670c0 0x7f31c01428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 0x7f31c00ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c01398b0 con 0x7f31c00af200 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c0139b50 con 0x7f31c00670c0 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c0139df0 con 0x7f31c013f9b0 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57028/0 (socket says 172.21.15.185:57028) 2020-11-16T22:22:20.800 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 -- 172.21.15.185:0/3106214618 learned_addr learned my addr 172.21.15.185:0/3106214618 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c49a6700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 0x7f31c00ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31bffff700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c00670c0 0x7f31c01428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 -- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c00670c0 msgr2=0x7f31c01428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c00670c0 0x7f31c01428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 -- 172.21.15.185:0/3106214618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 msgr2=0x7f31c00ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 0x7f31c00ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:20.801 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 -- 172.21.15.185:0/3106214618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c013a090 con 0x7f31c013f9b0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c49a6700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 0x7f31c00ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31b77fe700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f31a800c710 tx=0x7f31a8007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8007020 con 0x7f31c013f9b0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8007180 con 0x7f31c013f9b0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c0142dd0 con 0x7f31c013f9b0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a800e560 con 0x7f31c013f9b0 2020-11-16T22:22:20.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c00adff0 con 0x7f31c013f9b0 2020-11-16T22:22:20.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.806+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c005d200 con 0x7f31c013f9b0 2020-11-16T22:22:20.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.810+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f31a8010070 con 0x7f31c013f9b0 2020-11-16T22:22:20.803 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.810+0000 7f31bcff9700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 0x7f319c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:20.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.810+0000 7f31bffff700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 0x7f319c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:20.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.810+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f31a806d160 con 0x7f31c013f9b0 2020-11-16T22:22:20.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.810+0000 7f31bffff700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 0x7f319c057520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f31b0006fd0 tx=0x7f31b0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:20.804 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:20.814+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f31a80433c0 con 0x7f31c013f9b0 2020-11-16T22:22:20.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.010+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f31c00ae1e0 con 0x7f31c013f9b0 2020-11-16T22:22:21.299 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:22:21.308+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-16T22:22:21.316 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.326+0000 7f31bcff9700 1 -- 172.21.15.185:0/3106214618 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7f31a80469e0 con 0x7f31c013f9b0 2020-11-16T22:22:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 msgr2=0x7f319c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 0x7f319c057520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f31b0006fd0 tx=0x7f31b0008040).stop 2020-11-16T22:22:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 msgr2=0x7f31c00ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f31a800c710 tx=0x7f31a8007c20).stop 2020-11-16T22:22:21.320 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 shutdown_connections 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f319c055060 0x7f319c057520 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f31c00670c0 0x7f31c01428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31c00af200 0x7f31c00ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 --2- 172.21.15.185:0/3106214618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f31c013f9b0 0x7f31c00ad590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 >> 172.21.15.185:0/3106214618 conn(0x7f31c0060130 msgr2=0x7f31c00b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 shutdown_connections 2020-11-16T22:22:21.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.330+0000 7f31c51a7700 1 -- 172.21.15.185:0/3106214618 wait complete. 2020-11-16T22:22:21.322 INFO:teuthology.orchestra.run.smithi185.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-16T22:22:21.323 INFO:tasks.ceph.mds.b.smithi096.stderr: -1209> 2020-11-16T22:21:51.813+0000 7fe1a1be7700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1) n(v2 rc2020-11-16T22:21:28.420895+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@4},l=7251 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55a326cb4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:21:28.420895+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.323 INFO:tasks.ceph.mds.b.smithi096.stderr: -1103> 2020-11-16T22:21:51.813+0000 7fe1a7bf3700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1) n(v1 rc2020-11-16T22:21:28.420895+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7251=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55a326cb4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.984892+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:21:28.420895+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.323 INFO:tasks.ceph.mds.c.smithi096.stderr: -8087> 2020-11-16T22:21:28.178+0000 7fd96fe0c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:22:21.323 INFO:tasks.ceph.mds.c.smithi096.stderr: -8085> 2020-11-16T22:21:28.178+0000 7fd96fe0c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:22:21.324 INFO:tasks.ceph.mds.a.smithi185.stderr: -1092> 2020-11-16T22:21:51.810+0000 7efdaa137700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1) n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@4},l=7251 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556b546c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.988892+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:21:27.988892+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.324 INFO:tasks.ceph.mds.a.smithi185.stderr: -1078> 2020-11-16T22:21:51.810+0000 7efdaa137700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/AsLsXsFsx@2},l=7251 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556b5489e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.329 INFO:tasks.ceph.mds.c.smithi096.stderr: -2410> 2020-11-16T22:21:51.809+0000 7fd96c605700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v21 ap=1 f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1) n(v4 rc2020-11-16T22:21:28.420895+0000 6=0+6)/n(v3 rc2020-11-16T22:21:28.420895+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={7251=pAsLsXsFsx/pAsLsXsFsx@5},l=7251 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=1 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x563afb0aa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:21:28.420895+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.330 INFO:tasks.ceph.mds.c.smithi096.stderr: -2303> 2020-11-16T22:21:51.813+0000 7fd972611700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1) n(v3 rc2020-11-16T22:21:28.420895+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7251=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x563afb0aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v21>,<0x1/d1 v18>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:21:28.420895+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:21:27.980892+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:21:28.420895+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:22:21.338 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-16T22:22:21.343 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:22:21.343 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:22:21.352 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:22:21.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 Processor -- start 2020-11-16T22:22:21.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 -- start start 2020-11-16T22:22:21.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 0x7f1554148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:21.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155414c660 con 0x7f15540b13d0 2020-11-16T22:22:21.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15540ac120 con 0x7f15540b3080 2020-11-16T22:22:21.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f155a474700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540ac3c0 con 0x7f15540b3920 2020-11-16T22:22:21.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559472700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f15540b3080 0x7f15540ac860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57040/0 (socket says 172.21.15.185:57040) 2020-11-16T22:22:21.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559472700 1 -- 172.21.15.185:0/1736281501 learned_addr learned my addr 172.21.15.185:0/1736281501 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:21.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559c73700 1 --2- 172.21.15.185:0/1736281501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 0x7f1554148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:21.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1547fff700 1 -- 172.21.15.185:0/1736281501 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267576873 0 0) 0x7f15540ac120 con 0x7f15540b3080 2020-11-16T22:22:21.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1547fff700 1 -- 172.21.15.185:0/1736281501 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1538003240 con 0x7f15540b3080 2020-11-16T22:22:21.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559c73700 1 -- 172.21.15.185:0/1736281501 >> v1:172.21.15.185:6789/0 conn(0x7f15540b3080 legacy=0x7f15540ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559c73700 1 -- 172.21.15.185:0/1736281501 >> v1:172.21.15.96:6789/0 conn(0x7f15540b13d0 legacy=0x7f15540b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559c73700 1 -- 172.21.15.185:0/1736281501 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540b3460 con 0x7f15540b3920 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1559c73700 1 --2- 172.21.15.185:0/1736281501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 0x7f1554148750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f155000b9d0 tx=0x7f1550006df0).ready entity=mon.2 client_cookie=c14a99150be955d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1547fff700 1 -- 172.21.15.185:0/1736281501 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500097b0 con 0x7f15540b3920 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1547fff700 1 -- 172.21.15.185:0/1736281501 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550009910 con 0x7f15540b3920 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.470+0000 7f1547fff700 1 -- 172.21.15.185:0/1736281501 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550009c90 con 0x7f15540b3920 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- 172.21.15.185:0/1736281501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 msgr2=0x7f1554148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 --2- 172.21.15.185:0/1736281501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 0x7f1554148750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f155000b9d0 tx=0x7f1550006df0).stop 2020-11-16T22:22:21.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- 172.21.15.185:0/1736281501 shutdown_connections 2020-11-16T22:22:21.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 --2- 172.21.15.185:0/1736281501 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b3920 0x7f1554148750 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- 172.21.15.185:0/1736281501 >> 172.21.15.185:0/1736281501 conn(0x7f1554060130 msgr2=0x7f1554060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:21.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- 172.21.15.185:0/1736281501 shutdown_connections 2020-11-16T22:22:21.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- 172.21.15.185:0/1736281501 wait complete. 2020-11-16T22:22:21.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 Processor -- start 2020-11-16T22:22:21.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- start start 2020-11-16T22:22:21.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b13d0 0x7f15540afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:21.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:21.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 0x7f1554112340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554113650 con 0x7f15540b3920 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554112f30 con 0x7f15540b3080 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f155a474700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15541133b0 con 0x7f15540b13d0 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1559472700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b13d0 0x7f15540afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35378/0 (socket says 172.21.15.185:35378) 2020-11-16T22:22:21.469 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 -- 172.21.15.185:0/1444660273 learned_addr learned my addr 172.21.15.185:0/1444660273 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1559c73700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 0x7f1554112340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 -- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b13d0 msgr2=0x7f15540afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b13d0 0x7f15540afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 -- 172.21.15.185:0/1444660273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 msgr2=0x7f1554112340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 0x7f1554112340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1558c71700 1 -- 172.21.15.185:0/1444660273 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15541581b0 con 0x7f15540b3080 2020-11-16T22:22:21.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.474+0000 7f1559c73700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 0x7f1554112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f1558c71700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f154800c740 tx=0x7f1548007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154800e560 con 0x7f15540b3080 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800e6c0 con 0x7f15540b3080 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554158450 con 0x7f15540b3080 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154800ea60 con 0x7f15540b3080 2020-11-16T22:22:21.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15540af8d0 con 0x7f15540b3080 2020-11-16T22:22:21.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1554158000 con 0x7f15540b3080 2020-11-16T22:22:21.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1548019030 con 0x7f15540b3080 2020-11-16T22:22:21.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 0x7f152c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:21.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f1559472700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 0x7f152c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:21.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f154806d230 con 0x7f15540b3080 2020-11-16T22:22:21.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.478+0000 7f1559472700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 0x7f152c057520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f154c005920 tx=0x7f154c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:21.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.486+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f1548043440 con 0x7f15540b3080 2020-11-16T22:22:21.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:21.686+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1554112180 con 0x7f15540b3080 2020-11-16T22:22:22.321 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.330+0000 7f15457fa700 1 -- 172.21.15.185:0/1444660273 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15480170a0 con 0x7f15540b3080 2020-11-16T22:22:22.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 msgr2=0x7f152c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 0x7f152c057520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f154c005920 tx=0x7f154c009040).stop 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 msgr2=0x7f15540b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f154800c740 tx=0x7f1548007b80).stop 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 shutdown_connections 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f152c055060 0x7f152c057520 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15540b3920 0x7f1554112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f15540b13d0 0x7f15540afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 --2- 172.21.15.185:0/1444660273 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f15540b3080 0x7f15540b0360 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 >> 172.21.15.185:0/1444660273 conn(0x7f1554060130 msgr2=0x7f15540b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:22.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 shutdown_connections 2020-11-16T22:22:22.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.334+0000 7f155a474700 1 -- 172.21.15.185:0/1444660273 wait complete. 2020-11-16T22:22:22.342 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-16T22:22:22.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.534+0000 7ff3d7455700 1 Processor -- start 2020-11-16T22:22:22.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.534+0000 7ff3d7455700 1 -- start start 2020-11-16T22:22:22.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff3d00b7170 0x7ff3d00b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7ff3d00b64b0 0x7ff3d0144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d0144170 con 0x7ff3d00b7170 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d00b6ed0 con 0x7ff3d00b64b0 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00b6c30 con 0x7ff3d0148160 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57060/0 (socket says 172.21.15.185:57060) 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 -- 172.21.15.185:0/3582281954 learned_addr learned my addr 172.21.15.185:0/3582281954 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d6c54700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.185:3300/0 conn(0x7ff3d00b64b0 0x7ff3d0144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d6453700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.96:3300/0 conn(0x7ff3d00b7170 0x7ff3d00b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 -- 172.21.15.185:0/3582281954 >> v2:172.21.15.185:3300/0 conn(0x7ff3d00b64b0 msgr2=0x7ff3d0144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.185:3300/0 conn(0x7ff3d00b64b0 0x7ff3d0144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 -- 172.21.15.185:0/3582281954 >> v2:172.21.15.96:3300/0 conn(0x7ff3d00b7170 msgr2=0x7ff3d00b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.96:3300/0 conn(0x7ff3d00b7170 0x7ff3d00b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 -- 172.21.15.185:0/3582281954 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d01443d0 con 0x7ff3d0148160 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5c52700 1 --2- 172.21.15.185:0/3582281954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7ff3d0060d40 tx=0x7ff3c0006eb0).ready entity=mon.2 client_cookie=fce3564b6ee369dc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:22.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5451700 1 -- 172.21.15.185:0/3582281954 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00098d0 con 0x7ff3d0148160 2020-11-16T22:22:22.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5451700 1 -- 172.21.15.185:0/3582281954 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0009a30 con 0x7ff3d0148160 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d5451700 1 -- 172.21.15.185:0/3582281954 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0009db0 con 0x7ff3d0148160 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- 172.21.15.185:0/3582281954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 msgr2=0x7ff3d00acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- 172.21.15.185:0/3582281954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7ff3d0060d40 tx=0x7ff3c0006eb0).stop 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- 172.21.15.185:0/3582281954 shutdown_connections 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.185:3300/0 conn(0x7ff3d00b64b0 0x7ff3d0144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- 172.21.15.185:0/3582281954 >> v2:172.21.15.96:3300/0 conn(0x7ff3d00b7170 0x7ff3d00b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 --2- 172.21.15.185:0/3582281954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d0148160 0x7ff3d00acee0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- 172.21.15.185:0/3582281954 >> 172.21.15.185:0/3582281954 conn(0x7ff3d0067860 msgr2=0x7ff3d0067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- 172.21.15.185:0/3582281954 shutdown_connections 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.538+0000 7ff3d7455700 1 -- 172.21.15.185:0/3582281954 wait complete. 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 Processor -- start 2020-11-16T22:22:22.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 -- start start 2020-11-16T22:22:22.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 0x7ff3d00b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d00b7170 0x7ff3d0139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 0x7ff3d0139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01417f0 con 0x7ff3d00b7170 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00b9f00 con 0x7ff3d0148160 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d7455700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0139c40 con 0x7ff3d00b64b0 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6453700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 0x7ff3d00b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6453700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 0x7ff3d00b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57066/0 (socket says 172.21.15.185:57066) 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6453700 1 -- 172.21.15.185:0/1528027744 learned_addr learned my addr 172.21.15.185:0/1528027744 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 0x7ff3d0139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d5c52700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d00b7170 0x7ff3d0139260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 -- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 msgr2=0x7ff3d00b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 0x7ff3d00b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 -- 172.21.15.185:0/1528027744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d00b7170 msgr2=0x7ff3d0139260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d00b7170 0x7ff3d0139260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 -- 172.21.15.185:0/1528027744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00b9520 con 0x7ff3d0148160 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3d6c54700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 0x7ff3d0139750 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7ff3c800db60 tx=0x7ff3c8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:22.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8017070 con 0x7ff3d0148160 2020-11-16T22:22:22.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8009730 con 0x7ff3d0148160 2020-11-16T22:22:22.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.542+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8009ab0 con 0x7ff3d0148160 2020-11-16T22:22:22.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d00b97c0 con 0x7ff3d0148160 2020-11-16T22:22:22.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d00b5850 con 0x7ff3d0148160 2020-11-16T22:22:22.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7ff3c8013480 con 0x7ff3d0148160 2020-11-16T22:22:22.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d005d200 con 0x7ff3d0148160 2020-11-16T22:22:22.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3be7fc700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 0x7ff3a8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:22.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6331+0+0 (secure 0 0 0) 0x7ff3c806e0a0 con 0x7ff3d0148160 2020-11-16T22:22:22.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3d6453700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 0x7ff3a8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:22.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.546+0000 7ff3d6453700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 0x7ff3a8057520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ff3cc006fd0 tx=0x7ff3cc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:22.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.550+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7ff3c80441a0 con 0x7ff3d0148160 2020-11-16T22:22:22.694 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:22.702+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3d00603a0 con 0x7ff3d0148160 2020-11-16T22:22:23.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.338+0000 7ff3be7fc700 1 -- 172.21.15.185:0/1528027744 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff3c8006e00 con 0x7ff3d0148160 2020-11-16T22:22:23.334 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 msgr2=0x7ff3a8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 0x7ff3a8057520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ff3cc006fd0 tx=0x7ff3cc008040).stop 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 msgr2=0x7ff3d0139750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 0x7ff3d0139750 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7ff3c800db60 tx=0x7ff3c8008040).stop 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 shutdown_connections 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7ff3a8055060 0x7ff3a8057520 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7ff3d0148160 0x7ff3d0139750 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7ff3d00b64b0 0x7ff3d00b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 --2- 172.21.15.185:0/1528027744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d00b7170 0x7ff3d0139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.342+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 >> 172.21.15.185:0/1528027744 conn(0x7ff3d0067860 msgr2=0x7ff3d0143820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:23.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.346+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 shutdown_connections 2020-11-16T22:22:23.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.346+0000 7ff3d7455700 1 -- 172.21.15.185:0/1528027744 wait complete. 2020-11-16T22:22:23.337 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' removed 2020-11-16T22:22:23.349 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-16T22:22:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.486+0000 7f3b89836700 1 Processor -- start 2020-11-16T22:22:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.486+0000 7f3b89836700 1 -- start start 2020-11-16T22:22:23.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b840b4d40 0x7f3b84068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f3b841418d0 0x7f3b8413de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b840b7e10 con 0x7f3b840675b0 2020-11-16T22:22:23.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840b7670 con 0x7f3b841418d0 2020-11-16T22:22:23.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b78d0 con 0x7f3b840b4d40 2020-11-16T22:22:23.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b88834700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f3b840675b0 0x7f3b840b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57068/0 (socket says 172.21.15.185:57068) 2020-11-16T22:22:23.481 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b88834700 1 -- 172.21.15.185:0/1514592155 learned_addr learned my addr 172.21.15.185:0/1514592155 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 --2- 172.21.15.185:0/1514592155 >> v2:172.21.15.185:3300/0 conn(0x7f3b841418d0 0x7f3b8413de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b83fff700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b840b4d40 0x7f3b84068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 -- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b840b4d40 msgr2=0x7f3b84068460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b840b4d40 0x7f3b84068460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 -- 172.21.15.185:0/1514592155 >> v1:172.21.15.185:6789/0 conn(0x7f3b840675b0 legacy=0x7f3b840b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 -- 172.21.15.185:0/1514592155 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840b7b70 con 0x7f3b841418d0 2020-11-16T22:22:23.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89035700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b841418d0 0x7f3b8413de00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f3b7c009c90 tx=0x7f3b7c0091d0).ready entity=mon.1 client_cookie=97ec6ade65092732 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:23.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b837fe700 1 -- 172.21.15.185:0/1514592155 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00ba70 con 0x7f3b841418d0 2020-11-16T22:22:23.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b837fe700 1 -- 172.21.15.185:0/1514592155 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00bbd0 con 0x7f3b841418d0 2020-11-16T22:22:23.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b837fe700 1 -- 172.21.15.185:0/1514592155 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c011400 con 0x7f3b841418d0 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b841418d0 msgr2=0x7f3b8413de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b841418d0 0x7f3b8413de00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f3b7c009c90 tx=0x7f3b7c0091d0).stop 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- 172.21.15.185:0/1514592155 shutdown_connections 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b841418d0 0x7f3b8413de00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 --2- 172.21.15.185:0/1514592155 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b840b4d40 0x7f3b84068460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- 172.21.15.185:0/1514592155 >> 172.21.15.185:0/1514592155 conn(0x7f3b8405ba80 msgr2=0x7f3b8405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:23.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- 172.21.15.185:0/1514592155 shutdown_connections 2020-11-16T22:22:23.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.490+0000 7f3b89836700 1 -- 172.21.15.185:0/1514592155 wait complete. 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 Processor -- start 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 -- start start 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b840675b0 0x7f3b840ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b841418d0 0x7f3b841380d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b84143d00 con 0x7f3b840675b0 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b1130 con 0x7f3b840b4d40 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89836700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b84143f60 con 0x7f3b841418d0 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b89035700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b841418d0 0x7f3b841380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35406/0 (socket says 172.21.15.185:35406) 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 -- 172.21.15.185:0/1065416767 learned_addr learned my addr 172.21.15.185:0/1065416767 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:23.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b88834700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b840675b0 0x7f3b840ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 -- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b841418d0 msgr2=0x7f3b841380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b841418d0 0x7f3b841380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 -- 172.21.15.185:0/1065416767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b840675b0 msgr2=0x7f3b840ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b840675b0 0x7f3b840ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 -- 172.21.15.185:0/1065416767 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840adaf0 con 0x7f3b840b4d40 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.494+0000 7f3b83fff700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f3b7800eba0 tx=0x7f3b78006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800c8e0 con 0x7f3b840b4d40 2020-11-16T22:22:23.489 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7800ca40 con 0x7f3b840b4d40 2020-11-16T22:22:23.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800cdc0 con 0x7f3b840b4d40 2020-11-16T22:22:23.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840b13d0 con 0x7f3b840b4d40 2020-11-16T22:22:23.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b840af510 con 0x7f3b840b4d40 2020-11-16T22:22:23.490 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b840af6d0 con 0x7f3b840b4d40 2020-11-16T22:22:23.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f3b78012600 con 0x7f3b840b4d40 2020-11-16T22:22:23.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 0x7f3b5c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:23.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f3b7806e0a0 con 0x7f3b840b4d40 2020-11-16T22:22:23.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b88834700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 0x7f3b5c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:23.491 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.498+0000 7f3b88834700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 0x7f3b5c057520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3b7400bfd0 tx=0x7f3b7400d040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:23.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.506+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f3b780443b0 con 0x7f3b840b4d40 2020-11-16T22:22:23.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:23.658+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b840b40d0 con 0x7f3b840b4d40 2020-11-16T22:22:24.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.342+0000 7f3b80ff9700 1 -- 172.21.15.185:0/1065416767 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3b780479d0 con 0x7f3b840b4d40 2020-11-16T22:22:24.335 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 msgr2=0x7f3b5c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 0x7f3b5c057520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3b7400bfd0 tx=0x7f3b7400d040).stop 2020-11-16T22:22:24.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 msgr2=0x7f3b840af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.336 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f3b7800eba0 tx=0x7f3b78006df0).stop 2020-11-16T22:22:24.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 shutdown_connections 2020-11-16T22:22:24.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f3b5c055060 0x7f3b5c057520 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.337 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f3b841418d0 0x7f3b841380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b840675b0 0x7f3b840ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 --2- 172.21.15.185:0/1065416767 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f3b840b4d40 0x7f3b840af8f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 >> 172.21.15.185:0/1065416767 conn(0x7f3b8405ba80 msgr2=0x7f3b84140ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:24.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 shutdown_connections 2020-11-16T22:22:24.338 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.346+0000 7f3b89836700 1 -- 172.21.15.185:0/1065416767 wait complete. 2020-11-16T22:22:24.339 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' removed 2020-11-16T22:22:24.352 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-16T22:22:24.353 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-16T22:22:24.353 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-16T22:22:24.353 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:22:24.353 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-16T22:22:24.353 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-16T22:22:24.354 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-16T22:22:24.354 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:22:24.354 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-16T22:22:24.354 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-16T22:22:24.355 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-16T22:22:24.355 INFO:teuthology.orchestra.run:waiting for 300 2020-11-16T22:22:24.363 INFO:tasks.ceph.mds.a:Stopped 2020-11-16T22:22:24.363 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-16T22:22:24.366 INFO:tasks.ceph.mds.b:Stopped 2020-11-16T22:22:24.366 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-16T22:22:24.367 INFO:tasks.ceph.mds.c:Stopped 2020-11-16T22:22:24.368 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-16T22:22:24.408 INFO:tasks.ceph.mds.a:Started 2020-11-16T22:22:24.408 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:24.409 INFO:tasks.ceph.mds.b:Started 2020-11-16T22:22:24.410 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:24.411 INFO:tasks.ceph.mds.c:Started 2020-11-16T22:22:24.411 DEBUG:teuthology.parallel:result is None 2020-11-16T22:22:24.412 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-16T22:22:24.487 INFO:tasks.ceph.mds.a.smithi185.stdout:starting mds.a at 2020-11-16T22:22:24.488 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2020-11-16T22:22:24.491 INFO:tasks.ceph.mds.c.smithi096.stdout:starting mds.c at 2020-11-16T22:22:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 Processor -- start 2020-11-16T22:22:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 -- start start 2020-11-16T22:22:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f77f40b4520 0x7f77f414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f4068060 con 0x7f77f4099d80 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f4067260 con 0x7f77f40b4140 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77fa3e7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f4067ce0 con 0x7f77f40b4520 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f3fff700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f77f40b4140 0x7f77f40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57094/0 (socket says 172.21.15.185:57094) 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f3fff700 1 -- 172.21.15.185:0/299126269 learned_addr learned my addr 172.21.15.185:0/299126269 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f8984700 1 --2- 172.21.15.185:0/299126269 >> v2:172.21.15.185:3300/0 conn(0x7f77f40b4520 0x7f77f414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067235059 0 0) 0x7f77f4067260 con 0x7f77f40b4140 2020-11-16T22:22:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e0003240 con 0x7f77f40b4140 2020-11-16T22:22:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182474328 0 0) 0x7f77f4068060 con 0x7f77f4099d80 2020-11-16T22:22:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f8984700 1 -- 172.21.15.185:0/299126269 >> v1:172.21.15.185:6789/0 conn(0x7f77f40b4140 legacy=0x7f77f40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f8984700 1 -- 172.21.15.185:0/299126269 >> v1:172.21.15.96:6789/0 conn(0x7f77f4099d80 legacy=0x7f77f40b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f8984700 1 -- 172.21.15.185:0/299126269 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4067e20 con 0x7f77f40b4520 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f8984700 1 --2- 172.21.15.185:0/299126269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f40b4520 0x7f77f414ba90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f77ec009c90 tx=0x7f77ec0091d0).ready entity=mon.1 client_cookie=a4ff11d837df66df server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec00ba70 con 0x7f77f40b4520 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec00bbd0 con 0x7f77f40b4520 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.550+0000 7f77f2ffd700 1 -- 172.21.15.185:0/299126269 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011400 con 0x7f77f40b4520 2020-11-16T22:22:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- 172.21.15.185:0/299126269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f40b4520 msgr2=0x7f77f414ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/299126269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f40b4520 0x7f77f414ba90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f77ec009c90 tx=0x7f77ec0091d0).stop 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- 172.21.15.185:0/299126269 shutdown_connections 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/299126269 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f40b4520 0x7f77f414ba90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- 172.21.15.185:0/299126269 >> 172.21.15.185:0/299126269 conn(0x7f77f4061240 msgr2=0x7f77f4061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- 172.21.15.185:0/299126269 shutdown_connections 2020-11-16T22:22:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- 172.21.15.185:0/299126269 wait complete. 2020-11-16T22:22:24.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 Processor -- start 2020-11-16T22:22:24.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- start start 2020-11-16T22:22:24.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77f40b4520 0x7f77f4156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4093f90 con 0x7f77f40b4520 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4094230 con 0x7f77f4099d80 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77fa3e7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41567e0 con 0x7f77f40b4140 2020-11-16T22:22:24.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35430/0 (socket says 172.21.15.185:35430) 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57108/0 (socket says 172.21.15.185:57108) 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f3fff700 1 -- 172.21.15.185:0/3813793455 learned_addr learned my addr 172.21.15.185:0/3813793455 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.554+0000 7f77f8984700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77f40b4520 0x7f77f4156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 -- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 msgr2=0x7f77f40602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 -- 172.21.15.185:0/3813793455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77f40b4520 msgr2=0x7f77f4156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77f40b4520 0x7f77f4156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 -- 172.21.15.185:0/3813793455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40607c0 con 0x7f77f4099d80 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f37fe700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f3fff700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f77e80010e0 tx=0x7f77e8007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:24.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e80072e0 con 0x7f77f4099d80 2020-11-16T22:22:24.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8007440 con 0x7f77f4099d80 2020-11-16T22:22:24.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40b1dc0 con 0x7f77f4099d80 2020-11-16T22:22:24.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8005820 con 0x7f77f4099d80 2020-11-16T22:22:24.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4153090 con 0x7f77f4099d80 2020-11-16T22:22:24.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f40943f0 con 0x7f77f4099d80 2020-11-16T22:22:24.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f77e8010070 con 0x7f77f4099d80 2020-11-16T22:22:24.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 0x7f77d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:24.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5511+0+0 (secure 0 0 0) 0x7f77e806d240 con 0x7f77f4099d80 2020-11-16T22:22:24.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.558+0000 7f77f37fe700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 0x7f77d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:24.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.562+0000 7f77f37fe700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 0x7f77d0057520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f77f4099b20 tx=0x7f77e4009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:24.558 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.566+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f77e80435c0 con 0x7f77f4099d80 2020-11-16T22:22:24.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:24.734+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f77f41569a0 con 0x7f77f4099d80 2020-11-16T22:22:25.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.526+0000 7f77dbfff700 1 -- 172.21.15.185:0/3813793455 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f77e8046be0 con 0x7f77f4099d80 2020-11-16T22:22:25.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 msgr2=0x7f77d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 0x7f77d0057520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f77f4099b20 tx=0x7f77e4009040).stop 2020-11-16T22:22:25.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 msgr2=0x7f77f4093930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f77e80010e0 tx=0x7f77e8007a00).stop 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 shutdown_connections 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f77d0055060 0x7f77d0057520 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f77f4099d80 0x7f77f4093930 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77f40b4520 0x7f77f4156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 --2- 172.21.15.185:0/3813793455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f77f40b4140 0x7f77f40602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 >> 172.21.15.185:0/3813793455 conn(0x7f77f4061240 msgr2=0x7f77f409bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:25.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 shutdown_connections 2020-11-16T22:22:25.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.530+0000 7f77fa3e7700 1 -- 172.21.15.185:0/3813793455 wait complete. 2020-11-16T22:22:25.523 INFO:teuthology.orchestra.run.smithi185.stderr: removed all blocklist entries 2020-11-16T22:22:25.539 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-16T22:22:25.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 Processor -- start 2020-11-16T22:22:25.664 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 -- start start 2020-11-16T22:22:25.665 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f72a409b270 0x7f72a409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f72a4144840 0x7f72a40b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a409d410 con 0x7f72a409c900 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a4144cf0 con 0x7f72a409b270 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72abf19700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40b6b20 con 0x7f72a4144840 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f72a409c900 0x7f72a41476e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57108/0 (socket says 172.21.15.185:57108) 2020-11-16T22:22:25.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 -- 172.21.15.185:0/4110990228 learned_addr learned my addr 172.21.15.185:0/4110990228 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72aa4b6700 1 --2- 172.21.15.185:0/4110990228 >> v2:172.21.15.185:3300/0 conn(0x7f72a4144840 0x7f72a40b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566208116 0 0) 0x7f72a409d410 con 0x7f72a409c900 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7290003240 con 0x7f72a409c900 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a94b4700 1 --2- 172.21.15.185:0/4110990228 >> v2:172.21.15.96:3300/0 conn(0x7f72a409b270 0x7f72a409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2970550895 0 0) 0x7f7290003240 con 0x7f72a409c900 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 >> v2:172.21.15.185:3300/0 conn(0x7f72a4144840 msgr2=0x7f72a40b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:25.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 --2- 172.21.15.185:0/4110990228 >> v2:172.21.15.185:3300/0 conn(0x7f72a4144840 0x7f72a40b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 >> v2:172.21.15.96:3300/0 conn(0x7f72a409b270 msgr2=0x7f72a409cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 --2- 172.21.15.185:0/4110990228 >> v2:172.21.15.96:3300/0 conn(0x7f72a409b270 0x7f72a409cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a409b010 con 0x7f72a409c900 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f72940014e0 con 0x7f72a409c900 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 >> v1:172.21.15.185:6789/0 conn(0x7f72a409c900 legacy=0x7f72a41476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 0x7f7290003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72900044e0 0x7f7290004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290004e50 0x7f729000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40b6b20 con 0x7f72900044e0 2020-11-16T22:22:25.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290004290 con 0x7f7290004e50 2020-11-16T22:22:25.669 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729000bc20 con 0x7f7290003760 2020-11-16T22:22:25.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 0x7f7290003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72aa4b6700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290004e50 0x7f729000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a94b4700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72900044e0 0x7f7290004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 -- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290004e50 msgr2=0x7f729000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290004e50 0x7f729000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 -- 172.21.15.185:0/4110990228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72900044e0 msgr2=0x7f7290004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72900044e0 0x7f7290004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72aa4b6700 1 -- 172.21.15.185:0/4110990228 reap_dead start 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 -- 172.21.15.185:0/4110990228 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290003240 con 0x7f7290003760 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a94b4700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72900044e0 0x7f7290004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.674+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 0x7f7290003b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f729400c410 tx=0x7f72940077d0).ready entity=mon.2 client_cookie=e2ae013a90f10510 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400ced0 con 0x7f7290003760 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72940070d0 con 0x7f7290003760 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72a8cb3700 1 -- 172.21.15.185:0/4110990228 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294007490 con 0x7f7290003760 2020-11-16T22:22:25.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 msgr2=0x7f7290003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 0x7f7290003b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f729400c410 tx=0x7f72940077d0).stop 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- 172.21.15.185:0/4110990228 shutdown_connections 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 --2- 172.21.15.185:0/4110990228 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7290003760 0x7f7290003b80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- 172.21.15.185:0/4110990228 >> 172.21.15.185:0/4110990228 conn(0x7f72a4068330 msgr2=0x7f72a40ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- 172.21.15.185:0/4110990228 shutdown_connections 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- 172.21.15.185:0/4110990228 wait complete. 2020-11-16T22:22:25.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 Processor -- start 2020-11-16T22:22:25.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.678+0000 7f72abf19700 1 -- start start 2020-11-16T22:22:25.675 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290003760 0x7f72a40b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72a4156d40 0x7f72a40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a409ded0 con 0x7f72a4156d40 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41541c0 con 0x7f7290003760 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40b1010 con 0x7f72a40b0b90 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.676 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57128/0 (socket says 172.21.15.185:57128) 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290003760 0x7f72a40b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 -- 172.21.15.185:0/3701683135 learned_addr learned my addr 172.21.15.185:0/3701683135 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72aa4b6700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72a4156d40 0x7f72a40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 -- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290003760 msgr2=0x7f72a40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290003760 0x7f72a40b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 -- 172.21.15.185:0/3701683135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72a4156d40 msgr2=0x7f72a40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.677 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72a4156d40 0x7f72a40b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 -- 172.21.15.185:0/3701683135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40a01e0 con 0x7f72a40b0b90 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72a94b4700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f729c00f840 tx=0x7f729c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c011660 con 0x7f72a40b0b90 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0117c0 con 0x7f72a40b0b90 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a409e100 con 0x7f72a40b0b90 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c011b60 con 0x7f72a40b0b90 2020-11-16T22:22:25.678 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41571a0 con 0x7f72a40b0b90 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.682+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f729c013400 con 0x7f72a40b0b90 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.686+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a405d4e0 con 0x7f72a40b0b90 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.686+0000 7f7299ffb700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 0x7f727c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.686+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f729c06dbe0 con 0x7f72a40b0b90 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.686+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 0x7f727c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:25.679 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.686+0000 7f72a9cb5700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 0x7f727c057520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7294000f90 tx=0x7f7294015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:25.681 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.690+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f729c044110 con 0x7f72a40b0b90 2020-11-16T22:22:25.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f72a4060530 con 0x7f72a40b0b90 2020-11-16T22:22:25.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f7299ffb700 1 -- 172.21.15.185:0/3701683135 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4274 (secure 0 0 0) 0x7f729c017070 con 0x7f72a40b0b90 2020-11-16T22:22:25.885 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:25.885 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:22:25.885 INFO:teuthology.orchestra.run.smithi185.stdout: "auth_dump": [ 2020-11-16T22:22:25.885 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.885 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.a", 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf2bHZARAAovXN1mHUfoCwcW/p7hRbyA==", 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.886 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.b", 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf+NGzBxAApbfrfsM/pKVLko1jgKTZYA==", 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.887 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mds.c", 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfL7lcEBAA1KHPsoVi5nZUcKioE/ESew==", 2020-11-16T22:22:25.888 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.889 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.0", 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB9+bJfxjYYCxAAZNN5g0TOhWuF1b8/p5gQIw==", 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.890 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.1", 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCA+bJfAvvMDBAADmnfp+xx2UOVqHm6lqmbEA==", 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.891 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.2", 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCD+bJfqh1VERAAiovsj5ZVkfMpBfRw6izHww==", 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.892 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.3", 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCG+bJf2xZ7FRAA+ItoeyxO2OeMgpmq9I0MwQ==", 2020-11-16T22:22:25.893 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.894 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.4", 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCM+bJfX3+jABAAuT44pYO66cQlxe9N69H7UQ==", 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.895 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.5", 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCP+bJfif88BRAAFvZvjsCu6Gg2QFD6nEy1Ag==", 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.896 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.6", 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCS+bJfMUukCxAAXIyUw/TfWrQMu9CL77aW9g==", 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.897 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "osd.7", 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQCV+bJf/IziDxAAHfcO2yZXTrPpSGFEEn4kMw==", 2020-11-16T22:22:25.898 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.0", 2020-11-16T22:22:25.899 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJfb/LfFxAAp22rfedqE9ywpP3SeQV4tg==", 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow r", 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.900 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.1", 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB6+bJf7XmoHBAA9oP0xLkWxcUbSaMcy4Sz4Q==", 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow", 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow r", 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow rw", 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow rwx" 2020-11-16T22:22:25.901 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "client.admin", 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJfd1Y4HhAAifbypDI2PC+8xU3mdc0jTQ==", 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:22:25.902 INFO:teuthology.orchestra.run.smithi185.stdout: "mgr": "allow *", 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow *", 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.x", 2020-11-16T22:22:25.903 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJf5g9hOhAAY5wZ1diW0yYlELQcBBmYug==", 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:25.904 INFO:teuthology.orchestra.run.smithi185.stdout: "entity": "mgr.y", 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: "key": "AQB5+bJffZUVOBAACAZMcZR2W4jSe8JywZ907A==", 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: "caps": { 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: "mds": "allow *", 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: "mon": "allow profile mgr", 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: "osd": "allow *" 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.905 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:25.906 INFO:teuthology.orchestra.run.smithi185.stdout: ] 2020-11-16T22:22:25.906 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:22:25.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 msgr2=0x7f727c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 0x7f727c057520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f7294000f90 tx=0x7f7294015000).stop 2020-11-16T22:22:25.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 msgr2=0x7f72a4156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:25.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f729c00f840 tx=0x7f729c008040).stop 2020-11-16T22:22:25.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 shutdown_connections 2020-11-16T22:22:25.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f727c055060 0x7f727c057520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7290003760 0x7f72a40b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72a4156d40 0x7f72a40b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:25.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 --2- 172.21.15.185:0/3701683135 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72a40b0b90 0x7f72a4156850 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 >> 172.21.15.185:0/3701683135 conn(0x7f72a4068330 msgr2=0x7f72a4142320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 shutdown_connections 2020-11-16T22:22:26.110 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:25.894+0000 7f72abf19700 1 -- 172.21.15.185:0/3701683135 wait complete. 2020-11-16T22:22:26.111 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:26.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 Processor -- start 2020-11-16T22:22:26.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 -- start start 2020-11-16T22:22:26.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91dc0b3140 0x7f91dc1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91dc144dc0 0x7f91dc0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.245 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc0b8400 con 0x7f91dc065cf0 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc140de0 con 0x7f91dc144dc0 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91db59e700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc0663b0 con 0x7f91dc0b3140 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91dc0b3140 0x7f91dc1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f91dc0b3140 0x7f91dc1419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35458/0 (socket says 172.21.15.185:35458) 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 -- 172.21.15.185:0/3757577798 learned_addr learned my addr 172.21.15.185:0/3757577798 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:26.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91d9d9b700 1 --2- 172.21.15.185:0/3757577798 >> v2:172.21.15.96:3300/0 conn(0x7f91dc144dc0 0x7f91dc0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 -- 172.21.15.185:0/3757577798 >> v2:172.21.15.96:3300/0 conn(0x7f91dc144dc0 msgr2=0x7f91dc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 --2- 172.21.15.185:0/3757577798 >> v2:172.21.15.96:3300/0 conn(0x7f91dc144dc0 0x7f91dc0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 -- 172.21.15.185:0/3757577798 >> v1:172.21.15.185:6789/0 conn(0x7f91dc065cf0 legacy=0x7f91dc0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91d959a700 1 -- 172.21.15.185:0/3757577798 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937726751 0 0) 0x7f91dc0b8400 con 0x7f91dc065cf0 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 -- 172.21.15.185:0/3757577798 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc140fd0 con 0x7f91dc0b3140 2020-11-16T22:22:26.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91da59c700 1 --2- 172.21.15.185:0/3757577798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc0b3140 0x7f91dc1419b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f91dc0b7510 tx=0x7f91cc006eb0).ready entity=mon.1 client_cookie=31246dde58c15386 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91d959a700 1 -- 172.21.15.185:0/3757577798 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0098d0 con 0x7f91dc0b3140 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91d959a700 1 -- 172.21.15.185:0/3757577798 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc009a30 con 0x7f91dc0b3140 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.254+0000 7f91d959a700 1 -- 172.21.15.185:0/3757577798 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009db0 con 0x7f91dc0b3140 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- 172.21.15.185:0/3757577798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc0b3140 msgr2=0x7f91dc1419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- 172.21.15.185:0/3757577798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc0b3140 0x7f91dc1419b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f91dc0b7510 tx=0x7f91cc006eb0).stop 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- 172.21.15.185:0/3757577798 shutdown_connections 2020-11-16T22:22:26.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- 172.21.15.185:0/3757577798 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc0b3140 0x7f91dc1419b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- 172.21.15.185:0/3757577798 >> v2:172.21.15.96:3300/0 conn(0x7f91dc144dc0 0x7f91dc0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- 172.21.15.185:0/3757577798 >> 172.21.15.185:0/3757577798 conn(0x7f91dc0666d0 msgr2=0x7f91dc066ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- 172.21.15.185:0/3757577798 shutdown_connections 2020-11-16T22:22:26.250 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- 172.21.15.185:0/3757577798 wait complete. 2020-11-16T22:22:26.252 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 Processor -- start 2020-11-16T22:22:26.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- start start 2020-11-16T22:22:26.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91dc0b3140 0x7f91dc1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91dc144dc0 0x7f91dc1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0b0d60 con 0x7f91dc144dc0 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc142430 con 0x7f91dc065cf0 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91db59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1426d0 con 0x7f91dc0b3140 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91d9d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91dc0b3140 0x7f91dc1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.455 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35466/0 (socket says 172.21.15.185:35466) 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 -- 172.21.15.185:0/1472853605 learned_addr learned my addr 172.21.15.185:0/1472853605 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91dad9d700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91dc144dc0 0x7f91dc1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 -- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91dc0b3140 msgr2=0x7f91dc1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91dc0b3140 0x7f91dc1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 -- 172.21.15.185:0/1472853605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91dc144dc0 msgr2=0x7f91dc1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91dc144dc0 0x7f91dc1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.258+0000 7f91da59c700 1 -- 172.21.15.185:0/1472853605 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc142970 con 0x7f91dc065cf0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91da59c700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f91cc006ef0 tx=0x7f91cc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0067a0 con 0x7f91dc065cf0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc011d50 con 0x7f91dc065cf0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc140020 con 0x7f91dc065cf0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc01e690 con 0x7f91dc065cf0 2020-11-16T22:22:26.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc0b1700 con 0x7f91dc065cf0 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f91cc025030 con 0x7f91dc065cf0 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc05d200 con 0x7f91dc065cf0 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 0x7f91b0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f91cc06f050 con 0x7f91dc065cf0 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91d9d9b700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 0x7f91b0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.262+0000 7f91d9d9b700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 0x7f91b0057520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f91dc111780 tx=0x7f91d0008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.270+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f91cc045710 con 0x7f91dc065cf0 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.418+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91dc05c720 con 0x7f91dc065cf0 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.418+0000 7f91ca7fc700 1 -- 172.21.15.185:0/1472853605 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13676 (secure 0 0 0) 0x7f91cc030580 con 0x7f91dc065cf0 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 msgr2=0x7f91b0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 0x7f91b0057520 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f91dc111780 tx=0x7f91d0008040).stop 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 msgr2=0x7f91dc13fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.461 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f91cc006ef0 tx=0x7f91cc015000).stop 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 shutdown_connections 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f91b0055060 0x7f91b0057520 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f91dc065cf0 0x7f91dc13fb30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91dc144dc0 0x7f91dc1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 --2- 172.21.15.185:0/1472853605 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f91dc0b3140 0x7f91dc1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 >> 172.21.15.185:0/1472853605 conn(0x7f91dc0666d0 msgr2=0x7f91dc060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 shutdown_connections 2020-11-16T22:22:26.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.422+0000 7f91db59e700 1 -- 172.21.15.185:0/1472853605 wait complete. 2020-11-16T22:22:26.463 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:26.463 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":79,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:22:25.517820+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:26.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T22:22:26.464 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:22:26.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 Processor -- start 2020-11-16T22:22:26.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 -- start start 2020-11-16T22:22:26.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 0x7fecb4067420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fecb4141690 0x7fecb40b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb40b3f50 con 0x7fecb40b4930 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb40b41f0 con 0x7fecb4141690 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecbaee8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40b4490 con 0x7fecb40b7690 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecba6e7700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fecb40b4930 0x7fecb4144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57144/0 (socket says 172.21.15.185:57144) 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 0x7fecb4067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecba6e7700 1 -- 172.21.15.185:0/800034057 learned_addr learned my addr 172.21.15.185:0/800034057 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:26.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb96e5700 1 --2- 172.21.15.185:0/800034057 >> v2:172.21.15.96:3300/0 conn(0x7fecb4141690 0x7fecb40b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb8ee4700 1 -- 172.21.15.185:0/800034057 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384047364 0 0) 0x7fecb40b3f50 con 0x7fecb40b4930 2020-11-16T22:22:26.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb8ee4700 1 -- 172.21.15.185:0/800034057 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca0003240 con 0x7fecb40b4930 2020-11-16T22:22:26.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 -- 172.21.15.185:0/800034057 >> v2:172.21.15.96:3300/0 conn(0x7fecb4141690 msgr2=0x7fecb40b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:26.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 --2- 172.21.15.185:0/800034057 >> v2:172.21.15.96:3300/0 conn(0x7fecb4141690 0x7fecb40b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 -- 172.21.15.185:0/800034057 >> v1:172.21.15.185:6789/0 conn(0x7fecb40b4930 legacy=0x7fecb4144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 -- 172.21.15.185:0/800034057 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40b2410 con 0x7fecb40b7690 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb9ee6700 1 --2- 172.21.15.185:0/800034057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 0x7fecb4067420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fecac009c90 tx=0x7fecac0091d0).ready entity=mon.2 client_cookie=98ed19390873dde2 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb8ee4700 1 -- 172.21.15.185:0/800034057 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac00b9c0 con 0x7fecb40b7690 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb8ee4700 1 -- 172.21.15.185:0/800034057 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac00bb20 con 0x7fecb40b7690 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.606+0000 7fecb8ee4700 1 -- 172.21.15.185:0/800034057 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac00bea0 con 0x7fecb40b7690 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- 172.21.15.185:0/800034057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 msgr2=0x7fecb4067420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 --2- 172.21.15.185:0/800034057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 0x7fecb4067420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fecac009c90 tx=0x7fecac0091d0).stop 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- 172.21.15.185:0/800034057 shutdown_connections 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 --2- 172.21.15.185:0/800034057 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b7690 0x7fecb4067420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 --2- 172.21.15.185:0/800034057 >> v2:172.21.15.96:3300/0 conn(0x7fecb4141690 0x7fecb40b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- 172.21.15.185:0/800034057 >> 172.21.15.185:0/800034057 conn(0x7fecb4064b90 msgr2=0x7fecb4063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- 172.21.15.185:0/800034057 shutdown_connections 2020-11-16T22:22:26.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- 172.21.15.185:0/800034057 wait complete. 2020-11-16T22:22:26.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 Processor -- start 2020-11-16T22:22:26.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.610+0000 7fecbaee8700 1 -- start start 2020-11-16T22:22:26.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 0x7fecb40adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecb40b7690 0x7fecb4146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4146c90 con 0x7fecb40b7690 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4146ec0 con 0x7fecb4141690 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4142840 con 0x7fecb40b4930 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb9ee6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 0x7fecb40adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35482/0 (socket says 172.21.15.185:35482) 2020-11-16T22:22:26.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 -- 172.21.15.185:0/251786718 learned_addr learned my addr 172.21.15.185:0/251786718 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb96e5700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecb40b7690 0x7fecb4146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 -- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 msgr2=0x7fecb40adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 0x7fecb40adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 -- 172.21.15.185:0/251786718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecb40b7690 msgr2=0x7fecb4146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecb40b7690 0x7fecb4146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 -- 172.21.15.185:0/251786718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4142ae0 con 0x7fecb4141690 2020-11-16T22:22:26.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb9ee6700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 0x7fecb40adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:26.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecba6e7700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7feca400c710 tx=0x7feca40079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40072e0 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4007440 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb4142d80 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4005800 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb40ae290 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.614+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb4142f60 con 0x7fecb4141690 2020-11-16T22:22:26.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.618+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7feca401a060 con 0x7fecb4141690 2020-11-16T22:22:26.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.618+0000 7fecb1ffb700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 0x7fec8c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.618+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7feca406d150 con 0x7fecb4141690 2020-11-16T22:22:26.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.618+0000 7fecb9ee6700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 0x7fec8c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.618+0000 7fecb9ee6700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 0x7fec8c057520 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fecac006ee0 tx=0x7fecac016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.622+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fecb4142f60 con 0x7fecb4141690 2020-11-16T22:22:26.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.774+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fecb4142f60 con 0x7fecb4141690 2020-11-16T22:22:26.767 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecb1ffb700 1 -- 172.21.15.185:0/251786718 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7fecb4142f60 con 0x7fecb4141690 2020-11-16T22:22:26.768 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:22:26.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 msgr2=0x7fec8c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 0x7fec8c057520 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fecac006ee0 tx=0x7fecac016000).stop 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 msgr2=0x7fecb4146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7feca400c710 tx=0x7feca40079e0).stop 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 shutdown_connections 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fec8c055060 0x7fec8c057520 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fecb4141690 0x7fecb4146520 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fecb40b4930 0x7fecb40adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 --2- 172.21.15.185:0/251786718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecb40b7690 0x7fecb4146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 >> 172.21.15.185:0/251786718 conn(0x7fecb4064b90 msgr2=0x7fecb4140c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 shutdown_connections 2020-11-16T22:22:26.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.778+0000 7fecbaee8700 1 -- 172.21.15.185:0/251786718 wait complete. 2020-11-16T22:22:26.782 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:22:26.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 Processor -- start 2020-11-16T22:22:26.912 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- start start 2020-11-16T22:22:26.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7ae80b3e10 0x7f7ae80bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80b3760 con 0x7f7ae80b3130 2020-11-16T22:22:26.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80b3a00 con 0x7f7ae80baf40 2020-11-16T22:22:26.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae8144190 con 0x7f7ae80b3e10 2020-11-16T22:22:26.913 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7ae80b3e10 0x7f7ae80bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f7ae80b3e10 0x7f7ae80bb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35488/0 (socket says 172.21.15.185:35488) 2020-11-16T22:22:26.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 -- 172.21.15.185:0/23603258 learned_addr learned my addr 172.21.15.185:0/23603258 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:26.914 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7ae659c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f7ae80baf40 0x7f7ae8144820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57156/0 (socket says 172.21.15.185:57156) 2020-11-16T22:22:26.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 -- 172.21.15.185:0/23603258 >> v1:172.21.15.185:6789/0 conn(0x7f7ae80baf40 legacy=0x7f7ae8144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 -- 172.21.15.185:0/23603258 >> v1:172.21.15.96:6789/0 conn(0x7f7ae80b3130 legacy=0x7f7ae80b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 -- 172.21.15.185:0/23603258 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8144380 con 0x7f7ae80b3e10 2020-11-16T22:22:26.915 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7adffff700 1 --2- 172.21.15.185:0/23603258 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae80bb320 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f7ad0006fd0 tx=0x7f7ad0006eb0).ready entity=mon.1 client_cookie=6f3b230b6a08be2a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7ae5d9b700 1 -- 172.21.15.185:0/23603258 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00098d0 con 0x7f7ae80b3e10 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7ae5d9b700 1 -- 172.21.15.185:0/23603258 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0009a30 con 0x7f7ae80b3e10 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7ae5d9b700 1 -- 172.21.15.185:0/23603258 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0009db0 con 0x7f7ae80b3e10 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- 172.21.15.185:0/23603258 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 msgr2=0x7f7ae80bb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/23603258 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae80bb320 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f7ad0006fd0 tx=0x7f7ad0006eb0).stop 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- 172.21.15.185:0/23603258 shutdown_connections 2020-11-16T22:22:26.916 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/23603258 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae80bb320 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- 172.21.15.185:0/23603258 >> 172.21.15.185:0/23603258 conn(0x7f7ae805ba80 msgr2=0x7f7ae805be90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:26.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- 172.21.15.185:0/23603258 shutdown_connections 2020-11-16T22:22:26.917 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.922+0000 7f7aecc6a700 1 -- 172.21.15.185:0/23603258 wait complete. 2020-11-16T22:22:26.918 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 Processor -- start 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- start start 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ae80b3130 0x7f7ae812c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 0x7f7ae812eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80ba8d0 con 0x7f7ae80baf40 2020-11-16T22:22:26.919 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81434e0 con 0x7f7ae80b3e10 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8143780 con 0x7f7ae80b3130 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ae659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ae80b3130 0x7f7ae812c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35492/0 (socket says 172.21.15.185:35492) 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 -- 172.21.15.185:0/3934856975 learned_addr learned my addr 172.21.15.185:0/3934856975 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ae6d9d700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 0x7f7ae812eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 -- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ae80b3130 msgr2=0x7f7ae812c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ae80b3130 0x7f7ae812c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 -- 172.21.15.185:0/3934856975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 msgr2=0x7f7ae812eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 0x7f7ae812eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 -- 172.21.15.185:0/3934856975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8143a20 con 0x7f7ae80b3e10 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ae6d9d700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 0x7f7ae812eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7adffff700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f7ad000b1b0 tx=0x7f7ad0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001e460 con 0x7f7ae80b3e10 2020-11-16T22:22:26.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad001e5c0 con 0x7f7ae80b3e10 2020-11-16T22:22:26.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001e960 con 0x7f7ae80b3e10 2020-11-16T22:22:26.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8143cc0 con 0x7f7ae80b3e10 2020-11-16T22:22:26.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.926+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae80b9440 con 0x7f7ae80b3e10 2020-11-16T22:22:26.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8143e50 con 0x7f7ae80b3e10 2020-11-16T22:22:26.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f7ad0025030 con 0x7f7ae80b3e10 2020-11-16T22:22:26.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7ade7fc700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 0x7f7ab8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:26.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7ae659c700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 0x7f7ab8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:26.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7ad006f280 con 0x7f7ae80b3e10 2020-11-16T22:22:26.924 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.930+0000 7f7ae659c700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 0x7f7ab8057520 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f7ad8000bc0 tx=0x7f7ad8007b40).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:26.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:26.938+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f7ad00458c0 con 0x7f7ae80b3e10 2020-11-16T22:22:27.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.086+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ae805d200 con 0x7f7ae80b3e10 2020-11-16T22:22:27.084 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7ade7fc700 1 -- 172.21.15.185:0/3934856975 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ad0023020 con 0x7f7ae80b3e10 2020-11-16T22:22:27.084 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:27.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 msgr2=0x7f7ab8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.086 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 0x7f7ab8057520 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f7ad8000bc0 tx=0x7f7ad8007b40).stop 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 msgr2=0x7f7ae812e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f7ad000b1b0 tx=0x7f7ad0015000).stop 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 shutdown_connections 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7ab8055060 0x7f7ab8057520 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ae80baf40 0x7f7ae812eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f7ae80b3130 0x7f7ae812c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.087 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 --2- 172.21.15.185:0/3934856975 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f7ae80b3e10 0x7f7ae812e680 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 >> 172.21.15.185:0/3934856975 conn(0x7f7ae805ba80 msgr2=0x7f7ae80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:27.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 shutdown_connections 2020-11-16T22:22:27.088 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.094+0000 7f7aecc6a700 1 -- 172.21.15.185:0/3934856975 wait complete. 2020-11-16T22:22:27.100 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-16T22:22:27.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 Processor -- start 2020-11-16T22:22:27.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 -- start start 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 0x7fbeb0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb0098cc0 con 0x7fbeb0066240 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb00b3950 con 0x7fbeb00b17e0 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb8759700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b3bf0 con 0x7fbeb00b2160 2020-11-16T22:22:27.221 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57180/0 (socket says 172.21.15.185:57180) 2020-11-16T22:22:27.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 -- 172.21.15.185:0/2778972785 learned_addr learned my addr 172.21.15.185:0/2778972785 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:27.222 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb64f5700 1 --2- 172.21.15.185:0/2778972785 >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 0x7fbeb0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 -- 172.21.15.185:0/2778972785 >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 msgr2=0x7fbeb0067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:27.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 --2- 172.21.15.185:0/2778972785 >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 0x7fbeb0067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.223 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 -- 172.21.15.185:0/2778972785 >> v1:172.21.15.96:6789/0 conn(0x7fbeb0066240 legacy=0x7fbeb0065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 -- 172.21.15.185:0/2778972785 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0067b10 con 0x7fbeb00b2160 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb64f5700 1 --2- 172.21.15.185:0/2778972785 >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 0x7fbeb0067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb5cf4700 1 --2- 172.21.15.185:0/2778972785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fbea0006fd0 tx=0x7fbea0006eb0).ready entity=mon.2 client_cookie=f0632f403dde798 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb54f3700 1 -- 172.21.15.185:0/2778972785 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0009810 con 0x7fbeb00b2160 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb54f3700 1 -- 172.21.15.185:0/2778972785 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea0009970 con 0x7fbeb00b2160 2020-11-16T22:22:27.224 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.230+0000 7fbeb54f3700 1 -- 172.21.15.185:0/2778972785 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0009cf0 con 0x7fbeb00b2160 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- 172.21.15.185:0/2778972785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 msgr2=0x7fbeb014bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2778972785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fbea0006fd0 tx=0x7fbea0006eb0).stop 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- 172.21.15.185:0/2778972785 shutdown_connections 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2778972785 >> v2:172.21.15.96:3300/0 conn(0x7fbeb00b17e0 0x7fbeb0067fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2778972785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb014bde0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- 172.21.15.185:0/2778972785 >> 172.21.15.185:0/2778972785 conn(0x7fbeb00604a0 msgr2=0x7fbeb00608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:27.225 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- 172.21.15.185:0/2778972785 shutdown_connections 2020-11-16T22:22:27.226 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- 172.21.15.185:0/2778972785 wait complete. 2020-11-16T22:22:27.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 Processor -- start 2020-11-16T22:22:27.227 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- start start 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0066240 0x7fbeb014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 0x7fbeb015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00aef80 con 0x7fbeb0066240 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b02c0 con 0x7fbeb00b17e0 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb8759700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014e8a0 con 0x7fbeb00b2160 2020-11-16T22:22:27.228 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb5cf4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 0x7fbeb015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb5cf4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 0x7fbeb015b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35508/0 (socket says 172.21.15.185:35508) 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb5cf4700 1 -- 172.21.15.185:0/2678070962 learned_addr learned my addr 172.21.15.185:0/2678070962 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb64f5700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0066240 0x7fbeb014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 -- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 msgr2=0x7fbeb015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 0x7fbeb015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.229 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 -- 172.21.15.185:0/2678070962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0066240 msgr2=0x7fbeb014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0066240 0x7fbeb014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.234+0000 7fbeb6cf6700 1 -- 172.21.15.185:0/2678070962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00ac8f0 con 0x7fbeb00b2160 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb6cf6700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb015bb40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fbea800ca60 tx=0x7fbea8007e20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800ce30 con 0x7fbeb00b2160 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800a730 con 0x7fbeb00b2160 2020-11-16T22:22:27.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800aab0 con 0x7fbeb00b2160 2020-11-16T22:22:27.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb015c030 con 0x7fbeb00b2160 2020-11-16T22:22:27.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb014deb0 con 0x7fbeb00b2160 2020-11-16T22:22:27.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb015b360 con 0x7fbeb00b2160 2020-11-16T22:22:27.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fbea8019030 con 0x7fbeb00b2160 2020-11-16T22:22:27.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 0x7fbe88057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbea806d250 con 0x7fbeb00b2160 2020-11-16T22:22:27.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb64f5700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 0x7fbe88057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.238+0000 7fbeb64f5700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 0x7fbe88057520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fbeb00619f0 tx=0x7fbeac006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.246+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fbea80437e0 con 0x7fbeb00b2160 2020-11-16T22:22:27.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbeb005d4e0 con 0x7fbeb00b2160 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbe9e7fc700 1 -- 172.21.15.185:0/2678070962 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+143 (secure 0 0 0) 0x7fbea8046e00 con 0x7fbeb00b2160 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi185.stdout:[ 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:27.392 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 1, 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "device_health_metrics" 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: }, 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: { 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: "poolnum": 2, 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: "poolname": "rbd" 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:22:27.393 INFO:teuthology.orchestra.run.smithi185.stdout:] 2020-11-16T22:22:27.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 msgr2=0x7fbe88057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.394 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 0x7fbe88057520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fbeb00619f0 tx=0x7fbeac006df0).stop 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 msgr2=0x7fbeb015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb015bb40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fbea800ca60 tx=0x7fbea8007e20).stop 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 shutdown_connections 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fbe88055060 0x7fbe88057520 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fbeb00b2160 0x7fbeb015bb40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb0066240 0x7fbeb014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 --2- 172.21.15.185:0/2678070962 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fbeb00b17e0 0x7fbeb015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 >> 172.21.15.185:0/2678070962 conn(0x7fbeb00604a0 msgr2=0x7fbeb00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:27.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 shutdown_connections 2020-11-16T22:22:27.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.402+0000 7fbeb8759700 1 -- 172.21.15.185:0/2678070962 wait complete. 2020-11-16T22:22:27.406 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-16T22:22:27.407 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-16T22:22:27.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.538+0000 7f498035a700 1 Processor -- start 2020-11-16T22:22:27.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.538+0000 7f498035a700 1 -- start start 2020-11-16T22:22:27.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4978063040 0x7f49780b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49780b3ed0 0x7f4978144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49780b4360 con 0x7f4978144e60 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780b4600 con 0x7f4978063040 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4978145310 con 0x7f49780b3ed0 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49780b3ed0 0x7f4978144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f49780b3ed0 0x7f4978144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35516/0 (socket says 172.21.15.185:35516) 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497e0f6700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4978063040 0x7f49780b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.533 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 -- 172.21.15.185:0/3900790600 learned_addr learned my addr 172.21.15.185:0/3900790600 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 -- 172.21.15.185:0/3900790600 >> v2:172.21.15.96:3300/0 conn(0x7f4978063040 msgr2=0x7f49780b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 --2- 172.21.15.185:0/3900790600 >> v2:172.21.15.96:3300/0 conn(0x7f4978063040 0x7f49780b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 -- 172.21.15.185:0/3900790600 >> v1:172.21.15.96:6789/0 conn(0x7f4978144e60 legacy=0x7f497813de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 -- 172.21.15.185:0/3900790600 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780b48a0 con 0x7f49780b3ed0 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d8f5700 1 --2- 172.21.15.185:0/3900790600 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780b3ed0 0x7f4978144970 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f49780b2fc0 tx=0x7f4968006eb0).ready entity=mon.1 client_cookie=459551bf05ffac2f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d0f4700 1 -- 172.21.15.185:0/3900790600 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009810 con 0x7f49780b3ed0 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d0f4700 1 -- 172.21.15.185:0/3900790600 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968009970 con 0x7f49780b3ed0 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f497d0f4700 1 -- 172.21.15.185:0/3900790600 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009cf0 con 0x7f49780b3ed0 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- 172.21.15.185:0/3900790600 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780b3ed0 msgr2=0x7f4978144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 --2- 172.21.15.185:0/3900790600 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780b3ed0 0x7f4978144970 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f49780b2fc0 tx=0x7f4968006eb0).stop 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- 172.21.15.185:0/3900790600 shutdown_connections 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 --2- 172.21.15.185:0/3900790600 >> v2:172.21.15.96:3300/0 conn(0x7f4978063040 0x7f49780b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 --2- 172.21.15.185:0/3900790600 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780b3ed0 0x7f4978144970 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- 172.21.15.185:0/3900790600 >> 172.21.15.185:0/3900790600 conn(0x7f4978061240 msgr2=0x7f4978061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:27.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- 172.21.15.185:0/3900790600 shutdown_connections 2020-11-16T22:22:27.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.542+0000 7f498035a700 1 -- 172.21.15.185:0/3900790600 wait complete. 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 Processor -- start 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- start start 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 0x7f4978153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4978144e60 0x7f49780b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978146ae0 con 0x7f4978144e60 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978146d10 con 0x7f49780ae860 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49781578d0 con 0x7f4978063040 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e0f6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 0x7f4978153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35524/0 (socket says 172.21.15.185:35524) 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 -- 172.21.15.185:0/817681785 learned_addr learned my addr 172.21.15.185:0/817681785 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:27.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497d8f5700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4978144e60 0x7f49780b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 -- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 msgr2=0x7f4978153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 0x7f4978153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 -- 172.21.15.185:0/817681785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4978144e60 msgr2=0x7f49780b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4978144e60 0x7f49780b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 -- 172.21.15.185:0/817681785 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978153b30 con 0x7f49780ae860 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e0f6700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 0x7f4978153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:27.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f497e8f7700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f497000c740 tx=0x7f4970007b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497000e490 con 0x7f49780ae860 2020-11-16T22:22:27.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497000e5f0 con 0x7f49780ae860 2020-11-16T22:22:27.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49780b1ae0 con 0x7f49780ae860 2020-11-16T22:22:27.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497000e970 con 0x7f49780ae860 2020-11-16T22:22:27.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.546+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4978114d20 con 0x7f49780ae860 2020-11-16T22:22:27.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4970019030 con 0x7f49780ae860 2020-11-16T22:22:27.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4978114f70 con 0x7f49780ae860 2020-11-16T22:22:27.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f49667fc700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 0x7f4950057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:27.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f497e0f6700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 0x7f4950057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:27.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f497006cd10 con 0x7f49780ae860 2020-11-16T22:22:27.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.550+0000 7f497e0f6700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 0x7f4950057520 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4974009e90 tx=0x7f49740091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:27.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.558+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4970043350 con 0x7f49780ae860 2020-11-16T22:22:27.695 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:27.706+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4978146e50 con 0x7f49780ae860 2020-11-16T22:22:28.364 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.374+0000 7f49667fc700 1 -- 172.21.15.185:0/817681785 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v80) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4978146e50 con 0x7f49780ae860 2020-11-16T22:22:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 msgr2=0x7f4950057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.367 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 0x7f4950057520 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4974009e90 tx=0x7f49740091d0).stop 2020-11-16T22:22:28.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 msgr2=0x7f4978143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f497000c740 tx=0x7f4970007b30).stop 2020-11-16T22:22:28.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 shutdown_connections 2020-11-16T22:22:28.368 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4950055060 0x7f4950057520 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f49780ae860 0x7f4978143840 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4978063040 0x7f4978153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 --2- 172.21.15.185:0/817681785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4978144e60 0x7f49780b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 >> 172.21.15.185:0/817681785 conn(0x7f4978061240 msgr2=0x7f497813f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 shutdown_connections 2020-11-16T22:22:28.369 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.378+0000 7f498035a700 1 -- 172.21.15.185:0/817681785 wait complete. 2020-11-16T22:22:28.370 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_metadata' created 2020-11-16T22:22:28.383 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-16T22:22:28.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 Processor -- start 2020-11-16T22:22:28.509 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 -- start start 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 0x7f6ec4142950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec4060e40 con 0x7f6ec4067030 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec4141f70 con 0x7f6ec40b76b0 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca953700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4142210 con 0x7f6ec4067490 2020-11-16T22:22:28.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57208/0 (socket says 172.21.15.185:57208) 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 -- 172.21.15.185:0/2918195987 learned_addr learned my addr 172.21.15.185:0/2918195987 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6ec9150700 1 --1- 172.21.15.185:0/2918195987 >> v1:172.21.15.185:6789/0 conn(0x7f6ec4067030 0x7f6ec4060450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57200/0 (socket says 172.21.15.185:57200) 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6ec9951700 1 --2- 172.21.15.185:0/2918195987 >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 0x7f6ec4142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 -- 172.21.15.185:0/2918195987 >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 msgr2=0x7f6ec4142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 --2- 172.21.15.185:0/2918195987 >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 0x7f6ec4142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 -- 172.21.15.185:0/2918195987 >> v1:172.21.15.185:6789/0 conn(0x7f6ec4067030 legacy=0x7f6ec4060450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6eca152700 1 -- 172.21.15.185:0/2918195987 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec41424b0 con 0x7f6ec4067490 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.518+0000 7f6ec9951700 1 --2- 172.21.15.185:0/2918195987 >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 0x7f6ec4142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca152700 1 --2- 172.21.15.185:0/2918195987 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f6ebc009d50 tx=0x7f6ebc009290).ready entity=mon.2 client_cookie=b0322bfd1b05f00e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6ec894f700 1 -- 172.21.15.185:0/2918195987 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00bc80 con 0x7f6ec4067490 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6ec894f700 1 -- 172.21.15.185:0/2918195987 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc00bde0 con 0x7f6ec4067490 2020-11-16T22:22:28.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6ec894f700 1 -- 172.21.15.185:0/2918195987 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011610 con 0x7f6ec4067490 2020-11-16T22:22:28.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- 172.21.15.185:0/2918195987 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 msgr2=0x7f6ec413ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 --2- 172.21.15.185:0/2918195987 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f6ebc009d50 tx=0x7f6ebc009290).stop 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- 172.21.15.185:0/2918195987 shutdown_connections 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 --2- 172.21.15.185:0/2918195987 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec413ba20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 --2- 172.21.15.185:0/2918195987 >> v2:172.21.15.96:3300/0 conn(0x7f6ec40b76b0 0x7f6ec4142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- 172.21.15.185:0/2918195987 >> 172.21.15.185:0/2918195987 conn(0x7f6ec4064b90 msgr2=0x7f6ec4063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- 172.21.15.185:0/2918195987 shutdown_connections 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- 172.21.15.185:0/2918195987 wait complete. 2020-11-16T22:22:28.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 Processor -- start 2020-11-16T22:22:28.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.522+0000 7f6eca953700 1 -- start start 2020-11-16T22:22:28.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec4067030 0x7f6ec40b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ec40b76b0 0x7f6ec40ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40b1700 con 0x7f6ec4067030 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41418b0 con 0x7f6ec40b76b0 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40b08a0 con 0x7f6ec4067490 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca152700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ec40b76b0 0x7f6ec40ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57212/0 (socket says 172.21.15.185:57212) 2020-11-16T22:22:28.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 -- 172.21.15.185:0/3546157290 learned_addr learned my addr 172.21.15.185:0/3546157290 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9951700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec4067030 0x7f6ec40b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 -- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ec40b76b0 msgr2=0x7f6ec40ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ec40b76b0 0x7f6ec40ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 -- 172.21.15.185:0/3546157290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec4067030 msgr2=0x7f6ec40b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec4067030 0x7f6ec40b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 -- 172.21.15.185:0/3546157290 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40b0b40 con 0x7f6ec4067490 2020-11-16T22:22:28.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9150700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f6ec000c710 tx=0x7f6ec0007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00072e0 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0005530 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00058b0 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec4141b10 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec4144ff0 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6ec0010070 con 0x7f6ec4067490 2020-11-16T22:22:28.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec405d200 con 0x7f6ec4067490 2020-11-16T22:22:28.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 0x7f6e9c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:28.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9951700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 0x7f6e9c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:28.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f6ec006d170 con 0x7f6ec4067490 2020-11-16T22:22:28.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.526+0000 7f6ec9951700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 0x7f6e9c057520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6ec40685a0 tx=0x7f6eb8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:28.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.534+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6ec0043650 con 0x7f6ec4067490 2020-11-16T22:22:28.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:28.682+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f6ec4061020 con 0x7f6ec4067490 2020-11-16T22:22:29.366 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.374+0000 7f6eb5ffb700 1 -- 172.21.15.185:0/3546157290 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v81) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6ec0046c70 con 0x7f6ec4067490 2020-11-16T22:22:29.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 msgr2=0x7f6e9c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 0x7f6e9c057520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f6ec40685a0 tx=0x7f6eb8008040).stop 2020-11-16T22:22:29.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 msgr2=0x7f6ec4139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f6ec000c710 tx=0x7f6ec0007bb0).stop 2020-11-16T22:22:29.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 shutdown_connections 2020-11-16T22:22:29.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6e9c055060 0x7f6e9c057520 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f6ec40b76b0 0x7f6ec40ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec4067030 0x7f6ec40b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 --2- 172.21.15.185:0/3546157290 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f6ec4067490 0x7f6ec4139e30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.378+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 >> 172.21.15.185:0/3546157290 conn(0x7f6ec4064b90 msgr2=0x7f6ec41392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:29.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.382+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 shutdown_connections 2020-11-16T22:22:29.372 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.382+0000 7f6eca953700 1 -- 172.21.15.185:0/3546157290 wait complete. 2020-11-16T22:22:29.372 INFO:teuthology.orchestra.run.smithi185.stderr:pool 'cephfs_data' created 2020-11-16T22:22:29.387 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-16T22:22:29.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.522+0000 7fb21c936700 1 Processor -- start 2020-11-16T22:22:29.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.522+0000 7fb21c936700 1 -- start start 2020-11-16T22:22:29.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb218068220 0x7fb2180ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb218143a20 0x7fb21813d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2180ac830 con 0x7fb2180b7610 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2180ac630 con 0x7fb218143a20 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b1810 con 0x7fb218068220 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb2177fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb2180b7610 0x7fb2180b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57214/0 (socket says 172.21.15.185:57214) 2020-11-16T22:22:29.516 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb2177fe700 1 -- 172.21.15.185:0/421351158 learned_addr learned my addr 172.21.15.185:0/421351158 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb20effd700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb218068220 0x7fb2180ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 --2- 172.21.15.185:0/421351158 >> v2:172.21.15.185:3300/0 conn(0x7fb218143a20 0x7fb21813d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb216ffd700 1 -- 172.21.15.185:0/421351158 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467522256 0 0) 0x7fb2180ac830 con 0x7fb2180b7610 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb216ffd700 1 -- 172.21.15.185:0/421351158 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f8003240 con 0x7fb2180b7610 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 -- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb218068220 msgr2=0x7fb2180ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb218068220 0x7fb2180ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 -- 172.21.15.185:0/421351158 >> v1:172.21.15.185:6789/0 conn(0x7fb2180b7610 legacy=0x7fb2180b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 -- 172.21.15.185:0/421351158 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180b1ab0 con 0x7fb218143a20 2020-11-16T22:22:29.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb217fff700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218143a20 0x7fb21813d220 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb21000b9a0 tx=0x7fb210006df0).ready entity=mon.1 client_cookie=28e579303f59451f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb216ffd700 1 -- 172.21.15.185:0/421351158 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210009680 con 0x7fb218143a20 2020-11-16T22:22:29.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb216ffd700 1 -- 172.21.15.185:0/421351158 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2100097e0 con 0x7fb218143a20 2020-11-16T22:22:29.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb216ffd700 1 -- 172.21.15.185:0/421351158 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210009b60 con 0x7fb218143a20 2020-11-16T22:22:29.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218143a20 msgr2=0x7fb21813d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218143a20 0x7fb21813d220 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb21000b9a0 tx=0x7fb210006df0).stop 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- 172.21.15.185:0/421351158 shutdown_connections 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218143a20 0x7fb21813d220 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- 172.21.15.185:0/421351158 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb218068220 0x7fb2180ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- 172.21.15.185:0/421351158 >> 172.21.15.185:0/421351158 conn(0x7fb218063780 msgr2=0x7fb2180b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- 172.21.15.185:0/421351158 shutdown_connections 2020-11-16T22:22:29.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- 172.21.15.185:0/421351158 wait complete. 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 Processor -- start 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 -- start start 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.526+0000 7fb21c936700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 0x7fb218142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218143a20 0x7fb21813c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218144280 con 0x7fb218143a20 2020-11-16T22:22:29.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21813df30 con 0x7fb218068220 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181446b0 con 0x7fb2180b7610 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 0x7fb218142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35550/0 (socket says 172.21.15.185:35550) 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 -- 172.21.15.185:0/4169956119 learned_addr learned my addr 172.21.15.185:0/4169956119 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb217fff700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218143a20 0x7fb21813c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 -- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 msgr2=0x7fb218142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 0x7fb218142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 -- 172.21.15.185:0/4169956119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218143a20 msgr2=0x7fb21813c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218143a20 0x7fb21813c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 -- 172.21.15.185:0/4169956119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218143300 con 0x7fb218068220 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20effd700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 0x7fb218142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb2177fe700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fb20800ca90 tx=0x7fb208007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208005b80 con 0x7fb218068220 2020-11-16T22:22:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208005ce0 con 0x7fb218068220 2020-11-16T22:22:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21813b9b0 con 0x7fb218068220 2020-11-16T22:22:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208007340 con 0x7fb218068220 2020-11-16T22:22:29.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218154580 con 0x7fb218068220 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21805d200 con 0x7fb218068220 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb208010030 con 0x7fb218068220 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 0x7fb1f4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fb20806dd60 con 0x7fb218068220 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.530+0000 7fb20effd700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 0x7fb1f4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.534+0000 7fb20effd700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 0x7fb1f4057520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fb218140790 tx=0x7fb200006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.538+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb208043fd0 con 0x7fb218068220 2020-11-16T22:22:29.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.738+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fb218143f30 con 0x7fb218068220 2020-11-16T22:22:29.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.750+0000 7fb20ffff700 1 -- 172.21.15.185:0/4169956119 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v98) v1 ==== 189+0+0 (secure 0 0 0) 0x7fb2080475f0 con 0x7fb218068220 2020-11-16T22:22:29.745 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 msgr2=0x7fb1f4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 0x7fb1f4057520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fb218140790 tx=0x7fb200006df0).stop 2020-11-16T22:22:29.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 msgr2=0x7fb21813d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fb20800ca90 tx=0x7fb208007e50).stop 2020-11-16T22:22:29.746 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 shutdown_connections 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1f4055060 0x7fb1f4057520 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb218068220 0x7fb21813d820 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218143a20 0x7fb21813c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 --2- 172.21.15.185:0/4169956119 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2180b7610 0x7fb218142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 >> 172.21.15.185:0/4169956119 conn(0x7fb218063780 msgr2=0x7fb2180b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:29.747 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 shutdown_connections 2020-11-16T22:22:29.748 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.754+0000 7fb21c936700 1 -- 172.21.15.185:0/4169956119 wait complete. 2020-11-16T22:22:29.748 INFO:teuthology.orchestra.run.smithi185.stderr:new fs with metadata pool 15 and data pool 16 2020-11-16T22:22:29.763 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:29.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 Processor -- start 2020-11-16T22:22:29.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 -- start start 2020-11-16T22:22:29.877 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066d80 0x7f1644064a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1644066740 0x7f16441440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16441476b0 con 0x7f16440646a0 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1644147950 con 0x7f1644066740 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f164998b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644147bf0 con 0x7f1644066d80 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1644066740 0x7f16441440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f1644066740 0x7f16441440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35568/0 (socket says 172.21.15.185:35568) 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16427fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066d80 0x7f1644064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 -- 172.21.15.185:0/1091923980 learned_addr learned my addr 172.21.15.185:0/1091923980 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 -- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066d80 msgr2=0x7f1644064a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 --2- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066d80 0x7f1644064a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 -- 172.21.15.185:0/1091923980 >> v1:172.21.15.96:6789/0 conn(0x7f16440646a0 legacy=0x7f16440b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 -- 172.21.15.185:0/1091923980 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644147e90 con 0x7f1644066740 2020-11-16T22:22:29.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f16437fe700 1 --2- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1644066740 0x7f16441440a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f163c009d50 tx=0x7f163c009290).ready entity=mon.1 client_cookie=96164b7f4b1676da server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f1641ffb700 1 -- 172.21.15.185:0/1091923980 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00bbc0 con 0x7f1644066740 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f1641ffb700 1 -- 172.21.15.185:0/1091923980 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00bd20 con 0x7f1644066740 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.886+0000 7f1641ffb700 1 -- 172.21.15.185:0/1091923980 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c011610 con 0x7f1644066740 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1644066740 msgr2=0x7f16441440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1644066740 0x7f16441440a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f163c009d50 tx=0x7f163c009290).stop 2020-11-16T22:22:29.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- 172.21.15.185:0/1091923980 shutdown_connections 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f1644066740 0x7f16441440a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- 172.21.15.185:0/1091923980 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066d80 0x7f1644064a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- 172.21.15.185:0/1091923980 >> 172.21.15.185:0/1091923980 conn(0x7f1644061240 msgr2=0x7f1644061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- 172.21.15.185:0/1091923980 shutdown_connections 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- 172.21.15.185:0/1091923980 wait complete. 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 Processor -- start 2020-11-16T22:22:29.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- start start 2020-11-16T22:22:29.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 0x7f1644142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1644066d80 0x7f164409c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644141e80 con 0x7f1644066d80 2020-11-16T22:22:29.884 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644153260 con 0x7f16440646a0 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f164998b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164408ed30 con 0x7f1644066740 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f16427fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 0x7f1644142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35570/0 (socket says 172.21.15.185:35570) 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f16427fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 0x7f1644142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57248/0 (socket says 172.21.15.185:57248) 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 -- 172.21.15.185:0/1975545633 learned_addr learned my addr 172.21.15.185:0/1975545633 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:29.885 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f16437fe700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1644066d80 0x7f164409c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 -- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 msgr2=0x7f1644142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 0x7f1644142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 -- 172.21.15.185:0/1975545633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1644066d80 msgr2=0x7f164409c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1644066d80 0x7f164409c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.890+0000 7f1642ffd700 1 -- 172.21.15.185:0/1975545633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164408efd0 con 0x7f16440646a0 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f1642ffd700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f163400c6e0 tx=0x7f1634007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.886 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634007020 con 0x7f16440646a0 2020-11-16T22:22:29.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1634007180 con 0x7f16440646a0 2020-11-16T22:22:29.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164408f270 con 0x7f16440646a0 2020-11-16T22:22:29.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340064f0 con 0x7f16440646a0 2020-11-16T22:22:29.887 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164409a030 con 0x7f16440646a0 2020-11-16T22:22:29.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164409a2e0 con 0x7f16440646a0 2020-11-16T22:22:29.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f1634010070 con 0x7f16440646a0 2020-11-16T22:22:29.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 0x7f1620057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:29.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f163406d080 con 0x7f16440646a0 2020-11-16T22:22:29.888 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.894+0000 7f16427fc700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 0x7f1620057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:29.889 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.898+0000 7f16427fc700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 0x7f1620057520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f164413f930 tx=0x7f1638006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:29.893 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:29.902+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f16340432e0 con 0x7f16440646a0 2020-11-16T22:22:30.047 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.058+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f164408f400 con 0x7f16440646a0 2020-11-16T22:22:30.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.058+0000 7f162affd700 1 -- 172.21.15.185:0/1975545633 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16855 (secure 0 0 0) 0x7f16340241f0 con 0x7f16440646a0 2020-11-16T22:22:30.049 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:30.050 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":82,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:22:29.743190+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:22:27.765410+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:22:28.746050+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:30.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 msgr2=0x7f1620057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:30.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 0x7f1620057520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f164413f930 tx=0x7f1638006df0).stop 2020-11-16T22:22:30.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 msgr2=0x7f1644153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:30.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f163400c6e0 tx=0x7f1634007c50).stop 2020-11-16T22:22:30.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 shutdown_connections 2020-11-16T22:22:30.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f1620055060 0x7f1620057520 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f16440646a0 0x7f1644153e20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1644066d80 0x7f164409c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.662 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 --2- 172.21.15.185:0/1975545633 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f1644066740 0x7f1644142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 >> 172.21.15.185:0/1975545633 conn(0x7f1644061240 msgr2=0x7f1644063c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:30.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 shutdown_connections 2020-11-16T22:22:30.663 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.062+0000 7f164998b700 1 -- 172.21.15.185:0/1975545633 wait complete. 2020-11-16T22:22:30.664 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:30.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.798+0000 7f97bce70700 1 Processor -- start 2020-11-16T22:22:30.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.798+0000 7f97bce70700 1 -- start start 2020-11-16T22:22:30.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97b80b4140 0x7f97b80b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:30.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b8068060 con 0x7f97b80b2760 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b8067260 con 0x7f97b80b4140 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8067ce0 con 0x7f97b8099d80 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57256/0 (socket says 172.21.15.185:57256) 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 -- 172.21.15.185:0/4254824445 learned_addr learned my addr 172.21.15.185:0/4254824445 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:30.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b659c700 1 --2- 172.21.15.185:0/4254824445 >> v2:172.21.15.96:3300/0 conn(0x7f97b80b4140 0x7f97b80b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:30.793 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 -- 172.21.15.185:0/4254824445 >> v2:172.21.15.96:3300/0 conn(0x7f97b80b4140 msgr2=0x7f97b80b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 --2- 172.21.15.185:0/4254824445 >> v2:172.21.15.96:3300/0 conn(0x7f97b80b4140 0x7f97b80b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 -- 172.21.15.185:0/4254824445 >> v1:172.21.15.96:6789/0 conn(0x7f97b80b2760 legacy=0x7f97b8063c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 -- 172.21.15.185:0/4254824445 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8066de0 con 0x7f97b8099d80 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b5d9b700 1 --2- 172.21.15.185:0/4254824445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f97a0009d50 tx=0x7f97a0009290).ready entity=mon.2 client_cookie=cf2b10da6c1613a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b559a700 1 -- 172.21.15.185:0/4254824445 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000bbc0 con 0x7f97b8099d80 2020-11-16T22:22:30.794 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b559a700 1 -- 172.21.15.185:0/4254824445 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000bd20 con 0x7f97b8099d80 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97b559a700 1 -- 172.21.15.185:0/4254824445 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0011610 con 0x7f97b8099d80 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- 172.21.15.185:0/4254824445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 msgr2=0x7f97b80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 --2- 172.21.15.185:0/4254824445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f97a0009d50 tx=0x7f97a0009290).stop 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- 172.21.15.185:0/4254824445 shutdown_connections 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 --2- 172.21.15.185:0/4254824445 >> v2:172.21.15.96:3300/0 conn(0x7f97b80b4140 0x7f97b80b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.795 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 --2- 172.21.15.185:0/4254824445 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b8099d80 0x7f97b80b2270 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:30.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- 172.21.15.185:0/4254824445 >> 172.21.15.185:0/4254824445 conn(0x7f97b8061240 msgr2=0x7f97b8061620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:30.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- 172.21.15.185:0/4254824445 shutdown_connections 2020-11-16T22:22:30.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 -- 172.21.15.185:0/4254824445 wait complete. 2020-11-16T22:22:31.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.802+0000 7f97bce70700 1 Processor -- start 2020-11-16T22:22:31.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- start start 2020-11-16T22:22:31.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97b8099d80 0x7f97b8063de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:31.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:31.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b80b4140 0x7f97b80ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80ac7f0 con 0x7f97b8099d80 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b814b290 con 0x7f97b80b2760 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b814b530 con 0x7f97b80b4140 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b6d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b80b4140 0x7f97b80ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35586/0 (socket says 172.21.15.185:35586) 2020-11-16T22:22:31.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 -- 172.21.15.185:0/1342407380 learned_addr learned my addr 172.21.15.185:0/1342407380 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:31.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b659c700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97b8099d80 0x7f97b8063de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 -- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b80b4140 msgr2=0x7f97b80ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b80b4140 0x7f97b80ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 -- 172.21.15.185:0/1342407380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97b8099d80 msgr2=0x7f97b8063de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:31.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97b8099d80 0x7f97b8063de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 -- 172.21.15.185:0/1342407380 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80aca90 con 0x7f97b80b2760 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97b5d9b700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f97b8067e20 tx=0x7f97a0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a001b5e0 con 0x7f97b80b2760 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a001b740 con 0x7f97b80b2760 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b80642d0 con 0x7f97b80b2760 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a001bac0 con 0x7f97b80b2760 2020-11-16T22:22:31.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b8064820 con 0x7f97b80b2760 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.806+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b805d4e0 con 0x7f97b80b2760 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.810+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f97a0017070 con 0x7f97b80b2760 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.810+0000 7f97ae7fc700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 0x7f9794057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.810+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f97a006fd30 con 0x7f97b80b2760 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.810+0000 7f97b659c700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 0x7f9794057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.810+0000 7f97b659c700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 0x7f9794057550 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f97b809d520 tx=0x7f97a8006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:31.208 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.814+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f97a0045f40 con 0x7f97b80b2760 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.970+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b80626c0 con 0x7f97b80b2760 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97ae7fc700 1 -- 172.21.15.185:0/1342407380 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16842 (secure 0 0 0) 0x7f97a0015070 con 0x7f97b80b2760 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 msgr2=0x7f9794057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 0x7f9794057550 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f97b809d520 tx=0x7f97a8006df0).stop 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 msgr2=0x7f97b80ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:31.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f97b8067e20 tx=0x7f97a0016000).stop 2020-11-16T22:22:31.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 shutdown_connections 2020-11-16T22:22:31.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f9794055090 0x7f9794057550 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f97b80b4140 0x7f97b80ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97b8099d80 0x7f97b8063de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 --2- 172.21.15.185:0/1342407380 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f97b80b2760 0x7f97b80ae080 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:31.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 >> 172.21.15.185:0/1342407380 conn(0x7f97b8061240 msgr2=0x7f97b80b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:31.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 shutdown_connections 2020-11-16T22:22:31.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:30.974+0000 7f97bce70700 1 -- 172.21.15.185:0/1342407380 wait complete. 2020-11-16T22:22:31.415 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:31.415 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":83,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:22:30.747593+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:22:27.765410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:22:28.746050+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:32.026 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-16T22:22:32.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.170+0000 7f087bfff700 1 Processor -- start 2020-11-16T22:22:32.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- start start 2020-11-16T22:22:32.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f087c0b2e80 0x7f087c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0ad200 con 0x7f087c13bdf0 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0ae100 con 0x7f087c0b2e80 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c13c280 con 0x7f087c0b39d0 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57268/0 (socket says 172.21.15.185:57268) 2020-11-16T22:22:32.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 -- 172.21.15.185:0/3102196875 learned_addr learned my addr 172.21.15.185:0/3102196875 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:32.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087affd700 1 --2- 172.21.15.185:0/3102196875 >> v2:172.21.15.96:3300/0 conn(0x7f087c0b2e80 0x7f087c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 -- 172.21.15.185:0/3102196875 >> v2:172.21.15.96:3300/0 conn(0x7f087c0b2e80 msgr2=0x7f087c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:32.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 --2- 172.21.15.185:0/3102196875 >> v2:172.21.15.96:3300/0 conn(0x7f087c0b2e80 0x7f087c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:32.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 -- 172.21.15.185:0/3102196875 >> v1:172.21.15.96:6789/0 conn(0x7f087c13bdf0 legacy=0x7f087c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:32.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 -- 172.21.15.185:0/3102196875 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0ac000 con 0x7f087c0b39d0 2020-11-16T22:22:32.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087a7fc700 1 --2- 172.21.15.185:0/3102196875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f0864006fd0 tx=0x7f0864006eb0).ready entity=mon.2 client_cookie=8540a8be1b0c5526 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:32.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f0879ffb700 1 -- 172.21.15.185:0/3102196875 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640098d0 con 0x7f087c0b39d0 2020-11-16T22:22:32.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f0879ffb700 1 -- 172.21.15.185:0/3102196875 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864009a30 con 0x7f087c0b39d0 2020-11-16T22:22:32.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f0879ffb700 1 -- 172.21.15.185:0/3102196875 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864009db0 con 0x7f087c0b39d0 2020-11-16T22:22:32.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- 172.21.15.185:0/3102196875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 msgr2=0x7f087c13b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:32.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 --2- 172.21.15.185:0/3102196875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f0864006fd0 tx=0x7f0864006eb0).stop 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- 172.21.15.185:0/3102196875 shutdown_connections 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 --2- 172.21.15.185:0/3102196875 >> v2:172.21.15.96:3300/0 conn(0x7f087c0b2e80 0x7f087c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 --2- 172.21.15.185:0/3102196875 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b39d0 0x7f087c13b900 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- 172.21.15.185:0/3102196875 >> 172.21.15.185:0/3102196875 conn(0x7f087c060130 msgr2=0x7f087c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- 172.21.15.185:0/3102196875 shutdown_connections 2020-11-16T22:22:32.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.174+0000 7f087bfff700 1 -- 172.21.15.185:0/3102196875 wait complete. 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 Processor -- start 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- start start 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f087c0b39d0 0x7f087c142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 0x7f087c13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c154fb0 con 0x7f087c0b39d0 2020-11-16T22:22:32.376 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c1551d0 con 0x7f087c13bdf0 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c155600 con 0x7f087c0b2e80 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 0x7f087c13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087affd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57274/0 (socket says 172.21.15.185:57274) 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087affd700 1 -- 172.21.15.185:0/3582702188 learned_addr learned my addr 172.21.15.185:0/3582702188 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087a7fc700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f087c0b39d0 0x7f087c142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 -- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 msgr2=0x7f087c1428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:32.377 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 -- 172.21.15.185:0/3582702188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f087c0b39d0 msgr2=0x7f087c142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f087c0b39d0 0x7f087c142dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 -- 172.21.15.185:0/3582702188 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0b70b0 con 0x7f087c13bdf0 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087affd700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087b7fe700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 0x7f087c13fd50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f087400ca90 tx=0x7f0874007e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400ce60 con 0x7f087c13bdf0 2020-11-16T22:22:32.378 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400a730 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c13c470 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400aab0 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.178+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c0b6300 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c13c600 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f0874019030 con 0x7f087c13bdf0 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f0872ffd700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 0x7f0858057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:32.379 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f087affd700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 0x7f0858057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:32.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f087406d550 con 0x7f087c13bdf0 2020-11-16T22:22:32.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.182+0000 7f087affd700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 0x7f0858057520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f087c143890 tx=0x7f086c008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:32.380 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.190+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f08740437b0 con 0x7f087c13bdf0 2020-11-16T22:22:32.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:32.394+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f087c05d200 con 0x7f087c13bdf0 2020-11-16T22:22:33.089 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.098+0000 7f0872ffd700 1 -- 172.21.15.185:0/3582702188 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0874046dd0 con 0x7f087c13bdf0 2020-11-16T22:22:33.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 msgr2=0x7f0858057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 0x7f0858057520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f087c143890 tx=0x7f086c008040).stop 2020-11-16T22:22:33.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 msgr2=0x7f087c13fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 0x7f087c13fd50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f087400ca90 tx=0x7f0874007e20).stop 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 shutdown_connections 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f0858055060 0x7f0858057520 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f087c13bdf0 0x7f087c13fd50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f087c0b2e80 0x7f087c1428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 --2- 172.21.15.185:0/3582702188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f087c0b39d0 0x7f087c142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 >> 172.21.15.185:0/3582702188 conn(0x7f087c060130 msgr2=0x7f087c1441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 shutdown_connections 2020-11-16T22:22:33.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.102+0000 7f087bfff700 1 -- 172.21.15.185:0/3582702188 wait complete. 2020-11-16T22:22:33.107 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:33.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 Processor -- start 2020-11-16T22:22:33.231 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- start start 2020-11-16T22:22:33.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb5d80b17e0 0x7fb5d8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb5d80b2160 0x7fb5d814bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d8098cc0 con 0x7fb5d8066240 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80b3950 con 0x7fb5d80b2160 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80b3bf0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb5d80b17e0 0x7fb5d8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d759e700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb5d8066240 0x7fb5d8065630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57276/0 (socket says 172.21.15.185:57276) 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d759e700 1 -- 172.21.15.185:0/3449691666 learned_addr learned my addr 172.21.15.185:0/3449691666 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:33.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d659c700 1 --2- 172.21.15.185:0/3449691666 >> v2:172.21.15.96:3300/0 conn(0x7fb5d80b2160 0x7fb5d814bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 -- 172.21.15.185:0/3449691666 >> v2:172.21.15.96:3300/0 conn(0x7fb5d80b2160 msgr2=0x7fb5d814bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 --2- 172.21.15.185:0/3449691666 >> v2:172.21.15.96:3300/0 conn(0x7fb5d80b2160 0x7fb5d814bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 -- 172.21.15.185:0/3449691666 >> v1:172.21.15.185:6789/0 conn(0x7fb5d8066240 legacy=0x7fb5d8065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d5d9b700 1 -- 172.21.15.185:0/3449691666 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809267894 0 0) 0x7fb5d8098cc0 con 0x7fb5d8066240 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 -- 172.21.15.185:0/3449691666 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8067b10 con 0x7fb5d80b17e0 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d6d9d700 1 --2- 172.21.15.185:0/3449691666 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b17e0 0x7fb5d8067fb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fb5d80651a0 tx=0x7fb5c8006df0).ready entity=mon.1 client_cookie=69fbfdfa15d00d39 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d5d9b700 1 -- 172.21.15.185:0/3449691666 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80097b0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.235 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d5d9b700 1 -- 172.21.15.185:0/3449691666 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8009910 con 0x7fb5d80b17e0 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5d5d9b700 1 -- 172.21.15.185:0/3449691666 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8009c90 con 0x7fb5d80b17e0 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- 172.21.15.185:0/3449691666 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b17e0 msgr2=0x7fb5d8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3449691666 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b17e0 0x7fb5d8067fb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7fb5d80651a0 tx=0x7fb5c8006df0).stop 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- 172.21.15.185:0/3449691666 shutdown_connections 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3449691666 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b17e0 0x7fb5d8067fb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3449691666 >> v2:172.21.15.96:3300/0 conn(0x7fb5d80b2160 0x7fb5d814bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.236 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- 172.21.15.185:0/3449691666 >> 172.21.15.185:0/3449691666 conn(0x7fb5d80604a0 msgr2=0x7fb5d80608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- 172.21.15.185:0/3449691666 shutdown_connections 2020-11-16T22:22:33.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.242+0000 7fb5dd177700 1 -- 172.21.15.185:0/3449691666 wait complete. 2020-11-16T22:22:33.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 Processor -- start 2020-11-16T22:22:33.237 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 -- start start 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d8066240 0x7fb5d809b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 0x7fb5d814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 0x7fb5d814c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d81584a0 con 0x7fb5d8066240 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8158710 con 0x7fb5d80b2160 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5dd177700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80adbe0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5d759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 0x7fb5d814c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 0x7fb5d814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5d6d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d8066240 0x7fb5d809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5d759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 0x7fb5d814c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35616/0 (socket says 172.21.15.185:35616) 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.246+0000 7fb5d759e700 1 -- 172.21.15.185:0/3113405494 learned_addr learned my addr 172.21.15.185:0/3113405494 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 -- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 msgr2=0x7fb5d814c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 0x7fb5d814c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 -- 172.21.15.185:0/3113405494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d8066240 msgr2=0x7fb5d809b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d8066240 0x7fb5d809b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 -- 172.21.15.185:0/3113405494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d814cbb0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.241 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5d659c700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 0x7fb5d814c1d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fb5c000eba0 tx=0x7fb5c0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000c820 con 0x7fb5d80b17e0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000c980 con 0x7fb5d80b17e0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000cd00 con 0x7fb5d80b17e0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d809b900 con 0x7fb5d80b17e0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d809c240 con 0x7fb5d80b17e0 2020-11-16T22:22:33.242 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d80adda0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.250+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb5c0012600 con 0x7fb5d80b17e0 2020-11-16T22:22:33.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.254+0000 7fb5ceffd700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 0x7fb5b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.243 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.254+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb5c006dfc0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.254+0000 7fb5d6d9d700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 0x7fb5b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.244 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.254+0000 7fb5d6d9d700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 0x7fb5b4057520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb5c8006e30 tx=0x7fb5c8015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.258+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb5c0044220 con 0x7fb5d80b17e0 2020-11-16T22:22:33.453 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.462+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d81588d0 con 0x7fb5d80b17e0 2020-11-16T22:22:33.454 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.462+0000 7fb5ceffd700 1 -- 172.21.15.185:0/3113405494 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+2491 (secure 0 0 0) 0x7fb5c000ce60 con 0x7fb5d80b17e0 2020-11-16T22:22:33.454 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:33.454 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":101,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":97},{"gid":7830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:33.095134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:33.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 msgr2=0x7fb5b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 0x7fb5b4057520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb5c8006e30 tx=0x7fb5c8015000).stop 2020-11-16T22:22:33.456 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 msgr2=0x7fb5d814c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 0x7fb5d814c1d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fb5c000eba0 tx=0x7fb5c0006df0).stop 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 shutdown_connections 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb5b4055060 0x7fb5b4057520 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb5d80b2160 0x7fb5d814c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d8066240 0x7fb5d809b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.457 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 --2- 172.21.15.185:0/3113405494 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb5d80b17e0 0x7fb5d814c1d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 >> 172.21.15.185:0/3113405494 conn(0x7fb5d80604a0 msgr2=0x7fb5d80b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 shutdown_connections 2020-11-16T22:22:33.458 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.466+0000 7fb5dd177700 1 -- 172.21.15.185:0/3113405494 wait complete. 2020-11-16T22:22:33.458 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 101 2020-11-16T22:22:33.469 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 Processor -- start 2020-11-16T22:22:33.593 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 -- start start 2020-11-16T22:22:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef8147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef8061fe0 con 0x7f4ef80b3090 2020-11-16T22:22:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef80b0fe0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80b2b60 con 0x7f4ef80ad110 2020-11-16T22:22:33.594 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef5d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4ef80b1d30 0x7f4ef81477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57290/0 (socket says 172.21.15.185:57290) 2020-11-16T22:22:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef5d9b700 1 -- 172.21.15.185:0/115308039 learned_addr learned my addr 172.21.15.185:0/115308039 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:33.595 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef6d9d700 1 --2- 172.21.15.185:0/115308039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef8147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954023266 0 0) 0x7f4ef80b0fe0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.596 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4edc003240 con 0x7f4ef80b1d30 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369211663 0 0) 0x7f4ef8061fe0 con 0x7f4ef80b3090 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.602+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef80b0fe0 con 0x7f4ef80b3090 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2363125312 0 0) 0x7f4edc003240 con 0x7f4ef80b1d30 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 -- 172.21.15.185:0/115308039 >> v1:172.21.15.185:6789/0 conn(0x7f4ef80b1d30 legacy=0x7f4ef81477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 -- 172.21.15.185:0/115308039 >> v1:172.21.15.96:6789/0 conn(0x7f4ef80b3090 legacy=0x7f4ef814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 -- 172.21.15.185:0/115308039 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80b1280 con 0x7f4ef80ad110 2020-11-16T22:22:33.597 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 --2- 172.21.15.185:0/115308039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef8147ec0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f4ef000b950 tx=0x7f4ef0006df0).ready entity=mon.2 client_cookie=493a1e5a6856675f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0001860 con 0x7f4ef80ad110 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef0011400 con 0x7f4ef80ad110 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef559a700 1 -- 172.21.15.185:0/115308039 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0011780 con 0x7f4ef80ad110 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- 172.21.15.185:0/115308039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 msgr2=0x7f4ef8147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 --2- 172.21.15.185:0/115308039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef8147ec0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f4ef000b950 tx=0x7f4ef0006df0).stop 2020-11-16T22:22:33.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- 172.21.15.185:0/115308039 shutdown_connections 2020-11-16T22:22:33.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 --2- 172.21.15.185:0/115308039 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef8147ec0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- 172.21.15.185:0/115308039 >> 172.21.15.185:0/115308039 conn(0x7f4ef8060130 msgr2=0x7f4ef8060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- 172.21.15.185:0/115308039 shutdown_connections 2020-11-16T22:22:33.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- 172.21.15.185:0/115308039 wait complete. 2020-11-16T22:22:33.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 Processor -- start 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- start start 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef814a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 0x7f4ef81122c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8111330 con 0x7f4ef80b1d30 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef81115d0 con 0x7f4ef80b3090 2020-11-16T22:22:33.601 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef759e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8111870 con 0x7f4ef80ad110 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 0x7f4ef81122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef814a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 0x7f4ef81122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35630/0 (socket says 172.21.15.185:35630) 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.185:39922/0 (socket says 172.21.15.185:39922) 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.606+0000 7f4ef6d9d700 1 -- 172.21.15.185:0/995295504 learned_addr learned my addr 172.21.15.185:0/995295504 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:33.602 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 -- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 msgr2=0x7f4ef814a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef814a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 -- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 msgr2=0x7f4ef81122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 0x7f4ef81122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 -- 172.21.15.185:0/995295504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8111e20 con 0x7f4ef80b1d30 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef5d9b700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f4ee000ce30 tx=0x7f4ee00018a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0005b80 con 0x7f4ef80b1d30 2020-11-16T22:22:33.603 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0005ce0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef811d6c0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000aa30 con 0x7f4ef80b1d30 2020-11-16T22:22:33.604 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef8112ed0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef805d200 con 0x7f4ef80b1d30 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4ee0010030 con 0x7f4ef80b1d30 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 0x7f4ec8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4ee006e5e0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.610+0000 7f4ef659c700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 0x7f4ec8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.605 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.614+0000 7f4ef659c700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 0x7f4ec8057520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f4ee80033d0 tx=0x7f4ee8007a20).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.618+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4ee0044840 con 0x7f4ef80b1d30 2020-11-16T22:22:33.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.774+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef80b2ca0 con 0x7f4ef80b1d30 2020-11-16T22:22:33.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.774+0000 7f4eee7fc700 1 -- 172.21.15.185:0/995295504 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f4ee0047e60 con 0x7f4ef80b1d30 2020-11-16T22:22:33.766 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:33.766 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":84,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:22:31.765374+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:22:27.765410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:22:28.746050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:33.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 msgr2=0x7f4ec8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 0x7f4ec8057520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f4ee80033d0 tx=0x7f4ee8007a20).stop 2020-11-16T22:22:33.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 msgr2=0x7f4ef8146c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f4ee000ce30 tx=0x7f4ee00018a0).stop 2020-11-16T22:22:33.768 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 shutdown_connections 2020-11-16T22:22:33.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4ec8055060 0x7f4ec8057520 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ef80b3090 0x7f4ef81122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ef80ad110 0x7f4ef814a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.769 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 --2- 172.21.15.185:0/995295504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ef80b1d30 0x7f4ef8146c40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 >> 172.21.15.185:0/995295504 conn(0x7f4ef8060130 msgr2=0x7f4ef80b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 shutdown_connections 2020-11-16T22:22:33.770 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.778+0000 7f4ef759e700 1 -- 172.21.15.185:0/995295504 wait complete. 2020-11-16T22:22:33.782 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-16T22:22:33.920 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 Processor -- start 2020-11-16T22:22:33.921 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 -- start start 2020-11-16T22:22:33.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4210062ea0 0x7f42100660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.922 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4210140fa0 0x7f4210141410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100665a0 con 0x7f42100b7eb0 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4210063ac0 con 0x7f4210140fa0 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4216584700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4210063d60 con 0x7f4210062ea0 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4214d81700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4210062ea0 0x7f42100660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4214d81700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4210062ea0 0x7f42100660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35636/0 (socket says 172.21.15.185:35636) 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4214d81700 1 -- 172.21.15.185:0/3681944976 learned_addr learned my addr 172.21.15.185:0/3681944976 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:33.923 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.930+0000 7f4215d83700 1 --2- 172.21.15.185:0/3681944976 >> v2:172.21.15.96:3300/0 conn(0x7f4210140fa0 0x7f4210141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 -- 172.21.15.185:0/3681944976 >> v2:172.21.15.96:3300/0 conn(0x7f4210140fa0 msgr2=0x7f4210141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 --2- 172.21.15.185:0/3681944976 >> v2:172.21.15.96:3300/0 conn(0x7f4210140fa0 0x7f4210141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 -- 172.21.15.185:0/3681944976 >> v1:172.21.15.96:6789/0 conn(0x7f42100b7eb0 legacy=0x7f4210145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 -- 172.21.15.185:0/3681944976 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210145710 con 0x7f4210062ea0 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 --2- 172.21.15.185:0/3681944976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f42100660b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f42100b1b80 tx=0x7f4200006eb0).ready entity=mon.1 client_cookie=5a93cb214fac3fb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.925 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4207fff700 1 -- 172.21.15.185:0/3681944976 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200009810 con 0x7f4210062ea0 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4207fff700 1 -- 172.21.15.185:0/3681944976 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200009970 con 0x7f4210062ea0 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4207fff700 1 -- 172.21.15.185:0/3681944976 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200009cf0 con 0x7f4210062ea0 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- 172.21.15.185:0/3681944976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 msgr2=0x7f42100660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- 172.21.15.185:0/3681944976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f42100660b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f42100b1b80 tx=0x7f4200006eb0).stop 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- 172.21.15.185:0/3681944976 shutdown_connections 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- 172.21.15.185:0/3681944976 >> v2:172.21.15.96:3300/0 conn(0x7f4210140fa0 0x7f4210141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.926 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- 172.21.15.185:0/3681944976 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f42100660b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- 172.21.15.185:0/3681944976 >> 172.21.15.185:0/3681944976 conn(0x7f4210061d10 msgr2=0x7f42100b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:33.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- 172.21.15.185:0/3681944976 shutdown_connections 2020-11-16T22:22:33.927 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- 172.21.15.185:0/3681944976 wait complete. 2020-11-16T22:22:33.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 Processor -- start 2020-11-16T22:22:33.928 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- start start 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42100b7eb0 0x7f4210142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4210140fa0 0x7f421013f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42101153e0 con 0x7f42100b7eb0 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42101155d0 con 0x7f4210062ea0 2020-11-16T22:22:33.929 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4216584700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4210115870 con 0x7f4210140fa0 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4215582700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4215d83700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4210140fa0 0x7f421013f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4215582700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35640/0 (socket says 172.21.15.185:35640) 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4215582700 1 -- 172.21.15.185:0/2971372718 learned_addr learned my addr 172.21.15.185:0/2971372718 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.934+0000 7f4214d81700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42100b7eb0 0x7f4210142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 -- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4210140fa0 msgr2=0x7f421013f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4210140fa0 0x7f421013f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 -- 172.21.15.185:0/2971372718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42100b7eb0 msgr2=0x7f4210142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42100b7eb0 0x7f4210142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 -- 172.21.15.185:0/2971372718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210115b10 con 0x7f4210062ea0 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4215582700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f420c00c6c0 tx=0x7f420c007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c007020 con 0x7f4210062ea0 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4210115db0 con 0x7f4210062ea0 2020-11-16T22:22:33.931 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c007180 con 0x7f4210062ea0 2020-11-16T22:22:33.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c006540 con 0x7f4210062ea0 2020-11-16T22:22:33.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4210142730 con 0x7f4210062ea0 2020-11-16T22:22:33.932 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421005d200 con 0x7f4210062ea0 2020-11-16T22:22:33.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f420c010070 con 0x7f4210062ea0 2020-11-16T22:22:33.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 0x7f41e8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:33.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f420c06d120 con 0x7f4210062ea0 2020-11-16T22:22:33.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.938+0000 7f4214d81700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 0x7f41e8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:33.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.942+0000 7f4214d81700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 0x7f41e8057520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4200006ef0 tx=0x7f4200015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:33.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:33.946+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f420c043400 con 0x7f4210062ea0 2020-11-16T22:22:34.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.154+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f4210065410 con 0x7f4210062ea0 2020-11-16T22:22:34.152 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.162+0000 7f42057fa700 1 -- 172.21.15.185:0/2971372718 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f420c046a20 con 0x7f4210062ea0 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 msgr2=0x7f41e8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 0x7f41e8057520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4200006ef0 tx=0x7f4200015000).stop 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 msgr2=0x7f4210153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f420c00c6c0 tx=0x7f420c007c40).stop 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 shutdown_connections 2020-11-16T22:22:34.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f41e8055060 0x7f41e8057520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4210062ea0 0x7f4210153cd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4210140fa0 0x7f421013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 --2- 172.21.15.185:0/2971372718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42100b7eb0 0x7f4210142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 >> 172.21.15.185:0/2971372718 conn(0x7f4210061d10 msgr2=0x7f42101400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 shutdown_connections 2020-11-16T22:22:34.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.166+0000 7f4216584700 1 -- 172.21.15.185:0/2971372718 wait complete. 2020-11-16T22:22:34.157 INFO:teuthology.orchestra.run.smithi185.stderr:updated caps for client.0 2020-11-16T22:22:34.174 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:34.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 Processor -- start 2020-11-16T22:22:34.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- start start 2020-11-16T22:22:34.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f769c0b17e0 0x7f769c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c098cc0 con 0x7f769c066240 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0b3950 con 0x7f769c0b2160 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c14c4f0 con 0x7f769c0b17e0 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f7699d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f769c0b2160 0x7f769c14bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57320/0 (socket says 172.21.15.185:57320) 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f7699d9b700 1 -- 172.21.15.185:0/18530898 learned_addr learned my addr 172.21.15.185:0/18530898 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769a59c700 1 --2- 172.21.15.185:0/18530898 >> v2:172.21.15.96:3300/0 conn(0x7f769c0b17e0 0x7f769c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540155181 0 0) 0x7f769c0b3950 con 0x7f769c0b2160 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767c003240 con 0x7f769c0b2160 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928822477 0 0) 0x7f769c098cc0 con 0x7f769c066240 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f769c0b3950 con 0x7f769c066240 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v1:172.21.15.185:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2292157259 0 0) 0x7f767c003240 con 0x7f769c0b2160 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 >> v2:172.21.15.96:3300/0 conn(0x7f769c0b17e0 msgr2=0x7f769c067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:34.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 --2- 172.21.15.185:0/18530898 >> v2:172.21.15.96:3300/0 conn(0x7f769c0b17e0 0x7f769c067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 >> v1:172.21.15.96:6789/0 conn(0x7f769c066240 legacy=0x7f769c065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> v1:172.21.15.185:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0b3bf0 con 0x7f769c0b2160 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v1:172.21.15.185:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1491608937 0 0) 0x7f76840014e0 con 0x7f769c0b2160 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 >> v1:172.21.15.185:6789/0 conn(0x7f769c0b2160 legacy=0x7f769c14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f767c0038c0 0x7f767c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f767c0044e0 0x7f767c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.301 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f767c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c14c4f0 con 0x7f767c0044e0 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c00bc20 con 0x7f767c004e50 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c00bd60 con 0x7f767c0038c0 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769a59c700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f767c0038c0 0x7f767c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f767c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f7699d9b700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f767c0044e0 0x7f767c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 -- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f767c0038c0 msgr2=0x7f767c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f767c0038c0 0x7f767c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 -- 172.21.15.185:0/18530898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f767c0044e0 msgr2=0x7f767c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f767c0044e0 0x7f767c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 -- 172.21.15.185:0/18530898 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c003240 con 0x7f767c004e50 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f7699d9b700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f767c0044e0 0x7f767c004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:34.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 -- 172.21.15.185:0/18530898 reap_dead start 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769ad9d700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f767c00b730 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f76940018d0 tx=0x7f7694007b00).ready entity=mon.1 client_cookie=b63847e279107713 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694007020 con 0x7f767c004e50 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694007180 con 0x7f767c004e50 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f769959a700 1 -- 172.21.15.185:0/18530898 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400c830 con 0x7f767c004e50 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 msgr2=0x7f767c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f767c00b730 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f76940018d0 tx=0x7f7694007b00).stop 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- 172.21.15.185:0/18530898 shutdown_connections 2020-11-16T22:22:34.304 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 --2- 172.21.15.185:0/18530898 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f767c00b730 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- 172.21.15.185:0/18530898 >> 172.21.15.185:0/18530898 conn(0x7f769c0604a0 msgr2=0x7f769c0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:34.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- 172.21.15.185:0/18530898 shutdown_connections 2020-11-16T22:22:34.305 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.306+0000 7f76a0c76700 1 -- 172.21.15.185:0/18530898 wait complete. 2020-11-16T22:22:34.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 Processor -- start 2020-11-16T22:22:34.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- start start 2020-11-16T22:22:34.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f769c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f769c0ad850 0x7f769c0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c157a70 con 0x7f769c0ad850 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c157c50 con 0x7f767c004e50 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0ac0b0 con 0x7f769c0b0720 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57338/0 (socket says 172.21.15.185:57338) 2020-11-16T22:22:34.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f769a59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f769c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 -- 172.21.15.185:0/2858266618 learned_addr learned my addr 172.21.15.185:0/2858266618 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f769ad9d700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f769c0ad850 0x7f769c0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 -- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 msgr2=0x7f769c0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f769c0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 -- 172.21.15.185:0/2858266618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f769c0ad850 msgr2=0x7f769c0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f769c0ad850 0x7f769c0adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 -- 172.21.15.185:0/2858266618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0abe10 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f7699d9b700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f7684000f20 tx=0x7f769c0678b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400a730 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7684010040 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400aaa0 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c157eb0 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.310+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c0b02c0 con 0x7f769c0b0720 2020-11-16T22:22:34.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c0b3dd0 con 0x7f769c0b0720 2020-11-16T22:22:34.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f768401b020 con 0x7f769c0b0720 2020-11-16T22:22:34.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f76927fc700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 0x7f7678057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:34.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f768406d260 con 0x7f769c0b0720 2020-11-16T22:22:34.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f769a59c700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 0x7f7678057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:34.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.314+0000 7f769a59c700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 0x7f7678057520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f768c0094d0 tx=0x7f768c0091d0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:34.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.318+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f76840433b0 con 0x7f769c0b0720 2020-11-16T22:22:34.390 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:22:34.398+0000 7f5dfcaae700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-16T22:22:34.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.522+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f769c05d4e0 con 0x7f769c0b0720 2020-11-16T22:22:34.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76927fc700 1 -- 172.21.15.185:0/2858266618 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+2491 (secure 0 0 0) 0x7f76840469d0 con 0x7f769c0b0720 2020-11-16T22:22:34.516 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:34.516 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":101,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":97},{"gid":7830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:33.095134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 msgr2=0x7f7678057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 0x7f7678057520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f768c0094d0 tx=0x7f768c0091d0).stop 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 msgr2=0x7f769c158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f7684000f20 tx=0x7f769c0678b0).stop 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 shutdown_connections 2020-11-16T22:22:34.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f7678055060 0x7f7678057520 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f767c004e50 0x7f769c0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f769c0ad850 0x7f769c0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 --2- 172.21.15.185:0/2858266618 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f769c0b0720 0x7f769c158350 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 >> 172.21.15.185:0/2858266618 conn(0x7f769c0604a0 msgr2=0x7f769c067ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 shutdown_connections 2020-11-16T22:22:34.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:34.526+0000 7f76a0c76700 1 -- 172.21.15.185:0/2858266618 wait complete. 2020-11-16T22:22:34.520 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 101 2020-11-16T22:22:34.840 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:22:29.743585+0000', 'modified': '2020-11-16T22:22:33.095134+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7993': {'gid': 7993, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.185:6835/1118138229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1118138229}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1118138229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:22:34.841 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:22:34.841 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:22:34.976 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7993,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.2301287570000001,"mdsmap_epoch":101,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":10.477005728} 2020-11-16T22:22:34.977 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7993, "mdsmap_epoch": 101, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 5.230128757, "state": "up:active", "uptime": 10.477005728, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:22:34.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-16T22:22:34.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2020-11-16T22:22:34.978 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-16T22:22:34.978 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-16T22:22:34.979 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-16T22:22:34.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-16T22:22:34.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-16T22:22:34.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:22:34.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:34.980 INFO:teuthology.orchestra.run.smithi062:> ip addr 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2020-11-16T22:22:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:22:35.000 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-16T22:22:35.001 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6465:d4ff:fee9:8dde/64 scope link 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 52:8c:68:b2:97:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-16T22:22:35.002 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::508c:68ff:feb2:97fb/64 scope link 2020-11-16T22:22:35.003 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.003 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-16T22:22:35.003 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5a:4e:63:45:8c:18 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-16T22:22:35.003 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::584e:63ff:fe45:8c18/64 scope link 2020-11-16T22:22:35.003 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2020-11-16T22:22:35.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:35.004 INFO:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-16T22:22:35.081 INFO:teuthology.orchestra.run.smithi062.stdout:1 2020-11-16T22:22:35.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:35.085 INFO:teuthology.orchestra.run.smithi062:> route 2020-11-16T22:22:35.102 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2020-11-16T22:22:35.103 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-16T22:22:35.103 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-16T22:22:35.103 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-16T22:22:35.103 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-16T22:22:35.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:35.104 INFO:teuthology.orchestra.run.smithi062:> (cd / && exec sudo bash -c ' 2020-11-16T22:22:35.104 INFO:teuthology.orchestra.run.smithi062:> set -e 2020-11-16T22:22:35.104 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-16T22:22:35.105 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-16T22:22:35.105 INFO:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-16T22:22:35.105 INFO:teuthology.orchestra.run.smithi062:> ') 2020-11-16T22:22:35.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:35.206 INFO:teuthology.orchestra.run.smithi062:> ip netns list 2020-11-16T22:22:35.221 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2020-11-16T22:22:35.221 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-16T22:22:35.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-16T22:22:35.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:35.223 INFO:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2020-11-16T22:22:35.252 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T22:22:35.381 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.390+0000 7f5b2e1a2700 1 Processor -- start 2020-11-16T22:22:35.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.390+0000 7f5b2e1a2700 1 -- start start 2020-11-16T22:22:35.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5b280b8ab0 0x7f5b281475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:35.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b280b4060 con 0x7f5b280b30e0 2020-11-16T22:22:35.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b280b4430 con 0x7f5b280b86d0 2020-11-16T22:22:35.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b280b46d0 con 0x7f5b280b8ab0 2020-11-16T22:22:35.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5b280b8ab0 0x7f5b281475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:35.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f5b280b8ab0 0x7f5b281475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35670/0 (socket says 172.21.15.185:35670) 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 -- 172.21.15.185:0/11820869 learned_addr learned my addr 172.21.15.185:0/11820869 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d1a0700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f5b280b86d0 0x7f5b280b38b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57338/0 (socket says 172.21.15.185:57338) 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 -- 172.21.15.185:0/11820869 >> v1:172.21.15.185:6789/0 conn(0x7f5b280b86d0 legacy=0x7f5b280b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 -- 172.21.15.185:0/11820869 >> v1:172.21.15.96:6789/0 conn(0x7f5b280b30e0 legacy=0x7f5b28062970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 -- 172.21.15.185:0/11820869 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b280b4970 con 0x7f5b280b8ab0 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2d9a1700 1 --2- 172.21.15.185:0/11820869 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b8ab0 0x7f5b281475c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f5b20009d50 tx=0x7f5b20009290).ready entity=mon.1 client_cookie=92b4b509e0e2d4eb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:35.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b1ffff700 1 -- 172.21.15.185:0/11820869 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000bc80 con 0x7f5b280b8ab0 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b1ffff700 1 -- 172.21.15.185:0/11820869 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2000bde0 con 0x7f5b280b8ab0 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b1ffff700 1 -- 172.21.15.185:0/11820869 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20011610 con 0x7f5b280b8ab0 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/11820869 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b8ab0 msgr2=0x7f5b281475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/11820869 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b8ab0 0x7f5b281475c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f5b20009d50 tx=0x7f5b20009290).stop 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/11820869 shutdown_connections 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/11820869 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b8ab0 0x7f5b281475c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/11820869 >> 172.21.15.185:0/11820869 conn(0x7f5b280645d0 msgr2=0x7f5b280649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:35.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/11820869 shutdown_connections 2020-11-16T22:22:35.387 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/11820869 wait complete. 2020-11-16T22:22:35.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 Processor -- start 2020-11-16T22:22:35.388 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.394+0000 7f5b2e1a2700 1 -- start start 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 0x7f5b280aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b280b8ab0 0x7f5b28156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280b2790 con 0x7f5b280b8ab0 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280acf80 con 0x7f5b280b30e0 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28146dd0 con 0x7f5b280b86d0 2020-11-16T22:22:35.389 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2d1a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 0x7f5b280aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57352/0 (socket says 172.21.15.185:57352) 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2d1a0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 0x7f5b280aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35674/0 (socket says 172.21.15.185:35674) 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 -- 172.21.15.185:0/1613498245 learned_addr learned my addr 172.21.15.185:0/1613498245 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2d9a1700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b280b8ab0 0x7f5b28156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 -- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 msgr2=0x7f5b280aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 0x7f5b280aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 -- 172.21.15.185:0/1613498245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b280b8ab0 msgr2=0x7f5b28156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b280b8ab0 0x7f5b28156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 -- 172.21.15.185:0/1613498245 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28157570 con 0x7f5b280b86d0 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2c99f700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f5b2400c740 tx=0x7f5b24007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24007020 con 0x7f5b280b86d0 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b24007180 con 0x7f5b280b86d0 2020-11-16T22:22:35.391 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b28147030 con 0x7f5b280b86d0 2020-11-16T22:22:35.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240064f0 con 0x7f5b280b86d0 2020-11-16T22:22:35.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b28147a40 con 0x7f5b280b86d0 2020-11-16T22:22:35.392 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.398+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2805d200 con 0x7f5b280b86d0 2020-11-16T22:22:35.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.402+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5b24010070 con 0x7f5b280b86d0 2020-11-16T22:22:35.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.402+0000 7f5b1d7fa700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 0x7f5b00057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:35.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.402+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5b2406d010 con 0x7f5b280b86d0 2020-11-16T22:22:35.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.402+0000 7f5b2d1a0700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 0x7f5b00057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:35.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.402+0000 7f5b2d1a0700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 0x7f5b00057520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5b18007270 tx=0x7f5b18007b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:35.396 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.406+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f5b24043270 con 0x7f5b280b86d0 2020-11-16T22:22:35.551 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.562+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b28068500 con 0x7f5b280b86d0 2020-11-16T22:22:35.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.562+0000 7f5b1d7fa700 1 -- 172.21.15.185:0/1613498245 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f5b240241d0 con 0x7f5b280b86d0 2020-11-16T22:22:35.553 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:35.553 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":84,"fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","created":"2020-11-16T22:13:47.140034+0000","modified":"2020-11-16T22:22:31.765374+0000","last_up_change":"2020-11-16T22:13:54.059816+0000","last_in_change":"2020-11-16T22:13:54.059816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T22:13:51.099755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T22:13:58.125677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-16T22:22:27.765410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-16T22:22:28.746050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bc3b03d0-fcaf-4134-b2f3-f83255339276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6808","nonce":15202},{"type":"v1","addr":"172.21.15.185:6809","nonce":15202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6810","nonce":15202},{"type":"v1","addr":"172.21.15.185:6811","nonce":15202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6817","nonce":15202},{"type":"v1","addr":"172.21.15.185:6818","nonce":15202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6813","nonce":15202},{"type":"v1","addr":"172.21.15.185:6815","nonce":15202}]},"public_addr":"172.21.15.185:6809/15202","cluster_addr":"172.21.15.185:6811/15202","heartbeat_back_addr":"172.21.15.185:6818/15202","heartbeat_front_addr":"172.21.15.185:6815/15202","state":["exists","up"]},{"osd":1,"uuid":"47286fad-a235-46ea-92f4-3e0b3dff9c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6812","nonce":15205},{"type":"v1","addr":"172.21.15.185:6814","nonce":15205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6816","nonce":15205},{"type":"v1","addr":"172.21.15.185:6819","nonce":15205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6822","nonce":15205},{"type":"v1","addr":"172.21.15.185:6823","nonce":15205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6820","nonce":15205},{"type":"v1","addr":"172.21.15.185:6821","nonce":15205}]},"public_addr":"172.21.15.185:6814/15205","cluster_addr":"172.21.15.185:6819/15205","heartbeat_back_addr":"172.21.15.185:6823/15205","heartbeat_front_addr":"172.21.15.185:6821/15205","state":["exists","up"]},{"osd":2,"uuid":"027cf982-cd16-41e5-93f0-339e136d5781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6800","nonce":15193},{"type":"v1","addr":"172.21.15.185:6801","nonce":15193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6802","nonce":15193},{"type":"v1","addr":"172.21.15.185:6803","nonce":15193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6806","nonce":15193},{"type":"v1","addr":"172.21.15.185:6807","nonce":15193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6804","nonce":15193},{"type":"v1","addr":"172.21.15.185:6805","nonce":15193}]},"public_addr":"172.21.15.185:6801/15193","cluster_addr":"172.21.15.185:6803/15193","heartbeat_back_addr":"172.21.15.185:6807/15193","heartbeat_front_addr":"172.21.15.185:6805/15193","state":["exists","up"]},{"osd":3,"uuid":"d27e1166-11a5-401b-b82d-8ed20760bd20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6824","nonce":15203},{"type":"v1","addr":"172.21.15.185:6825","nonce":15203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6826","nonce":15203},{"type":"v1","addr":"172.21.15.185:6827","nonce":15203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6830","nonce":15203},{"type":"v1","addr":"172.21.15.185:6831","nonce":15203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6828","nonce":15203},{"type":"v1","addr":"172.21.15.185:6829","nonce":15203}]},"public_addr":"172.21.15.185:6825/15203","cluster_addr":"172.21.15.185:6827/15203","heartbeat_back_addr":"172.21.15.185:6831/15203","heartbeat_front_addr":"172.21.15.185:6829/15203","state":["exists","up"]},{"osd":4,"uuid":"91b77cc0-739c-4dcc-ab56-82e9b9de75fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":15251},{"type":"v1","addr":"172.21.15.96:6801","nonce":15251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":15251},{"type":"v1","addr":"172.21.15.96:6803","nonce":15251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":15251},{"type":"v1","addr":"172.21.15.96:6807","nonce":15251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":15251},{"type":"v1","addr":"172.21.15.96:6805","nonce":15251}]},"public_addr":"172.21.15.96:6801/15251","cluster_addr":"172.21.15.96:6803/15251","heartbeat_back_addr":"172.21.15.96:6807/15251","heartbeat_front_addr":"172.21.15.96:6805/15251","state":["exists","up"]},{"osd":5,"uuid":"137c0d5c-501f-4955-a1c0-cd3daf2a78a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":15269},{"type":"v1","addr":"172.21.15.96:6812","nonce":15269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":15269},{"type":"v1","addr":"172.21.15.96:6819","nonce":15269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":15269},{"type":"v1","addr":"172.21.15.96:6831","nonce":15269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":15269},{"type":"v1","addr":"172.21.15.96:6824","nonce":15269}]},"public_addr":"172.21.15.96:6812/15269","cluster_addr":"172.21.15.96:6819/15269","heartbeat_back_addr":"172.21.15.96:6831/15269","heartbeat_front_addr":"172.21.15.96:6824/15269","state":["exists","up"]},{"osd":6,"uuid":"9c13a225-67c7-4d0e-8706-b3264ac0367d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":15253},{"type":"v1","addr":"172.21.15.96:6810","nonce":15253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":15253},{"type":"v1","addr":"172.21.15.96:6816","nonce":15253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6823","nonce":15253},{"type":"v1","addr":"172.21.15.96:6826","nonce":15253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6817","nonce":15253},{"type":"v1","addr":"172.21.15.96:6820","nonce":15253}]},"public_addr":"172.21.15.96:6810/15253","cluster_addr":"172.21.15.96:6816/15253","heartbeat_back_addr":"172.21.15.96:6826/15253","heartbeat_front_addr":"172.21.15.96:6820/15253","state":["exists","up"]},{"osd":7,"uuid":"c789075c-0198-4420-9039-6f6ae5d5f906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":15252},{"type":"v1","addr":"172.21.15.96:6814","nonce":15252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":15252},{"type":"v1","addr":"172.21.15.96:6821","nonce":15252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":15252},{"type":"v1","addr":"172.21.15.96:6829","nonce":15252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6825","nonce":15252},{"type":"v1","addr":"172.21.15.96:6827","nonce":15252}]},"public_addr":"172.21.15.96:6814/15252","cluster_addr":"172.21.15.96:6821/15252","heartbeat_back_addr":"172.21.15.96:6829/15252","heartbeat_front_addr":"172.21.15.96:6827/15252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.711616+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.698377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.678330+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.703068+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.704722+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.729004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.690190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T22:13:52.449639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T22:22:35.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 msgr2=0x7f5b00057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 0x7f5b00057520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5b18007270 tx=0x7f5b18007b80).stop 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 msgr2=0x7f5b281474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f5b2400c740 tx=0x7f5b24007c20).stop 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 shutdown_connections 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5b00055060 0x7f5b00057520 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5b280b30e0 0x7f5b280aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b280b8ab0 0x7f5b28156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 --2- 172.21.15.185:0/1613498245 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5b280b86d0 0x7f5b281474d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:35.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 >> 172.21.15.185:0/1613498245 conn(0x7f5b280645d0 msgr2=0x7f5b28142060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:35.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 shutdown_connections 2020-11-16T22:22:35.958 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:35.566+0000 7f5b2e1a2700 1 -- 172.21.15.185:0/1613498245 wait complete. 2020-11-16T22:22:35.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T22:13:51.099755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T22:13:58.125677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-16T22:22:27.765410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-16T22:22:28.746050+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-16T22:22:35.960 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T22:22:36.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 Processor -- start 2020-11-16T22:22:36.095 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 -- start start 2020-11-16T22:22:36.096 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 0x7fb280143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800612e0 con 0x7fb280062c30 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800b4bd0 con 0x7fb2800b39b0 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb2885e7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800656e0 con 0x7fb280143830 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb286b84700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 0x7fb280143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb285b82700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb2800b39b0 0x7fb280143120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57354/0 (socket says 172.21.15.185:57354) 2020-11-16T22:22:36.097 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb285b82700 1 -- 172.21.15.185:0/2295019114 learned_addr learned my addr 172.21.15.185:0/2295019114 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:36.098 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb286b84700 1 -- 172.21.15.185:0/2295019114 >> v1:172.21.15.185:6789/0 conn(0x7fb2800b39b0 legacy=0x7fb280143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb286b84700 1 -- 172.21.15.185:0/2295019114 >> v1:172.21.15.96:6789/0 conn(0x7fb280062c30 legacy=0x7fb2800b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb286b84700 1 -- 172.21.15.185:0/2295019114 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800619e0 con 0x7fb280143830 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb286b84700 1 --2- 172.21.15.185:0/2295019114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 0x7fb280143cc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fb278009cc0 tx=0x7fb2780091d0).ready entity=mon.2 client_cookie=990c9af830f45dde server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb285381700 1 -- 172.21.15.185:0/2295019114 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800bbc0 con 0x7fb280143830 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb285381700 1 -- 172.21.15.185:0/2295019114 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27800bd20 con 0x7fb280143830 2020-11-16T22:22:36.099 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.106+0000 7fb285381700 1 -- 172.21.15.185:0/2295019114 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278011610 con 0x7fb280143830 2020-11-16T22:22:36.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- 172.21.15.185:0/2295019114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 msgr2=0x7fb280143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 --2- 172.21.15.185:0/2295019114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 0x7fb280143cc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fb278009cc0 tx=0x7fb2780091d0).stop 2020-11-16T22:22:36.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- 172.21.15.185:0/2295019114 shutdown_connections 2020-11-16T22:22:36.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 --2- 172.21.15.185:0/2295019114 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb280143830 0x7fb280143cc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.302 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- 172.21.15.185:0/2295019114 >> 172.21.15.185:0/2295019114 conn(0x7fb280060130 msgr2=0x7fb280060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:36.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- 172.21.15.185:0/2295019114 shutdown_connections 2020-11-16T22:22:36.303 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- 172.21.15.185:0/2295019114 wait complete. 2020-11-16T22:22:36.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 Processor -- start 2020-11-16T22:22:36.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.110+0000 7fb2885e7700 1 -- start start 2020-11-16T22:22:36.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280062c30 0x7fb28013fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b39b0 0x7fb28013ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.306 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28013d420 con 0x7fb280062c30 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800aca40 con 0x7fb280143830 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800ac440 con 0x7fb2800b39b0 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35688/0 (socket says 172.21.15.185:35688) 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 -- 172.21.15.185:0/3315924697 learned_addr learned my addr 172.21.15.185:0/3315924697 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:36.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb285b82700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b39b0 0x7fb28013ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286383700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280062c30 0x7fb28013fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 -- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b39b0 msgr2=0x7fb28013ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b39b0 0x7fb28013ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 -- 172.21.15.185:0/3315924697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280062c30 msgr2=0x7fb28013fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280062c30 0x7fb28013fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 -- 172.21.15.185:0/3315924697 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800b22e0 con 0x7fb280143830 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb286b84700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fb278009210 tx=0x7fb278016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:36.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278006ea0 con 0x7fb280143830 2020-11-16T22:22:36.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278011e30 con 0x7fb280143830 2020-11-16T22:22:36.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278010c30 con 0x7fb280143830 2020-11-16T22:22:36.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2800b2580 con 0x7fb280143830 2020-11-16T22:22:36.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.114+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb280068540 con 0x7fb280143830 2020-11-16T22:22:36.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.118+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28013ffa0 con 0x7fb280143830 2020-11-16T22:22:36.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.122+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb278017030 con 0x7fb280143830 2020-11-16T22:22:36.510 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.126+0000 7fb26e7fc700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 0x7fb258057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.126+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb27806fb90 con 0x7fb280143830 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.126+0000 7fb286383700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 0x7fb258057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.126+0000 7fb286383700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 0x7fb258057520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fb27c003300 tx=0x7fb27c000b80).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.126+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb278045e70 con 0x7fb280143830 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb28005d4e0 con 0x7fb280143830 2020-11-16T22:22:36.511 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb26e7fc700 1 -- 172.21.15.185:0/3315924697 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb278023070 con 0x7fb280143830 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 msgr2=0x7fb258057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 0x7fb258057520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fb27c003300 tx=0x7fb27c000b80).stop 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 msgr2=0x7fb28013cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fb278009210 tx=0x7fb278016000).stop 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 shutdown_connections 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb258055060 0x7fb258057520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb280143830 0x7fb28013cf30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb280062c30 0x7fb28013fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 --2- 172.21.15.185:0/3315924697 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2800b39b0 0x7fb28013ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 >> 172.21.15.185:0/3315924697 conn(0x7fb280060130 msgr2=0x7fb280146ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 shutdown_connections 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.278+0000 7fb2885e7700 1 -- 172.21.15.185:0/3315924697 wait complete. 2020-11-16T22:22:36.513 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 1 2020-11-16T22:22:36.514 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T22:22:36.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 Processor -- start 2020-11-16T22:22:36.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 -- start start 2020-11-16T22:22:36.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05e813de10 0x7f05e80666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80b7c20 con 0x7f05e813e1f0 2020-11-16T22:22:36.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80b3a80 con 0x7f05e813de10 2020-11-16T22:22:36.649 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b3d20 con 0x7f05e80b3570 2020-11-16T22:22:36.650 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57376/0 (socket says 172.21.15.185:57376) 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f05e813de10 0x7f05e80666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e77fe700 1 -- 172.21.15.185:0/1135309170 learned_addr learned my addr 172.21.15.185:0/1135309170 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 -- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 msgr2=0x7f05e8143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.651 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 -- 172.21.15.185:0/1135309170 >> v1:172.21.15.185:6789/0 conn(0x7f05e813e1f0 legacy=0x7f05e80b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e5ffb700 1 -- 172.21.15.185:0/1135309170 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968383468 0 0) 0x7f05e80b7c20 con 0x7f05e813e1f0 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 -- 172.21.15.185:0/1135309170 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80b3fc0 con 0x7f05e813de10 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e77fe700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e6ffd700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e813de10 0x7f05e80666b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f05d8009c90 tx=0x7f05d80091d0).ready entity=mon.1 client_cookie=88221eb3cf951191 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.658+0000 7f05e5ffb700 1 -- 172.21.15.185:0/1135309170 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800ba70 con 0x7f05e813de10 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e5ffb700 1 -- 172.21.15.185:0/1135309170 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800bbd0 con 0x7f05e813de10 2020-11-16T22:22:36.652 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e5ffb700 1 -- 172.21.15.185:0/1135309170 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8011400 con 0x7f05e813de10 2020-11-16T22:22:36.653 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e813de10 msgr2=0x7f05e80666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e813de10 0x7f05e80666b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f05d8009c90 tx=0x7f05d80091d0).stop 2020-11-16T22:22:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- 172.21.15.185:0/1135309170 shutdown_connections 2020-11-16T22:22:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e80b3570 0x7f05e8143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 --2- 172.21.15.185:0/1135309170 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e813de10 0x7f05e80666b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.854 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- 172.21.15.185:0/1135309170 >> 172.21.15.185:0/1135309170 conn(0x7f05e8064b90 msgr2=0x7f05e8063760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- 172.21.15.185:0/1135309170 shutdown_connections 2020-11-16T22:22:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- 172.21.15.185:0/1135309170 wait complete. 2020-11-16T22:22:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 Processor -- start 2020-11-16T22:22:36.855 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.662+0000 7f05e7fff700 1 -- start start 2020-11-16T22:22:36.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e80b3570 0x7f05e80b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 0x7f05e8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80b14d0 con 0x7f05e80b3570 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81456a0 con 0x7f05e81413b0 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8157470 con 0x7f05e813de10 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 0x7f05e8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e67fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57380/0 (socket says 172.21.15.185:57380) 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e67fc700 1 -- 172.21.15.185:0/3029668099 learned_addr learned my addr 172.21.15.185:0/3029668099 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e6ffd700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e80b3570 0x7f05e80b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 -- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 msgr2=0x7f05e813efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 -- 172.21.15.185:0/3029668099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e80b3570 msgr2=0x7f05e80b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:36.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e80b3570 0x7f05e80b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 -- 172.21.15.185:0/3029668099 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e8157710 con 0x7f05e81413b0 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e67fc700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e77fe700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 0x7f05e8156bb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f05e0006fd0 tx=0x7f05e0006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000c8e0 con 0x7f05e81413b0 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000ca40 con 0x7f05e81413b0 2020-11-16T22:22:36.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e8145390 con 0x7f05e81413b0 2020-11-16T22:22:37.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000cdc0 con 0x7f05e81413b0 2020-11-16T22:22:37.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e814a080 con 0x7f05e81413b0 2020-11-16T22:22:37.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e81578a0 con 0x7f05e81413b0 2020-11-16T22:22:37.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.666+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f05e0022070 con 0x7f05e81413b0 2020-11-16T22:22:37.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.670+0000 7f05ceffd700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 0x7f05c4057550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.670+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05e006e220 con 0x7f05e81413b0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.670+0000 7f05e6ffd700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 0x7f05c4057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.670+0000 7f05e6ffd700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 0x7f05c4057550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f05d800b770 tx=0x7f05d8016000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.674+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f05e0044480 con 0x7f05e81413b0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05e8145870 con 0x7f05e81413b0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05ceffd700 1 -- 172.21.15.185:0/3029668099 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05e0016020 con 0x7f05e81413b0 2020-11-16T22:22:37.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 msgr2=0x7f05c4057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 0x7f05c4057550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f05d800b770 tx=0x7f05d8016000).stop 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 msgr2=0x7f05e8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 0x7f05e8156bb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f05e0006fd0 tx=0x7f05e0006eb0).stop 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 shutdown_connections 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f05c4055090 0x7f05c4057550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f05e81413b0 0x7f05e8156bb0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e80b3570 0x7f05e80b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 --2- 172.21.15.185:0/3029668099 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f05e813de10 0x7f05e813efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 >> 172.21.15.185:0/3029668099 conn(0x7f05e8064b90 msgr2=0x7f05e813d310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:37.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 shutdown_connections 2020-11-16T22:22:37.065 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:36.830+0000 7f05e7fff700 1 -- 172.21.15.185:0/3029668099 wait complete. 2020-11-16T22:22:37.065 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:37.066 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-16T22:22:37.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.210+0000 7fb706b2a700 1 Processor -- start 2020-11-16T22:22:37.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.210+0000 7fb706b2a700 1 -- start start 2020-11-16T22:22:37.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb7000b2e80 0x7fb7000b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb7000b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7000ad200 con 0x7fb7000b39d0 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000ae100 con 0x7fb7000b2e80 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000ac000 con 0x7fb70013c010 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb705327700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb7000b39d0 0x7fb70013b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57380/0 (socket says 172.21.15.185:57380) 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb705327700 1 -- 172.21.15.185:0/364469136 learned_addr learned my addr 172.21.15.185:0/364469136 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb7000b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb705b28700 1 --2- 172.21.15.185:0/364469136 >> v2:172.21.15.96:3300/0 conn(0x7fb7000b2e80 0x7fb7000b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb704b26700 1 -- 172.21.15.185:0/364469136 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686334025 0 0) 0x7fb7000ad200 con 0x7fb7000b39d0 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb704b26700 1 -- 172.21.15.185:0/364469136 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e8003240 con 0x7fb7000b39d0 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 -- 172.21.15.185:0/364469136 >> v2:172.21.15.96:3300/0 conn(0x7fb7000b2e80 msgr2=0x7fb7000b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 --2- 172.21.15.185:0/364469136 >> v2:172.21.15.96:3300/0 conn(0x7fb7000b2e80 0x7fb7000b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 -- 172.21.15.185:0/364469136 >> v1:172.21.15.185:6789/0 conn(0x7fb7000b39d0 legacy=0x7fb70013b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 -- 172.21.15.185:0/364469136 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000ac2a0 con 0x7fb70013c010 2020-11-16T22:22:37.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706329700 1 --2- 172.21.15.185:0/364469136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb7000b1170 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fb6f8009c90 tx=0x7fb6f80091d0).ready entity=mon.2 client_cookie=d4ff0e52161184ac server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb704b26700 1 -- 172.21.15.185:0/364469136 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800ba70 con 0x7fb70013c010 2020-11-16T22:22:37.207 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb704b26700 1 -- 172.21.15.185:0/364469136 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f800bbd0 con 0x7fb70013c010 2020-11-16T22:22:37.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb704b26700 1 -- 172.21.15.185:0/364469136 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8011400 con 0x7fb70013c010 2020-11-16T22:22:37.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- 172.21.15.185:0/364469136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 msgr2=0x7fb7000b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 --2- 172.21.15.185:0/364469136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb7000b1170 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fb6f8009c90 tx=0x7fb6f80091d0).stop 2020-11-16T22:22:37.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- 172.21.15.185:0/364469136 shutdown_connections 2020-11-16T22:22:37.410 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 --2- 172.21.15.185:0/364469136 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb7000b1170 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 --2- 172.21.15.185:0/364469136 >> v2:172.21.15.96:3300/0 conn(0x7fb7000b2e80 0x7fb7000b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- 172.21.15.185:0/364469136 >> 172.21.15.185:0/364469136 conn(0x7fb700060130 msgr2=0x7fb700060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- 172.21.15.185:0/364469136 shutdown_connections 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- 172.21.15.185:0/364469136 wait complete. 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 Processor -- start 2020-11-16T22:22:37.411 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.214+0000 7fb706b2a700 1 -- start start 2020-11-16T22:22:37.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7000b2e80 0x7fb700066730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 0x7fb70013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000b6cc0 con 0x7fb7000b2e80 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700156d40 con 0x7fb7000b39d0 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb700156fb0 con 0x7fb70013c010 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 0x7fb70013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706329700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706329700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57392/0 (socket says 172.21.15.185:57392) 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706329700 1 -- 172.21.15.185:0/3062772836 learned_addr learned my addr 172.21.15.185:0/3062772836 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:37.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705b28700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7000b2e80 0x7fb700066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 -- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 msgr2=0x7fb70013b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 -- 172.21.15.185:0/3062772836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7000b2e80 msgr2=0x7fb700066730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7000b2e80 0x7fb700066730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 -- 172.21.15.185:0/3062772836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000b6ef0 con 0x7fb7000b39d0 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706329700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb705327700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 0x7fb70013aef0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fb6f000c710 tx=0x7fb6f0007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.416 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00072e0 con 0x7fb7000b39d0 2020-11-16T22:22:37.417 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f0007440 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0005820 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb700066c20 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb700065a60 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.218+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70005d200 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.222+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb6f0010070 con 0x7fb7000b39d0 2020-11-16T22:22:37.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.222+0000 7fb6edffb700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 0x7fb6d8057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.222+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb6f006d370 con 0x7fb7000b39d0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.222+0000 7fb705b28700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 0x7fb6d8057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.222+0000 7fb705b28700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 0x7fb6d8057520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb6fc00d5f0 tx=0x7fb6fc008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.226+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb6f00435d0 con 0x7fb7000b39d0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.378+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7000acec0 con 0x7fb7000b39d0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.378+0000 7fb6edffb700 1 -- 172.21.15.185:0/3062772836 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb6f0046bf0 con 0x7fb7000b39d0 2020-11-16T22:22:37.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 msgr2=0x7fb6d8057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 0x7fb6d8057520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb6fc00d5f0 tx=0x7fb6fc008040).stop 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 msgr2=0x7fb70013aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 0x7fb70013aef0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fb6f000c710 tx=0x7fb6f0007a20).stop 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 shutdown_connections 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb6d8055060 0x7fb6d8057520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb70013c010 0x7fb70013b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7000b2e80 0x7fb700066730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 --2- 172.21.15.185:0/3062772836 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb7000b39d0 0x7fb70013aef0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 >> 172.21.15.185:0/3062772836 conn(0x7fb700060130 msgr2=0x7fb7000680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 shutdown_connections 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.382+0000 7fb706b2a700 1 -- 172.21.15.185:0/3062772836 wait complete. 2020-11-16T22:22:37.621 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:37.622 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-16T22:22:37.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 Processor -- start 2020-11-16T22:22:37.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 -- start start 2020-11-16T22:22:37.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7faddc05bc50 0x7faddc0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7faddc140cc0 0x7faddc141130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc0b2690 con 0x7faddc0b38a0 2020-11-16T22:22:37.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc0ad350 con 0x7faddc140cc0 2020-11-16T22:22:37.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade238b700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc0b2930 con 0x7faddc05bc50 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1389700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7faddc0b38a0 0x7faddc144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57392/0 (socket says 172.21.15.185:57392) 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1389700 1 -- 172.21.15.185:0/2234594295 learned_addr learned my addr 172.21.15.185:0/2234594295 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 --2- 172.21.15.185:0/2234594295 >> v2:172.21.15.96:3300/0 conn(0x7faddc140cc0 0x7faddc141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade0b88700 1 --2- 172.21.15.185:0/2234594295 >> v2:172.21.15.185:3300/0 conn(0x7faddc05bc50 0x7faddc0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fadcbfff700 1 -- 172.21.15.185:0/2234594295 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724285662 0 0) 0x7faddc0b2690 con 0x7faddc0b38a0 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fadcbfff700 1 -- 172.21.15.185:0/2234594295 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc4003240 con 0x7faddc0b38a0 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 -- 172.21.15.185:0/2234594295 >> v2:172.21.15.185:3300/0 conn(0x7faddc05bc50 msgr2=0x7faddc0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:37.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 --2- 172.21.15.185:0/2234594295 >> v2:172.21.15.185:3300/0 conn(0x7faddc05bc50 0x7faddc0b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 -- 172.21.15.185:0/2234594295 >> v1:172.21.15.185:6789/0 conn(0x7faddc0b38a0 legacy=0x7faddc144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 -- 172.21.15.185:0/2234594295 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc1443d0 con 0x7faddc140cc0 2020-11-16T22:22:37.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.770+0000 7fade1b8a700 1 --2- 172.21.15.185:0/2234594295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc141130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fadd4009c90 tx=0x7fadd40091d0).ready entity=mon.0 client_cookie=b057fbcfff4f097 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fadcbfff700 1 -- 172.21.15.185:0/2234594295 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400b980 con 0x7faddc140cc0 2020-11-16T22:22:37.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fadcbfff700 1 -- 172.21.15.185:0/2234594295 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400bae0 con 0x7faddc140cc0 2020-11-16T22:22:37.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fadcbfff700 1 -- 172.21.15.185:0/2234594295 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400be60 con 0x7faddc140cc0 2020-11-16T22:22:37.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- 172.21.15.185:0/2234594295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 msgr2=0x7faddc141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.966 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- 172.21.15.185:0/2234594295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc141130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fadd4009c90 tx=0x7fadd40091d0).stop 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- 172.21.15.185:0/2234594295 shutdown_connections 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- 172.21.15.185:0/2234594295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc141130 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- 172.21.15.185:0/2234594295 >> v2:172.21.15.185:3300/0 conn(0x7faddc05bc50 0x7faddc0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- 172.21.15.185:0/2234594295 >> 172.21.15.185:0/2234594295 conn(0x7faddc063df0 msgr2=0x7faddc0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- 172.21.15.185:0/2234594295 shutdown_connections 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- 172.21.15.185:0/2234594295 wait complete. 2020-11-16T22:22:37.967 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 Processor -- start 2020-11-16T22:22:37.968 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- start start 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faddc05bc50 0x7faddc139d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc142250 con 0x7faddc140cc0 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc152dd0 con 0x7faddc0b38a0 2020-11-16T22:22:37.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade238b700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc153070 con 0x7faddc05bc50 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35728/0 (socket says 172.21.15.185:35728) 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade1389700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faddc05bc50 0x7faddc139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 -- 172.21.15.185:0/3620318552 learned_addr learned my addr 172.21.15.185:0/3620318552 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade1b8a700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 -- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faddc05bc50 msgr2=0x7faddc139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faddc05bc50 0x7faddc139d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 -- 172.21.15.185:0/3620318552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 msgr2=0x7faddc142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 -- 172.21.15.185:0/3620318552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc142440 con 0x7faddc0b38a0 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fade0b88700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fadcc00eba0 tx=0x7fadcc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00c9b0 con 0x7faddc0b38a0 2020-11-16T22:22:37.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.774+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc00cb10 con 0x7faddc0b38a0 2020-11-16T22:22:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc13a200 con 0x7faddc0b38a0 2020-11-16T22:22:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00ce90 con 0x7faddc0b38a0 2020-11-16T22:22:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc13ce30 con 0x7faddc0b38a0 2020-11-16T22:22:38.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc05d200 con 0x7faddc0b38a0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fadcc012600 con 0x7faddc0b38a0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fadc97fa700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 0x7fadb4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadcc06e250 con 0x7faddc0b38a0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fade1389700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 0x7fadb4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.778+0000 7fade1389700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 0x7fadb4057520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fadd800bfd0 tx=0x7fadd800d040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.786+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fadcc0444b0 con 0x7faddc0b38a0 2020-11-16T22:22:38.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.938+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faddc061e20 con 0x7faddc0b38a0 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fadc97fa700 1 -- 172.21.15.185:0/3620318552 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7fadcc047ad0 con 0x7faddc0b38a0 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 msgr2=0x7fadb4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 0x7fadb4057520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fadd800bfd0 tx=0x7fadd800d040).stop 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 msgr2=0x7faddc1428e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7fadcc00eba0 tx=0x7fadcc006df0).stop 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 shutdown_connections 2020-11-16T22:22:38.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fadb4055060 0x7fadb4057520 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc140cc0 0x7faddc142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faddc05bc50 0x7faddc139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 --2- 172.21.15.185:0/3620318552 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faddc0b38a0 0x7faddc1428e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 >> 172.21.15.185:0/3620318552 conn(0x7faddc063df0 msgr2=0x7faddc066c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 shutdown_connections 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:37.942+0000 7fade238b700 1 -- 172.21.15.185:0/3620318552 wait complete. 2020-11-16T22:22:38.177 INFO:teuthology.orchestra.run.smithi185.stdout:pg_num: 8 2020-11-16T22:22:38.178 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-16T22:22:38.178 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 Processor -- start 2020-11-16T22:22:38.308 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 -- start start 2020-11-16T22:22:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 0x7f72b40b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.309 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b41484c0 con 0x7f72b40b0300 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b4144ff0 con 0x7f72b4063020 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72bbcee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41482a0 con 0x7f72b4144ba0 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72b9289700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 0x7f72b40b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57416/0 (socket says 172.21.15.185:57416) 2020-11-16T22:22:38.310 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72b9289700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 0x7f72b40b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35738/0 (socket says 172.21.15.185:35738) 2020-11-16T22:22:38.311 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 -- 172.21.15.185:0/759005246 learned_addr learned my addr 172.21.15.185:0/759005246 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 -- 172.21.15.185:0/759005246 >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 msgr2=0x7f72b40b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 --2- 172.21.15.185:0/759005246 >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 0x7f72b40b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72b8a88700 1 -- 172.21.15.185:0/759005246 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044109757 0 0) 0x7f72b41484c0 con 0x7f72b40b0300 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 -- 172.21.15.185:0/759005246 >> v1:172.21.15.96:6789/0 conn(0x7f72b40b0300 legacy=0x7f72b413dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 -- 172.21.15.185:0/759005246 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b41441c0 con 0x7f72b4144ba0 2020-11-16T22:22:38.312 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72ba28b700 1 --2- 172.21.15.185:0/759005246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f72ac009d50 tx=0x7f72ac009290).ready entity=mon.2 client_cookie=42122aa36513067c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:38.313 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.318+0000 7f72b8a88700 1 -- 172.21.15.185:0/759005246 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00bc50 con 0x7f72b4144ba0 2020-11-16T22:22:38.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b8a88700 1 -- 172.21.15.185:0/759005246 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00bdb0 con 0x7f72b4144ba0 2020-11-16T22:22:38.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b8a88700 1 -- 172.21.15.185:0/759005246 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac005cb0 con 0x7f72b4144ba0 2020-11-16T22:22:38.514 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- 172.21.15.185:0/759005246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 msgr2=0x7f72b4147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- 172.21.15.185:0/759005246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f72ac009d50 tx=0x7f72ac009290).stop 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- 172.21.15.185:0/759005246 shutdown_connections 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- 172.21.15.185:0/759005246 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4144ba0 0x7f72b4147db0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- 172.21.15.185:0/759005246 >> v2:172.21.15.185:3300/0 conn(0x7f72b4063020 0x7f72b40b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- 172.21.15.185:0/759005246 >> 172.21.15.185:0/759005246 conn(0x7f72b40b4410 msgr2=0x7f72b4060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- 172.21.15.185:0/759005246 shutdown_connections 2020-11-16T22:22:38.515 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- 172.21.15.185:0/759005246 wait complete. 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 Processor -- start 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- start start 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4063020 0x7f72b414aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72b4144ba0 0x7f72b4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.518 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b408ea20 con 0x7f72b4144ba0 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b408ec50 con 0x7f72b40b0300 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72bbcee700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b408eef0 con 0x7f72b4063020 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b9289700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b9289700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35744/0 (socket says 172.21.15.185:35744) 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b9289700 1 -- 172.21.15.185:0/780381053 learned_addr learned my addr 172.21.15.185:0/780381053 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72b9a8a700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4063020 0x7f72b414aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.519 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.322+0000 7f72ba28b700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72b4144ba0 0x7f72b4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 -- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4063020 msgr2=0x7f72b414aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4063020 0x7f72b414aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 -- 172.21.15.185:0/780381053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72b4144ba0 msgr2=0x7f72b4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72b4144ba0 0x7f72b4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 -- 172.21.15.185:0/780381053 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4156ba0 con 0x7f72b40b0300 2020-11-16T22:22:38.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9289700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f72a800eba0 tx=0x7f72a8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800c8e0 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a800ca40 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b414b550 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800cdc0 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b4143ce0 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b405d4e0 con 0x7f72b40b0300 2020-11-16T22:22:38.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f72a8012600 con 0x7f72b40b0300 2020-11-16T22:22:38.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 0x7f728c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:38.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72a806e080 con 0x7f72b40b0300 2020-11-16T22:22:38.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9a8a700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 0x7f728c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:38.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.326+0000 7f72b9a8a700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 0x7f728c057520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f72a40011a0 tx=0x7f72a4007970).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:38.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.334+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f72a8044280 con 0x7f72b40b0300 2020-11-16T22:22:38.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.538+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72b4061840 con 0x7f72b40b0300 2020-11-16T22:22:38.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.538+0000 7f72b1ffb700 1 -- 172.21.15.185:0/780381053 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+2491 (secure 0 0 0) 0x7f72a80478a0 con 0x7f72b40b0300 2020-11-16T22:22:38.528 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:38.528 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":101,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":97},{"gid":7830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:33.095134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:38.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 msgr2=0x7f728c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 0x7f728c057520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f72a40011a0 tx=0x7f72a4007970).stop 2020-11-16T22:22:38.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 msgr2=0x7f72b4157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:38.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f72a800eba0 tx=0x7f72a8006df0).stop 2020-11-16T22:22:38.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 shutdown_connections 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f728c055060 0x7f728c057520 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72b4144ba0 0x7f72b4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f72b4063020 0x7f72b414aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 --2- 172.21.15.185:0/780381053 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f72b40b0300 0x7f72b4157040 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 >> 172.21.15.185:0/780381053 conn(0x7f72b40b4410 msgr2=0x7f72b40643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 shutdown_connections 2020-11-16T22:22:38.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:38.542+0000 7f72bbcee700 1 -- 172.21.15.185:0/780381053 wait complete. 2020-11-16T22:22:38.533 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 101 2020-11-16T22:22:38.849 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:22:29.743585+0000', 'modified': '2020-11-16T22:22:33.095134+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7993': {'gid': 7993, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.185:6835/1118138229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1118138229}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1118138229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:22:38.850 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-16T22:22:38.851 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:22:38.992 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7993,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.2463647780000002,"mdsmap_epoch":101,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":14.49323388} 2020-11-16T22:22:38.993 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7993, "mdsmap_epoch": 101, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 9.246364778, "state": "up:active", "uptime": 14.49323388, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:22:38.993 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-16T22:22:38.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:38.994 INFO:teuthology.orchestra.run.smithi062:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-16T22:22:39.011 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-16T22:22:39.012 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:22:39.088 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:22:39.088 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:22:39.115 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-16T22:22:39.116 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-16T22:22:39.123 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-16T22:22:39.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-16T22:22:39.124 INFO:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-16T22:22:39.152 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2020-11-16T22:22:39.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 Processor -- start 2020-11-16T22:22:39.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 -- start start 2020-11-16T22:22:39.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6894060130 0x7f689413fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6894060970 con 0x7f68940b3830 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68940b4610 con 0x7f6894060130 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6899522700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6894060ba0 con 0x7f68940ac820 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57428/0 (socket says 172.21.15.185:57428) 2020-11-16T22:22:39.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 -- 172.21.15.185:0/4194752317 learned_addr learned my addr 172.21.15.185:0/4194752317 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:39.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f6898d21700 1 --2- 172.21.15.185:0/4194752317 >> v2:172.21.15.96:3300/0 conn(0x7f6894060130 0x7f689413fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 -- 172.21.15.185:0/4194752317 >> v2:172.21.15.96:3300/0 conn(0x7f6894060130 msgr2=0x7f689413fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:39.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 --2- 172.21.15.185:0/4194752317 >> v2:172.21.15.96:3300/0 conn(0x7f6894060130 0x7f689413fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:39.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 -- 172.21.15.185:0/4194752317 >> v1:172.21.15.96:6789/0 conn(0x7f68940b3830 legacy=0x7f6894143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:39.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 -- 172.21.15.185:0/4194752317 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894060e40 con 0x7f68940ac820 2020-11-16T22:22:39.292 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.298+0000 7f68937fe700 1 --2- 172.21.15.185:0/4194752317 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f6884006fd0 tx=0x7f6884006eb0).ready entity=mon.2 client_cookie=877df2411d61a050 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6892ffd700 1 -- 172.21.15.185:0/4194752317 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884009810 con 0x7f68940ac820 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6892ffd700 1 -- 172.21.15.185:0/4194752317 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884009970 con 0x7f68940ac820 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6892ffd700 1 -- 172.21.15.185:0/4194752317 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884009cf0 con 0x7f68940ac820 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- 172.21.15.185:0/4194752317 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 msgr2=0x7f689413f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- 172.21.15.185:0/4194752317 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f6884006fd0 tx=0x7f6884006eb0).stop 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- 172.21.15.185:0/4194752317 shutdown_connections 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- 172.21.15.185:0/4194752317 >> v2:172.21.15.96:3300/0 conn(0x7f6894060130 0x7f689413fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:39.293 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- 172.21.15.185:0/4194752317 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940ac820 0x7f689413f840 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:39.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- 172.21.15.185:0/4194752317 >> 172.21.15.185:0/4194752317 conn(0x7f6894063df0 msgr2=0x7f68940641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:39.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- 172.21.15.185:0/4194752317 shutdown_connections 2020-11-16T22:22:39.294 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- 172.21.15.185:0/4194752317 wait complete. 2020-11-16T22:22:39.295 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 Processor -- start 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- start start 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6894060130 0x7f689413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940b3830 0x7f6894112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941523e0 con 0x7f6894060130 2020-11-16T22:22:39.296 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940b1630 con 0x7f68940ac820 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6899522700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940b18d0 con 0x7f68940b3830 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35758/0 (socket says 172.21.15.185:35758) 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 -- 172.21.15.185:0/1987414527 learned_addr learned my addr 172.21.15.185:0/1987414527 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6898d21700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940b3830 0x7f6894112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f6893fff700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6894060130 0x7f689413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.297 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 -- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940b3830 msgr2=0x7f6894112eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940b3830 0x7f6894112eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 -- 172.21.15.185:0/1987414527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6894060130 msgr2=0x7f689413b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6894060130 0x7f689413b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.302+0000 7f68937fe700 1 -- 172.21.15.185:0/1987414527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940b1b70 con 0x7f68940ac820 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f68937fe700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f6884010000 tx=0x7f6884015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884011c80 con 0x7f68940ac820 2020-11-16T22:22:39.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884011de0 con 0x7f68940ac820 2020-11-16T22:22:39.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68940b05a0 con 0x7f68940ac820 2020-11-16T22:22:39.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688401e670 con 0x7f68940ac820 2020-11-16T22:22:39.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6894152120 con 0x7f68940ac820 2020-11-16T22:22:39.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689405d200 con 0x7f68940ac820 2020-11-16T22:22:39.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6884025030 con 0x7f68940ac820 2020-11-16T22:22:39.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 0x7f6870057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:39.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f688406f380 con 0x7f68940ac820 2020-11-16T22:22:39.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f6893fff700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 0x7f6870057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:39.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.306+0000 7f6893fff700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 0x7f6870057520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f68940b5090 tx=0x7f687c009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:39.307 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.318+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f6884045690 con 0x7f68940ac820 2020-11-16T22:22:39.512 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:39.522+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f6894061e20 con 0x7f68940ac820 2020-11-16T22:22:40.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.150+0000 7f688bfff700 1 -- 172.21.15.185:0/1987414527 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v102) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6884023020 con 0x7f68940ac820 2020-11-16T22:22:40.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 msgr2=0x7f6870057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 0x7f6870057520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f68940b5090 tx=0x7f687c009040).stop 2020-11-16T22:22:40.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 msgr2=0x7f689413b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.144 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f6884010000 tx=0x7f6884015000).stop 2020-11-16T22:22:40.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 shutdown_connections 2020-11-16T22:22:40.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6870055060 0x7f6870057520 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.145 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f68940b3830 0x7f6894112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6894060130 0x7f689413b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 --2- 172.21.15.185:0/1987414527 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f68940ac820 0x7f689413b540 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 >> 172.21.15.185:0/1987414527 conn(0x7f6894063df0 msgr2=0x7f68940b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:40.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 shutdown_connections 2020-11-16T22:22:40.146 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.154+0000 7f6899522700 1 -- 172.21.15.185:0/1987414527 wait complete. 2020-11-16T22:22:40.167 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:40.281 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.290+0000 7fdf5237f700 1 Processor -- start 2020-11-16T22:22:40.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.290+0000 7fdf5237f700 1 -- start start 2020-11-16T22:22:40.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdf4c0b3090 0x7fdf4c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdf4c0b1d30 0x7fdf4c14b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b2b60 con 0x7fdf4c0b3090 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c061fe0 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b0fe0 con 0x7fdf4c0ad110 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdf4c0b1d30 0x7fdf4c14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf51b7e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fdf4c0b1d30 0x7fdf4c14b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35766/0 (socket says 172.21.15.185:35766) 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 -- 172.21.15.185:0/1053030550 learned_addr learned my addr 172.21.15.185:0/1053030550 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5137d700 1 --2- 172.21.15.185:0/1053030550 >> v2:172.21.15.96:3300/0 conn(0x7fdf4c0b3090 0x7fdf4c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 -- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 msgr2=0x7fdf4c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c1477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 -- 172.21.15.185:0/1053030550 >> v2:172.21.15.96:3300/0 conn(0x7fdf4c0b3090 msgr2=0x7fdf4c0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/1053030550 >> v2:172.21.15.96:3300/0 conn(0x7fdf4c0b3090 0x7fdf4c0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.285 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 -- 172.21.15.185:0/1053030550 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0b1280 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.286 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c14b950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fdf4c066720 tx=0x7fdf48009290).ready entity=mon.1 client_cookie=b5a9c67563a445ff server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf3ffff700 1 -- 172.21.15.185:0/1053030550 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800bb10 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.287 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf3ffff700 1 -- 172.21.15.185:0/1053030550 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800bc70 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf3ffff700 1 -- 172.21.15.185:0/1053030550 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011400 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 msgr2=0x7fdf4c14b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c14b950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fdf4c066720 tx=0x7fdf48009290).stop 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- 172.21.15.185:0/1053030550 shutdown_connections 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.288 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- 172.21.15.185:0/1053030550 >> v2:172.21.15.96:3300/0 conn(0x7fdf4c0b3090 0x7fdf4c0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 --2- 172.21.15.185:0/1053030550 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c14b950 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- 172.21.15.185:0/1053030550 >> 172.21.15.185:0/1053030550 conn(0x7fdf4c060130 msgr2=0x7fdf4c060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- 172.21.15.185:0/1053030550 shutdown_connections 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- 172.21.15.185:0/1053030550 wait complete. 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 Processor -- start 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.294+0000 7fdf5237f700 1 -- start start 2020-11-16T22:22:40.289 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c10f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf4c0b3090 0x7fdf4c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b0750 con 0x7fdf4c0b3090 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c111270 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c111510 con 0x7fdf4c0ad110 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.290 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5137d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c10f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35772/0 (socket says 172.21.15.185:35772) 2020-11-16T22:22:40.291 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 -- 172.21.15.185:0/2661187504 learned_addr learned my addr 172.21.15.185:0/2661187504 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf51b7e700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf4c0b3090 0x7fdf4c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 -- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 msgr2=0x7fdf4c10f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c10f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 -- 172.21.15.185:0/2661187504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf4c0b3090 msgr2=0x7fdf4c0b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.494 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf4c0b3090 0x7fdf4c0b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.495 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 -- 172.21.15.185:0/2661187504 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c1117b0 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf50b7c700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fdf480092d0 tx=0x7fdf48016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:40.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011e70 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.496 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4801c680 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c111c50 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4801ca20 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c15afb0 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.298+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c05d200 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.497 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.302+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdf48017070 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.302+0000 7fdf3d7fa700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 0x7fdf24057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:40.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.302+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf4806fc90 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.302+0000 7fdf5137d700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 0x7fdf24057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:40.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.302+0000 7fdf5137d700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 0x7fdf24057520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fdf44006fd0 tx=0x7fdf44006df0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:40.698 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.306+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fdf48045ed0 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf4c0b2ca0 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.699 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf3d7fa700 1 -- 172.21.15.185:0/2661187504 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+2508 (secure 0 0 0) 0x7fdf48015070 con 0x7fdf4c0b1d30 2020-11-16T22:22:40.699 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:40.699 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":103,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:40.150413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7993,"mds_1":7830},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:creating","state_seq":1,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:40.701 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:40.701 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 msgr2=0x7fdf24057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 0x7fdf24057520 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fdf44006fd0 tx=0x7fdf44006df0).stop 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 msgr2=0x7fdf4c0afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fdf480092d0 tx=0x7fdf48016000).stop 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 shutdown_connections 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdf24055060 0x7fdf24057520 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf4c0b3090 0x7fdf4c0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.702 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fdf4c0ad110 0x7fdf4c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 --2- 172.21.15.185:0/2661187504 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fdf4c0b1d30 0x7fdf4c0afcb0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:40.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 >> 172.21.15.185:0/2661187504 conn(0x7fdf4c060130 msgr2=0x7fdf4c14c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:40.703 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 shutdown_connections 2020-11-16T22:22:40.906 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:40.506+0000 7fdf5237f700 1 -- 172.21.15.185:0/2661187504 wait complete. 2020-11-16T22:22:40.906 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 103 2020-11-16T22:22:40.912 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:40.914 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:22:41.209 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:22:29.743585+0000', 'modified': '2020-11-16T22:22:40.150413+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7993, 'mds_1': 7830}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7830': {'gid': 7830, 'name': 'c', 'rank': 1, 'incarnation': 103, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.96:6835/477883541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 477883541}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 477883541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7993': {'gid': 7993, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.185:6835/1118138229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1118138229}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1118138229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:22:41.209 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7830:up:creating 2020-11-16T22:22:42.210 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:42.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 Processor -- start 2020-11-16T22:22:42.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 -- start start 2020-11-16T22:22:42.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ba80b72e0 0x7f4ba80b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4ba80b64b0 0x7f4ba80b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.325 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba8144870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80af020 con 0x7f4ba80b64b0 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba8147f90 con 0x7f4ba80b72e0 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4bace64700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8148460 con 0x7f4ba80acee0 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ba80b72e0 0x7f4ba80b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba8144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.326 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4ba80b72e0 0x7f4ba80b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35780/0 (socket says 172.21.15.185:35780) 2020-11-16T22:22:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 -- 172.21.15.185:0/3556346371 learned_addr learned my addr 172.21.15.185:0/3556346371 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:42.327 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba6ffd700 1 --2- 172.21.15.185:0/3556346371 >> v2:172.21.15.96:3300/0 conn(0x7f4ba80b64b0 0x7f4ba80b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 -- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 msgr2=0x7f4ba8144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 --2- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba8144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 -- 172.21.15.185:0/3556346371 >> v2:172.21.15.96:3300/0 conn(0x7f4ba80b64b0 msgr2=0x7f4ba80b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:22:42.328 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 --2- 172.21.15.185:0/3556346371 >> v2:172.21.15.96:3300/0 conn(0x7f4ba80b64b0 0x7f4ba80b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 -- 172.21.15.185:0/3556346371 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80b4810 con 0x7f4ba80b72e0 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba77fe700 1 --2- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba80b76c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f4ba8080a30 tx=0x7f4ba0006e50).ready entity=mon.1 client_cookie=6abd8262d481cdb4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba67fc700 1 -- 172.21.15.185:0/3556346371 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0006030 con 0x7f4ba80b72e0 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba67fc700 1 -- 172.21.15.185:0/3556346371 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba0006190 con 0x7f4ba80b72e0 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.334+0000 7f4ba67fc700 1 -- 172.21.15.185:0/3556346371 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0006510 con 0x7f4ba80b72e0 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 msgr2=0x7f4ba80b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:42.329 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba80b76c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f4ba8080a30 tx=0x7f4ba0006e50).stop 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- 172.21.15.185:0/3556346371 shutdown_connections 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba8144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- 172.21.15.185:0/3556346371 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba80b76c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- 172.21.15.185:0/3556346371 >> v2:172.21.15.96:3300/0 conn(0x7f4ba80b64b0 0x7f4ba80b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- 172.21.15.185:0/3556346371 >> 172.21.15.185:0/3556346371 conn(0x7f4ba8067860 msgr2=0x7f4ba8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- 172.21.15.185:0/3556346371 shutdown_connections 2020-11-16T22:22:42.330 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- 172.21.15.185:0/3556346371 wait complete. 2020-11-16T22:22:42.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 Processor -- start 2020-11-16T22:22:42.332 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- start start 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba81575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 0x7f4ba80b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81409d0 con 0x7f4ba80b64b0 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8156c10 con 0x7f4ba80b72e0 2020-11-16T22:22:42.333 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4bace64700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8156eb0 con 0x7f4ba80acee0 2020-11-16T22:22:42.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba77fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba81575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35790/0 (socket says 172.21.15.185:35790) 2020-11-16T22:22:42.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 -- 172.21.15.185:0/397141954 learned_addr learned my addr 172.21.15.185:0/397141954 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:42.534 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba6ffd700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 0x7f4ba80b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 -- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 msgr2=0x7f4ba81575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba81575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 -- 172.21.15.185:0/397141954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 msgr2=0x7f4ba80b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 0x7f4ba80b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 -- 172.21.15.185:0/397141954 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8157150 con 0x7f4ba80b72e0 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba6ffd700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 0x7f4ba80b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:22:42.535 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.338+0000 7f4ba7fff700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f4b9800ecf0 tx=0x7f4b980091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:42.536 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98009520 con 0x7f4ba80b72e0 2020-11-16T22:22:42.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800ca00 con 0x7f4ba80b72e0 2020-11-16T22:22:42.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80b3cd0 con 0x7f4ba80b72e0 2020-11-16T22:22:42.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8143c60 con 0x7f4ba80b72e0 2020-11-16T22:22:42.738 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800cd80 con 0x7f4ba80b72e0 2020-11-16T22:22:42.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80ba660 con 0x7f4ba80b72e0 2020-11-16T22:22:42.739 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4b98022070 con 0x7f4ba80b72e0 2020-11-16T22:22:42.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 0x7f4b80057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:42.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4ba77fe700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 0x7f4b80057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:42.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b9806e450 con 0x7f4ba80b72e0 2020-11-16T22:22:42.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.342+0000 7f4ba77fe700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 0x7f4b80057520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f4ba0006e90 tx=0x7f4ba0015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:42.741 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.346+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4ba80ba660 con 0x7f4ba80b72e0 2020-11-16T22:22:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.546+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ba80ba660 con 0x7f4ba80b72e0 2020-11-16T22:22:42.742 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4b877fe700 1 -- 172.21.15.185:0/397141954 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+2522 (secure 0 0 0) 0x7f4ba80ba660 con 0x7f4ba80b72e0 2020-11-16T22:22:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 msgr2=0x7f4b80057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 0x7f4b80057520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f4ba0006e90 tx=0x7f4ba0015000).stop 2020-11-16T22:22:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 msgr2=0x7f4ba8153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:42.946 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f4b9800ecf0 tx=0x7f4b980091d0).stop 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 shutdown_connections 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4b80055060 0x7f4b80057520 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4ba80b72e0 0x7f4ba8153560 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4ba80acee0 0x7f4ba81575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 --2- 172.21.15.185:0/397141954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ba80b64b0 0x7f4ba80b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 >> 172.21.15.185:0/397141954 conn(0x7f4ba8067860 msgr2=0x7f4ba80ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:42.947 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 shutdown_connections 2020-11-16T22:22:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:42.550+0000 7f4bace64700 1 -- 172.21.15.185:0/397141954 wait complete. 2020-11-16T22:22:42.948 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 106 2020-11-16T22:22:42.948 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:42.948 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":106,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:42.170266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:43.452 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-16T22:22:29.743585+0000', 'modified': '2020-11-16T22:22:42.170266+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7993, 'mds_1': 7830, 'mds_2': 7821}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7821': {'gid': 7821, 'name': 'b', 'rank': 2, 'incarnation': 105, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.96:6833/2921285835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2921285835}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 2921285835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7830': {'gid': 7830, 'name': 'c', 'rank': 1, 'incarnation': 103, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6835/477883541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 477883541}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 477883541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}, 'gid_7993': {'gid': 7993, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.185:6835/1118138229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.185:6834', 'nonce': 1118138229}, {'type': 'v1', 'addr': '172.21.15.185:6835', 'nonce': 1118138229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-16T22:22:43.453 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2020-11-16T22:22:43.453 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-16T22:22:43.593 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":2,"id":7821,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":2.4278853730000001,"mdsmap_epoch":106,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":19.096005866999999} 2020-11-16T22:22:43.594 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7821, "mdsmap_epoch": 106, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 2.427885373, "state": "up:active", "uptime": 19.096005867, "want_state": "up:active", "whoami": 2 } 2020-11-16T22:22:43.594 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-16T22:22:43.721 INFO:teuthology.orchestra.run.smithi096.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":1,"id":7830,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":3.5671122949999998,"mdsmap_epoch":106,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":19.218048578000001} 2020-11-16T22:22:43.721 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7830, "mdsmap_epoch": 106, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 3.567112295, "state": "up:active", "uptime": 19.218048578, "want_state": "up:active", "whoami": 1 } 2020-11-16T22:22:43.722 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-16T22:22:43.861 INFO:teuthology.orchestra.run.smithi185.stdout:{"cluster_fsid":"7d23284a-d49b-4d10-9d1b-194105919b4f","whoami":0,"id":7993,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.115632445999999,"mdsmap_epoch":106,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":19.362511677000001} 2020-11-16T22:22:43.862 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "7d23284a-d49b-4d10-9d1b-194105919b4f", "fs_name": "cephfs", "id": 7993, "mdsmap_epoch": 106, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 14.115632446, "state": "up:active", "uptime": 19.362511677, "want_state": "up:active", "whoami": 0 } 2020-11-16T22:22:43.862 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:22:43.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 Processor -- start 2020-11-16T22:22:43.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 -- start start 2020-11-16T22:22:43.986 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb080669a0 0x7fcb08066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080abf90 con 0x7fcb080acc10 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080ac230 con 0x7fcb08061e60 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0f477700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb080ac4d0 con 0x7fcb080669a0 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0ec76700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fcb08061e60 0x7fcb081442f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57468/0 (socket says 172.21.15.185:57468) 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0ec76700 1 -- 172.21.15.185:0/3970392005 learned_addr learned my addr 172.21.15.185:0/3970392005 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:22:43.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.994+0000 7fcb0e475700 1 --2- 172.21.15.185:0/3970392005 >> v2:172.21.15.96:3300/0 conn(0x7fcb080669a0 0x7fcb08066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:43.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0d473700 1 -- 172.21.15.185:0/3970392005 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812919960 0 0) 0x7fcb080ac230 con 0x7fcb08061e60 2020-11-16T22:22:43.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0d473700 1 -- 172.21.15.185:0/3970392005 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf0003240 con 0x7fcb08061e60 2020-11-16T22:22:43.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0e475700 1 -- 172.21.15.185:0/3970392005 >> v1:172.21.15.185:6789/0 conn(0x7fcb08061e60 legacy=0x7fcb081442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:43.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0e475700 1 -- 172.21.15.185:0/3970392005 >> v1:172.21.15.96:6789/0 conn(0x7fcb080acc10 legacy=0x7fcb08147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:43.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0e475700 1 -- 172.21.15.185:0/3970392005 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080ac770 con 0x7fcb080669a0 2020-11-16T22:22:43.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0e475700 1 --2- 172.21.15.185:0/3970392005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb080669a0 0x7fcb08066d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fcb0400b9a0 tx=0x7fcb04006df0).ready entity=mon.0 client_cookie=ea86759c191095cb server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:43.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0d473700 1 -- 172.21.15.185:0/3970392005 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040097b0 con 0x7fcb080669a0 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0d473700 1 -- 172.21.15.185:0/3970392005 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04009910 con 0x7fcb080669a0 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0d473700 1 -- 172.21.15.185:0/3970392005 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04009c90 con 0x7fcb080669a0 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 -- 172.21.15.185:0/3970392005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb080669a0 msgr2=0x7fcb08066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 --2- 172.21.15.185:0/3970392005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb080669a0 0x7fcb08066d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fcb0400b9a0 tx=0x7fcb04006df0).stop 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 -- 172.21.15.185:0/3970392005 shutdown_connections 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 --2- 172.21.15.185:0/3970392005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb080669a0 0x7fcb08066d80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:43.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 -- 172.21.15.185:0/3970392005 >> 172.21.15.185:0/3970392005 conn(0x7fcb08060130 msgr2=0x7fcb08060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:43.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 -- 172.21.15.185:0/3970392005 shutdown_connections 2020-11-16T22:22:43.991 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:43.998+0000 7fcb0f477700 1 -- 172.21.15.185:0/3970392005 wait complete. 2020-11-16T22:22:43.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 Processor -- start 2020-11-16T22:22:43.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- start start 2020-11-16T22:22:43.992 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb08061e60 0x7fcb08140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081538b0 con 0x7fcb08061e60 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08145ab0 con 0x7fcb080669a0 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08145810 con 0x7fcb08145370 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0dc74700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:43.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0dc74700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35800/0 (socket says 172.21.15.185:35800) 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57480/0 (socket says 172.21.15.185:57480) 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0dc74700 1 -- 172.21.15.185:0/2338446108 learned_addr learned my addr 172.21.15.185:0/2338446108 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0e475700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb08061e60 0x7fcb08140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 -- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 msgr2=0x7fcb08153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:43.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:43.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 -- 172.21.15.185:0/2338446108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb08061e60 msgr2=0x7fcb08140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:43.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb08061e60 0x7fcb08140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:43.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 -- 172.21.15.185:0/2338446108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb08153b10 con 0x7fcb08145370 2020-11-16T22:22:43.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0dc74700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:22:43.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0ec76700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fcb0000ca90 tx=0x7fcb00007ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:43.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000ce60 con 0x7fcb08145370 2020-11-16T22:22:43.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00005b80 con 0x7fcb08145370 2020-11-16T22:22:43.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb08140a80 con 0x7fcb08145370 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00007110 con 0x7fcb08145370 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb081544a0 con 0x7fcb08145370 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.002+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08153ca0 con 0x7fcb08145370 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.006+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcb00010030 con 0x7fcb08145370 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.006+0000 7fcaf67fc700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 0x7fcae0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:22:43.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.006+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb0006dd10 con 0x7fcb08145370 2020-11-16T22:22:43.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.006+0000 7fcb0e475700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 0x7fcae0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:22:43.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.006+0000 7fcb0e475700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 0x7fcae0057520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fcb0400c010 tx=0x7fcb04015000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:22:44.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.014+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fcb00073020 con 0x7fcb08145370 2020-11-16T22:22:44.205 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.214+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb0805d200 con 0x7fcb08145370 2020-11-16T22:22:44.206 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcaf67fc700 1 -- 172.21.15.185:0/2338446108 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+2522 (secure 0 0 0) 0x7fcb00043e60 con 0x7fcb08145370 2020-11-16T22:22:44.207 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:22:44.207 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":106,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:22:42.170266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:22:44.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 msgr2=0x7fcae0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:44.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 0x7fcae0057520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fcb0400c010 tx=0x7fcb04015000).stop 2020-11-16T22:22:44.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 msgr2=0x7fcb081422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:22:44.209 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fcb0000ca90 tx=0x7fcb00007ca0).stop 2020-11-16T22:22:44.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 shutdown_connections 2020-11-16T22:22:44.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcae0055060 0x7fcae0057520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:44.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcb08145370 0x7fcb081422b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:44.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb08061e60 0x7fcb08140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:44.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 --2- 172.21.15.185:0/2338446108 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcb080669a0 0x7fcb08153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:22:44.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 >> 172.21.15.185:0/2338446108 conn(0x7fcb08060130 msgr2=0x7fcb080b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:22:44.414 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 shutdown_connections 2020-11-16T22:22:44.415 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:22:44.218+0000 7fcb0f477700 1 -- 172.21.15.185:0/2338446108 wait complete. 2020-11-16T22:22:44.416 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 106 2020-11-16T22:22:44.416 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:22:44.402+0000 7f5dfcaae700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2020-11-16T22:22:44.416 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:22:44.402+0000 7f5dfcaae700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2020-11-16T22:22:44.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:44.417 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:22:44.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:44.485 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo sync d1/d2/d3/d4/d5/d6/d7/d8) 2020-11-16T22:22:44.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:44.524 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 0 d1/d2) 2020-11-16T22:22:44.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:44.561 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2020-11-16T22:22:44.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:22:44.599 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2020-11-16T22:22:44.680 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:44.820 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:22:44.821 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:44.821 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:46.823 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:46.963 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:22:46.964 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:46.965 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:48.966 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:49.105 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:22:49.106 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:49.106 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:51.107 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:51.243 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:22:51.244 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:51.245 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:53.246 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:53.383 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2020-11-16T22:22:53.383 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:53.384 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:55.384 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2020-11-16T22:22:55.521 INFO:teuthology.orchestra.run.smithi185.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"12","version":"12","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2020-11-16T22:22:55.522 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "12", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:22:55.726 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2020-11-16T22:22:55.726 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:22:55.865 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:22:56.070 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:56.071 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:22:58.072 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:22:58.209 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:22:58.210 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:22:58.210 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:23:00.212 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2020-11-16T22:23:00.353 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"12","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"16","version":"16","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2020-11-16T22:23:00.354 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "12" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2020-11-16T22:23:00.354 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:23:00.355 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2020-11-16T22:23:00.478 INFO:teuthology.orchestra.run.smithi096.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2020-11-16T22:23:00.682 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2020-11-16T22:23:00.886 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2020-11-16T22:23:00.887 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:01.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 Processor -- start 2020-11-16T22:23:01.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 -- start start 2020-11-16T22:23:01.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e28062900 0x7f4e28062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4e281446c0 0x7f4e280b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e28067930 con 0x7f4e28067de0 2020-11-16T22:23:01.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e280632e0 con 0x7f4e28062900 2020-11-16T22:23:01.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e2d37d700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e28066ae0 con 0x7f4e281446c0 2020-11-16T22:23:01.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e267fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4e281446c0 0x7f4e280b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e267fc700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f4e281446c0 0x7f4e280b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35808/0 (socket says 172.21.15.185:35808) 2020-11-16T22:23:01.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e267fc700 1 -- 172.21.15.185:0/3998492125 learned_addr learned my addr 172.21.15.185:0/3998492125 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:01.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e277fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f4e28067de0 0x7f4e28067220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57480/0 (socket says 172.21.15.185:57480) 2020-11-16T22:23:01.023 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.030+0000 7f4e26ffd700 1 --2- 172.21.15.185:0/3998492125 >> v2:172.21.15.96:3300/0 conn(0x7f4e28062900 0x7f4e28062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e267fc700 1 -- 172.21.15.185:0/3998492125 >> v2:172.21.15.96:3300/0 conn(0x7f4e28062900 msgr2=0x7f4e28062ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e267fc700 1 --2- 172.21.15.185:0/3998492125 >> v2:172.21.15.96:3300/0 conn(0x7f4e28062900 0x7f4e28062ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e25ffb700 1 -- 172.21.15.185:0/3998492125 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888567903 0 0) 0x7f4e28067930 con 0x7f4e28067de0 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e267fc700 1 -- 172.21.15.185:0/3998492125 >> v1:172.21.15.185:6789/0 conn(0x7f4e28067de0 legacy=0x7f4e28067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e267fc700 1 -- 172.21.15.185:0/3998492125 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e28066d80 con 0x7f4e281446c0 2020-11-16T22:23:01.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e267fc700 1 --2- 172.21.15.185:0/3998492125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e281446c0 0x7f4e280b1840 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f4e28066940 tx=0x7f4e10006f40).ready entity=mon.1 client_cookie=45cd1ddc239383ca server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e25ffb700 1 -- 172.21.15.185:0/3998492125 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000cb30 con 0x7f4e281446c0 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e25ffb700 1 -- 172.21.15.185:0/3998492125 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1000cc90 con 0x7f4e281446c0 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e25ffb700 1 -- 172.21.15.185:0/3998492125 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10010620 con 0x7f4e281446c0 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 -- 172.21.15.185:0/3998492125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e281446c0 msgr2=0x7f4e280b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/3998492125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e281446c0 0x7f4e280b1840 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f4e28066940 tx=0x7f4e10006f40).stop 2020-11-16T22:23:01.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 -- 172.21.15.185:0/3998492125 shutdown_connections 2020-11-16T22:23:01.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/3998492125 >> v2:172.21.15.96:3300/0 conn(0x7f4e28062900 0x7f4e28062ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/3998492125 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e281446c0 0x7f4e280b1840 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 -- 172.21.15.185:0/3998492125 >> 172.21.15.185:0/3998492125 conn(0x7f4e280b4410 msgr2=0x7f4e28060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:01.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 -- 172.21.15.185:0/3998492125 shutdown_connections 2020-11-16T22:23:01.230 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.034+0000 7f4e2d37d700 1 -- 172.21.15.185:0/3998492125 wait complete. 2020-11-16T22:23:01.232 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 Processor -- start 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- start start 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 0x7f4e280ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 0x7f4e28153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 0x7f4e2809c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280adcb0 con 0x7f4e28067de0 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280b11d0 con 0x7f4e28062900 2020-11-16T22:23:01.233 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280ada10 con 0x7f4e281446c0 2020-11-16T22:23:01.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 0x7f4e280ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 0x7f4e280ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35814/0 (socket says 172.21.15.185:35814) 2020-11-16T22:23:01.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e26ffd700 1 -- 172.21.15.185:0/2872612012 learned_addr learned my addr 172.21.15.185:0/2872612012 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:01.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 0x7f4e2809c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.234 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e267fc700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 0x7f4e28153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 -- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 msgr2=0x7f4e280ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 0x7f4e280ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 -- 172.21.15.185:0/2872612012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 msgr2=0x7f4e28153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.438 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 0x7f4e28153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 -- 172.21.15.185:0/2872612012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e281534d0 con 0x7f4e281446c0 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e267fc700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 0x7f4e28153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e277fe700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 0x7f4e2809c200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f4e2000c790 tx=0x7f4e200078d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200072e0 con 0x7f4e281446c0 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20005580 con 0x7f4e281446c0 2020-11-16T22:23:01.439 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e28153770 con 0x7f4e281446c0 2020-11-16T22:23:01.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20007440 con 0x7f4e281446c0 2020-11-16T22:23:01.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.038+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e28154100 con 0x7f4e281446c0 2020-11-16T22:23:01.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28153900 con 0x7f4e281446c0 2020-11-16T22:23:01.440 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f4e2001a060 con 0x7f4e281446c0 2020-11-16T22:23:01.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e1effd700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:01.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e2006d2e0 con 0x7f4e281446c0 2020-11-16T22:23:01.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e26ffd700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:01.642 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.042+0000 7f4e26ffd700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f4e2813ad20 tx=0x7f4e18008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:01.643 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.050+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7f4e20043430 con 0x7f4e281446c0 2020-11-16T22:23:01.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e2805d4e0 con 0x7f4e281446c0 2020-11-16T22:23:01.644 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e1effd700 1 -- 172.21.15.185:0/2872612012 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+2526 (secure 0 0 0) 0x7f4e20018020 con 0x7f4e281446c0 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 msgr2=0x7f4e04057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f4e2813ad20 tx=0x7f4e18008040).stop 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 msgr2=0x7f4e2809c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.250+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 0x7f4e2809c200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f4e2000c790 tx=0x7f4e200078d0).stop 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 shutdown_connections 2020-11-16T22:23:01.645 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f4e04055060 0x7f4e04057520 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f4e28062900 0x7f4e280ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f4e281446c0 0x7f4e2809c200 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 --2- 172.21.15.185:0/2872612012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e28067de0 0x7f4e28153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:01.846 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 >> 172.21.15.185:0/2872612012 conn(0x7f4e280b4410 msgr2=0x7f4e2813f680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:01.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 shutdown_connections 2020-11-16T22:23:01.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:01.254+0000 7f4e2d37d700 1 -- 172.21.15.185:0/2872612012 wait complete. 2020-11-16T22:23:01.847 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 108 2020-11-16T22:23:01.847 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:01.847 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:00.265142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:02.050 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a flush journal 2020-11-16T22:23:02.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.194+0000 7efe933c2700 1 Processor -- start 2020-11-16T22:23:02.185 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.194+0000 7efe933c2700 1 -- start start 2020-11-16T22:23:02.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efe8c060130 0x7efe8c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8c060970 con 0x7efe8c0b3830 2020-11-16T22:23:02.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c0b4610 con 0x7efe8c060130 2020-11-16T22:23:02.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c060ba0 con 0x7efe8c0ac820 2020-11-16T22:23:02.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57500/0 (socket says 172.21.15.185:57500) 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe923c0700 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efe8c0b3830 0x7efe8c143a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.185:43556/0 (socket says 172.21.15.185:43556) 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 -- 172.21.15.185:0/3608801838 learned_addr learned my addr 172.21.15.185:0/3608801838 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe92bc1700 1 --2- 172.21.15.185:0/3608801838 >> v2:172.21.15.96:3300/0 conn(0x7efe8c060130 0x7efe8c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 -- 172.21.15.185:0/3608801838 >> v2:172.21.15.96:3300/0 conn(0x7efe8c060130 msgr2=0x7efe8c13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:02.189 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 --2- 172.21.15.185:0/3608801838 >> v2:172.21.15.96:3300/0 conn(0x7efe8c060130 0x7efe8c13fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:02.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe913be700 1 -- 172.21.15.185:0/3608801838 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495123081 0 0) 0x7efe8c060970 con 0x7efe8c0b3830 2020-11-16T22:23:02.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 -- 172.21.15.185:0/3608801838 >> v1:172.21.15.96:6789/0 conn(0x7efe8c0b3830 legacy=0x7efe8c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:02.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 -- 172.21.15.185:0/3608801838 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c060e40 con 0x7efe8c0ac820 2020-11-16T22:23:02.390 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe91bbf700 1 --2- 172.21.15.185:0/3608801838 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7efe7c006fd0 tx=0x7efe7c006f40).ready entity=mon.2 client_cookie=4aba0e24cf06174c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:02.401 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe913be700 1 -- 172.21.15.185:0/3608801838 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00cad0 con 0x7efe8c0ac820 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe913be700 1 -- 172.21.15.185:0/3608801838 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c00cc30 con 0x7efe8c0ac820 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe913be700 1 -- 172.21.15.185:0/3608801838 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c010410 con 0x7efe8c0ac820 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- 172.21.15.185:0/3608801838 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 msgr2=0x7efe8c13f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 --2- 172.21.15.185:0/3608801838 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7efe7c006fd0 tx=0x7efe7c006f40).stop 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- 172.21.15.185:0/3608801838 shutdown_connections 2020-11-16T22:23:02.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 --2- 172.21.15.185:0/3608801838 >> v2:172.21.15.96:3300/0 conn(0x7efe8c060130 0x7efe8c13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:02.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 --2- 172.21.15.185:0/3608801838 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c13f840 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:02.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- 172.21.15.185:0/3608801838 >> 172.21.15.185:0/3608801838 conn(0x7efe8c063df0 msgr2=0x7efe8c0641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:02.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- 172.21.15.185:0/3608801838 shutdown_connections 2020-11-16T22:23:02.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.198+0000 7efe933c2700 1 -- 172.21.15.185:0/3608801838 wait complete. 2020-11-16T22:23:02.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 Processor -- start 2020-11-16T22:23:02.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- start start 2020-11-16T22:23:02.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 0x7efe8c13d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe8c0b3830 0x7efe8c1437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c141780 con 0x7efe8c060130 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0ad230 con 0x7efe8c0b3830 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c141b40 con 0x7efe8c0ac820 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57506/0 (socket says 172.21.15.185:57506) 2020-11-16T22:23:02.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 -- 172.21.15.185:0/442249076 learned_addr learned my addr 172.21.15.185:0/442249076 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:02.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe92bc1700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe8c0b3830 0x7efe8c1437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe923c0700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 0x7efe8c13d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 -- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe8c0b3830 msgr2=0x7efe8c1437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:02.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe8c0b3830 0x7efe8c1437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:02.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 -- 172.21.15.185:0/442249076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 msgr2=0x7efe8c13d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:02.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 0x7efe8c13d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:02.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 -- 172.21.15.185:0/442249076 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c142e60 con 0x7efe8c0ac820 2020-11-16T22:23:02.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe923c0700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 0x7efe8c13d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:02.810 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe91bbf700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7efe7c00c7d0 tx=0x7efe7c006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe7a7fc700 1 -- 172.21.15.185:0/442249076 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c010e30 con 0x7efe8c0ac820 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe7a7fc700 1 -- 172.21.15.185:0/442249076 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c01a650 con 0x7efe8c0ac820 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe7a7fc700 1 -- 172.21.15.185:0/442249076 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01a9d0 con 0x7efe8c0ac820 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c13d750 con 0x7efe8c0ac820 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.202+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c0ae6d0 con 0x7efe8c0ac820 2020-11-16T22:23:02.811 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.206+0000 7efe7a7fc700 1 -- 172.21.15.185:0/442249076 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efe7c00e040 con 0x7efe8c0ac820 2020-11-16T22:23:02.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.206+0000 7efe7a7fc700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 0x7efe64057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:02.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.206+0000 7efe923c0700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 0x7efe64057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:02.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.206+0000 7efe7a7fc700 1 -- 172.21.15.185:0/442249076 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe7c06fa40 con 0x7efe8c0ac820 2020-11-16T22:23:02.812 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.206+0000 7efe923c0700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 0x7efe64057520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efe88005b10 tx=0x7efe88007cb0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:03.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 Processor -- start 2020-11-16T22:23:03.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 -- start start 2020-11-16T22:23:03.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.014 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21424b0 con 0x20987b0 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x21f6930 con 0x202abb0 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x20b66f0 con 0x22b6da0 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35838/0 (socket says 172.21.15.185:35838) 2020-11-16T22:23:03.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 -- 172.21.15.185:0/4063711714 learned_addr learned my addr 172.21.15.185:0/4063711714 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:03.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe92bc1700 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe923c0700 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 -- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 msgr2=0x22b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:03.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 -- 172.21.15.185:0/4063711714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 msgr2=0x22b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 -- 172.21.15.185:0/4063711714 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20b15a0 con 0x202abb0 2020-11-16T22:23:03.218 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7efe7c010fd0 tx=0x7efe7c015000).ready entity=mon.1 client_cookie=8956ad528dc1a9c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe6b7fe700 1 -- 172.21.15.185:0/4063711714 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01adc0 con 0x202abb0 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe6b7fe700 1 -- 172.21.15.185:0/4063711714 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c047d00 con 0x202abb0 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.210+0000 7efe6b7fe700 1 -- 172.21.15.185:0/4063711714 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c048e50 con 0x202abb0 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 msgr2=0x22b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7efe7c010fd0 tx=0x7efe7c015000).stop 2020-11-16T22:23:03.219 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- 172.21.15.185:0/4063711714 shutdown_connections 2020-11-16T22:23:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- 172.21.15.185:0/4063711714 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b4e90 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- 172.21.15.185:0/4063711714 >> 172.21.15.185:0/4063711714 conn(0x2266e40 msgr2=0x22b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:03.220 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- 172.21.15.185:0/4063711714 shutdown_connections 2020-11-16T22:23:03.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- 172.21.15.185:0/4063711714 wait complete. 2020-11-16T22:23:03.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- start start 2020-11-16T22:23:03.422 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 Processor -- start 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 0x22b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e36e0 con 0x202abb0 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e3820 con 0x20987b0 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x21e4a90 con 0x22b6da0 2020-11-16T22:23:03.423 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe92bc1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efea30b5740 10 client.0 ll_register_callbacks cb 0x21d9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35844/0 (socket says 172.21.15.185:35844) 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 -- 172.21.15.185:0/1450546751 learned_addr learned my addr 172.21.15.185:0/1450546751 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe92bc1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57522/0 (socket says 172.21.15.185:57522) 2020-11-16T22:23:03.424 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe923c0700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 0x22b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 -- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 msgr2=0x22fe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:03.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22fe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 -- 172.21.15.185:0/1450546751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 msgr2=0x22b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:03.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 0x22b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 -- 172.21.15.185:0/1450546751 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x21d4280 con 0x20987b0 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe923c0700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 0x22b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe91bbf700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7efe7c00cc00 tx=0x7efe7c0495d0).ready entity=mon.1 client_cookie=3168f7e7ff34170b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.214+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c048e50 con 0x20987b0 2020-11-16T22:23:03.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe7c00c030 con 0x20987b0 2020-11-16T22:23:03.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x22976c0 con 0x20987b0 2020-11-16T22:23:03.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7c0488c0 con 0x20987b0 2020-11-16T22:23:03.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 10 client.8185 fetch_fsmap learned FSMap version 108 2020-11-16T22:23:03.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x2009fd0 con 0x20987b0 2020-11-16T22:23:03.628 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x22976c0 con 0x20987b0 2020-11-16T22:23:03.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 108) v1 ==== 1485+0+0 (secure 0 0 0) 0x7efe7c021070 con 0x20987b0 2020-11-16T22:23:03.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 10 client.8185 fetch_fsmap finished waiting for FSMap version 108 2020-11-16T22:23:03.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 10 client.8185 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:03.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 20 client.8185 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22ec4b0 0x22e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 4 client.8185 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x20d4c50 con 0x22ec4b0 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe923c0700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22ec4b0 0x22e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 10 client.8185 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:03.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe923c0700 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22ec4b0 0x22e5150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=29c33dbefb094459 server_cookie=ca9caf2e3ef28d9d in_seq=0 out_seq=0 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 1 -- 172.21.15.185:0/1450546751 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x20d4c50 con 0x22ec4b0 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 10 client.8185 handle_command_reply: tid=0 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 client.8185 shutdown 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 msgr2=0x22fa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7efe7c00cc00 tx=0x7efe7c0495d0).stop 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 shutdown_connections 2020-11-16T22:23:03.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22ec4b0 0x22e5150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efe68ff9700 0 client.8185 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:04.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x22b6da0 0x22fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x202abb0 0x22b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 --2- 172.21.15.185:0/1450546751 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x20987b0 0x22fa640 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 >> 172.21.15.185:0/1450546751 conn(0x2266e40 msgr2=0x22e8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 shutdown_connections 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 1 -- 172.21.15.185:0/1450546751 wait complete. 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.218+0000 7efea30b5740 20 client.8185 trim_cache size 0 max 16384 2020-11-16T22:23:04.035 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 Processor -- start 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 -- start start 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 0x22b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x21bbe40 con 0x20987b0 2020-11-16T22:23:04.036 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22b6a50 con 0x202abb0 2020-11-16T22:23:04.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2290800 con 0x2266e40 2020-11-16T22:23:04.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.037 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35850/0 (socket says 172.21.15.185:35850) 2020-11-16T22:23:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 -- 172.21.15.185:0/3944186790 learned_addr learned my addr 172.21.15.185:0/3944186790 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe923c0700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 0x22b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.238 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe92bc1700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 -- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 msgr2=0x22b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 0x22b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 -- 172.21.15.185:0/3944186790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 msgr2=0x22b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 -- 172.21.15.185:0/3944186790 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58008160 con 0x202abb0 2020-11-16T22:23:04.239 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe923c0700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 0x22b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:04.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe92bc1700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:23:04.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.234+0000 7efe91bbf700 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7efe7c049660 tx=0x7efe7c038040).ready entity=mon.1 client_cookie=5365b3a026522864 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:04.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe6bfff700 1 -- 172.21.15.185:0/3944186790 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0488c0 con 0x202abb0 2020-11-16T22:23:04.240 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe6bfff700 1 -- 172.21.15.185:0/3944186790 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c048e50 con 0x202abb0 2020-11-16T22:23:04.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe6bfff700 1 -- 172.21.15.185:0/3944186790 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c06f490 con 0x202abb0 2020-11-16T22:23:04.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 msgr2=0x22b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:04.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7efe7c049660 tx=0x7efe7c038040).stop 2020-11-16T22:23:04.442 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/3944186790 shutdown_connections 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x20987b0 0x22b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2266e40 0x22b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- 172.21.15.185:0/3944186790 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2760 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/3944186790 >> 172.21.15.185:0/3944186790 conn(0x7efe58002320 msgr2=0x22e5150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/3944186790 shutdown_connections 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/3944186790 wait complete. 2020-11-16T22:23:04.443 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- start start 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 Processor -- start 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 0x234fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x22fa6c0 con 0x2266e40 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x22f9940 con 0x202abb0 2020-11-16T22:23:04.444 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2354ac0 con 0x20987b0 2020-11-16T22:23:04.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 10 client.0 ll_register_callbacks cb 0x22b0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:04.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe923c0700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.646 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57534/0 (socket says 172.21.15.185:57534) 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 -- 172.21.15.185:0/4125883092 learned_addr learned my addr 172.21.15.185:0/4125883092 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe92bc1700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 0x234fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 -- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 msgr2=0x22b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 -- 172.21.15.185:0/4125883092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 msgr2=0x234fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:04.647 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 0x234fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:04.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 -- 172.21.15.185:0/4125883092 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2048720 con 0x20987b0 2020-11-16T22:23:04.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe92bc1700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 0x234fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:04.648 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe91bbf700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7efe7c000dd0 tx=0x7efe7c01d000).ready entity=mon.2 client_cookie=827b012729279ba8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:04.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:04.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0488c0 con 0x20987b0 2020-11-16T22:23:04.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe7c00c030 con 0x20987b0 2020-11-16T22:23:04.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe7c047d00 con 0x20987b0 2020-11-16T22:23:04.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7c048fc0 con 0x20987b0 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 10 client.7283 fetch_fsmap learned FSMap version 108 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x2053e00 con 0x20987b0 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c047d00 con 0x20987b0 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 108) v1 ==== 1485+0+0 (secure 0 0 0) 0x7efe7c021040 con 0x20987b0 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 10 client.7283 fetch_fsmap finished waiting for FSMap version 108 2020-11-16T22:23:04.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 10 client.7283 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 20 client.7283 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22b6da0 0x2350180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 4 client.7283 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.238+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x20dbf50 con 0x22b6da0 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.242+0000 7efe923c0700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22b6da0 0x2350180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:04.852 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.242+0000 7efe923c0700 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22b6da0 0x2350180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=dec2b004045c6224 server_cookie=a5835c69904569f6 in_seq=0 out_seq=0 2020-11-16T22:23:05.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.242+0000 7efe68ff9700 10 client.7283 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:05.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efe68ff9700 1 -- 172.21.15.185:0/4125883092 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x20dbf50 con 0x22b6da0 2020-11-16T22:23:05.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efe68ff9700 10 client.7283 handle_command_reply: tid=0 2020-11-16T22:23:05.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efea30b5740 1 client.7283 shutdown 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 msgr2=0x22b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7efe7c000dd0 tx=0x7efe7c01d000).stop 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 shutdown_connections 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.262+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x22b6da0 0x2350180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x202abb0 0x22b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2266e40 0x234fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 --2- 172.21.15.185:0/4125883092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x20987b0 0x22b9e20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe68ff9700 0 client.7283 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 >> 172.21.15.185:0/4125883092 conn(0x7efe58002320 msgr2=0x22b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 shutdown_connections 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 1 -- 172.21.15.185:0/4125883092 wait complete. 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efea30b5740 20 client.7283 trim_cache size 0 max 16384 2020-11-16T22:23:05.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 msgr2=0x7efe64057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:05.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 0x7efe64057520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7efe88005b10 tx=0x7efe88007cb0).stop 2020-11-16T22:23:05.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 msgr2=0x7efe8c143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:05.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7efe7c00c7d0 tx=0x7efe7c006f40).stop 2020-11-16T22:23:05.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 shutdown_connections 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efe64055060 0x7efe64057520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efe8c0b3830 0x7efe8c1437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c060130 0x7efe8c13d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 --2- 172.21.15.185:0/442249076 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efe8c0ac820 0x7efe8c143300 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 >> 172.21.15.185:0/442249076 conn(0x7efe8c063df0 msgr2=0x7efe8c0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:05.259 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 shutdown_connections 2020-11-16T22:23:05.260 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:02.266+0000 7efe933c2700 1 -- 172.21.15.185:0/442249076 wait complete. 2020-11-16T22:23:05.260 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:05.260 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:23:05.260 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:23:05.260 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:05.261 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:05.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 Processor -- start 2020-11-16T22:23:05.393 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 -- start start 2020-11-16T22:23:05.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7faa900b3090 0x7faa900b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:05.395 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7faa900b1d30 0x7faa9014b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa900b2b60 con 0x7faa900b3090 2020-11-16T22:23:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa90061fe0 con 0x7faa900b1d30 2020-11-16T22:23:05.598 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa963d2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900b0fe0 con 0x7faa900ad110 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa95bd1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa94bcf700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7faa900b1d30 0x7faa9014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa95bd1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57542/0 (socket says 172.21.15.185:57542) 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.402+0000 7faa95bd1700 1 -- 172.21.15.185:0/3197239555 learned_addr learned my addr 172.21.15.185:0/3197239555 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa953d0700 1 --2- 172.21.15.185:0/3197239555 >> v2:172.21.15.96:3300/0 conn(0x7faa900b3090 0x7faa900b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 -- 172.21.15.185:0/3197239555 >> v2:172.21.15.185:3300/0 conn(0x7faa900b1d30 msgr2=0x7faa9014b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:05.599 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 --2- 172.21.15.185:0/3197239555 >> v2:172.21.15.185:3300/0 conn(0x7faa900b1d30 0x7faa9014b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 -- 172.21.15.185:0/3197239555 >> v2:172.21.15.96:3300/0 conn(0x7faa900b3090 msgr2=0x7faa900b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 --2- 172.21.15.185:0/3197239555 >> v2:172.21.15.96:3300/0 conn(0x7faa900b3090 0x7faa900b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 -- 172.21.15.185:0/3197239555 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900b1280 con 0x7faa900ad110 2020-11-16T22:23:05.600 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 --2- 172.21.15.185:0/3197239555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7faa90066720 tx=0x7faa78006eb0).ready entity=mon.2 client_cookie=37a95952bfaa0763 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:05.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa8ffff700 1 -- 172.21.15.185:0/3197239555 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78009810 con 0x7faa900ad110 2020-11-16T22:23:05.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa8ffff700 1 -- 172.21.15.185:0/3197239555 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78009970 con 0x7faa900ad110 2020-11-16T22:23:05.802 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa8ffff700 1 -- 172.21.15.185:0/3197239555 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78009cf0 con 0x7faa900ad110 2020-11-16T22:23:05.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- 172.21.15.185:0/3197239555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 msgr2=0x7faa901477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:05.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- 172.21.15.185:0/3197239555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7faa90066720 tx=0x7faa78006eb0).stop 2020-11-16T22:23:05.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- 172.21.15.185:0/3197239555 shutdown_connections 2020-11-16T22:23:05.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- 172.21.15.185:0/3197239555 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900ad110 0x7faa901477b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.806 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- 172.21.15.185:0/3197239555 >> v2:172.21.15.96:3300/0 conn(0x7faa900b3090 0x7faa900b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- 172.21.15.185:0/3197239555 >> v2:172.21.15.185:3300/0 conn(0x7faa900b1d30 0x7faa9014b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- 172.21.15.185:0/3197239555 >> 172.21.15.185:0/3197239555 conn(0x7faa90060130 msgr2=0x7faa90060540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- 172.21.15.185:0/3197239555 shutdown_connections 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- 172.21.15.185:0/3197239555 wait complete. 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 Processor -- start 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- start start 2020-11-16T22:23:05.807 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa900ad110 0x7faa900afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:05.808 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:06.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faa900b3090 0x7faa9015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:06.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9014b120 con 0x7faa900ad110 2020-11-16T22:23:06.010 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9014b400 con 0x7faa900b3090 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa963d2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900af7e0 con 0x7faa900b1d30 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa94bcf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.406+0000 7faa95bd1700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faa900b3090 0x7faa9015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57548/0 (socket says 172.21.15.185:57548) 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 -- 172.21.15.185:0/2299428239 learned_addr learned my addr 172.21.15.185:0/2299428239 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:06.011 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa953d0700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa900ad110 0x7faa900afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 -- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faa900b3090 msgr2=0x7faa9015ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faa900b3090 0x7faa9015ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 -- 172.21.15.185:0/2299428239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa900ad110 msgr2=0x7faa900afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa900ad110 0x7faa900afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.012 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 -- 172.21.15.185:0/2299428239 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9014b630 con 0x7faa900b1d30 2020-11-16T22:23:06.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa94bcf700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7faa8000ef50 tx=0x7faa800091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:06.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800096d0 con 0x7faa900b1d30 2020-11-16T22:23:06.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000cc40 con 0x7faa900b1d30 2020-11-16T22:23:06.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800123f0 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa900afa40 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa900b04c0 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9005d200 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.410+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7faa80022070 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.414+0000 7faa8d7fa700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 0x7faa6c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.414+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa8006e610 con 0x7faa900b1d30 2020-11-16T22:23:06.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.414+0000 7faa953d0700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 0x7faa6c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.414+0000 7faa953d0700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 0x7faa6c057520 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7faa8800d5f0 tx=0x7faa88008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:06.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.418+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7faa80044870 con 0x7faa900b1d30 2020-11-16T22:23:06.216 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.614+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa900b2ca0 con 0x7faa900b1d30 2020-11-16T22:23:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa8d7fa700 1 -- 172.21.15.185:0/2299428239 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+2528 (secure 0 0 0) 0x7faa80016020 con 0x7faa900b1d30 2020-11-16T22:23:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 msgr2=0x7faa6c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 0x7faa6c057520 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7faa8800d5f0 tx=0x7faa88008040).stop 2020-11-16T22:23:06.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 msgr2=0x7faa9014c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7faa8000ef50 tx=0x7faa800091d0).stop 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 shutdown_connections 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7faa6c055060 0x7faa6c057520 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7faa900b3090 0x7faa9015ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa900ad110 0x7faa900afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 --2- 172.21.15.185:0/2299428239 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7faa900b1d30 0x7faa9014c280 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 >> 172.21.15.185:0/2299428239 conn(0x7faa90060130 msgr2=0x7faa90148ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:06.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 shutdown_connections 2020-11-16T22:23:06.420 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:05.618+0000 7faa963d2700 1 -- 172.21.15.185:0/2299428239 wait complete. 2020-11-16T22:23:06.420 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 109 2020-11-16T22:23:06.420 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:06.420 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:05.291578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:06.622 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c flush journal 2020-11-16T22:23:06.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 Processor -- start 2020-11-16T22:23:06.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- start start 2020-11-16T22:23:06.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb13c0b3700 0x7fb13c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:06.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:06.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c060550 con 0x7fb13c061560 2020-11-16T22:23:06.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0602b0 con 0x7fb13c0b3700 2020-11-16T22:23:06.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0b1710 con 0x7fb13c060a10 2020-11-16T22:23:06.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57554/0 (socket says 172.21.15.185:57554) 2020-11-16T22:23:06.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 -- 172.21.15.185:0/1158135843 learned_addr learned my addr 172.21.15.185:0/1158135843 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:06.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/1158135843 >> v2:172.21.15.96:3300/0 conn(0x7fb13c0b3700 0x7fb13c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:06.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 -- 172.21.15.185:0/1158135843 >> v2:172.21.15.96:3300/0 conn(0x7fb13c0b3700 msgr2=0x7fb13c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:06.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 --2- 172.21.15.185:0/1158135843 >> v2:172.21.15.96:3300/0 conn(0x7fb13c0b3700 0x7fb13c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:06.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 -- 172.21.15.185:0/1158135843 >> v1:172.21.15.96:6789/0 conn(0x7fb13c061560 legacy=0x7fb13c143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 -- 172.21.15.185:0/1158135843 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0b19b0 con 0x7fb13c060a10 2020-11-16T22:23:06.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13bfff700 1 --2- 172.21.15.185:0/1158135843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fb134009d50 tx=0x7fb134009290).ready entity=mon.2 client_cookie=96d6fc129d24bb1d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:06.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13affd700 1 -- 172.21.15.185:0/1158135843 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bbc0 con 0x7fb13c060a10 2020-11-16T22:23:06.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13affd700 1 -- 172.21.15.185:0/1158135843 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400bd20 con 0x7fb13c060a10 2020-11-16T22:23:06.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb13affd700 1 -- 172.21.15.185:0/1158135843 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134011610 con 0x7fb13c060a10 2020-11-16T22:23:06.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- 172.21.15.185:0/1158135843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 msgr2=0x7fb13c0b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:06.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 --2- 172.21.15.185:0/1158135843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fb134009d50 tx=0x7fb134009290).stop 2020-11-16T22:23:07.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- 172.21.15.185:0/1158135843 shutdown_connections 2020-11-16T22:23:07.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 --2- 172.21.15.185:0/1158135843 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c060a10 0x7fb13c0b1e50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 --2- 172.21.15.185:0/1158135843 >> v2:172.21.15.96:3300/0 conn(0x7fb13c0b3700 0x7fb13c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- 172.21.15.185:0/1158135843 >> 172.21.15.185:0/1158135843 conn(0x7fb13c063810 msgr2=0x7fb13c063df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- 172.21.15.185:0/1158135843 shutdown_connections 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.770+0000 7fb140c7c700 1 -- 172.21.15.185:0/1158135843 wait complete. 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 Processor -- start 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- start start 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 0x7fb13c147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb13c13d680 0x7fb13c144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c151cf0 con 0x7fb13c13d680 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c151f10 con 0x7fb13c0b3700 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c13dae0 con 0x7fb13c061560 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35884/0 (socket says 172.21.15.185:35884) 2020-11-16T22:23:07.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 -- 172.21.15.185:0/2710123261 learned_addr learned my addr 172.21.15.185:0/2710123261 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:07.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 0x7fb13c147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb13bfff700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb13c13d680 0x7fb13c144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 -- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 msgr2=0x7fb13c147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:07.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 0x7fb13c147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 -- 172.21.15.185:0/2710123261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb13c13d680 msgr2=0x7fb13c144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb13c13d680 0x7fb13c144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 -- 172.21.15.185:0/2710123261 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0b0b90 con 0x7fb13c0b3700 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 0x7fb13c147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb133fff700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb12400c710 tx=0x7fb124007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb1337fe700 1 -- 172.21.15.185:0/2710123261 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124007020 con 0x7fb13c0b3700 2020-11-16T22:23:07.383 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb1337fe700 1 -- 172.21.15.185:0/2710123261 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb124007180 con 0x7fb13c0b3700 2020-11-16T22:23:07.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c148220 con 0x7fb13c0b3700 2020-11-16T22:23:07.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb1337fe700 1 -- 172.21.15.185:0/2710123261 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400e560 con 0x7fb13c0b3700 2020-11-16T22:23:07.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c0af9a0 con 0x7fb13c0b3700 2020-11-16T22:23:07.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.774+0000 7fb1337fe700 1 -- 172.21.15.185:0/2710123261 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb124018020 con 0x7fb13c0b3700 2020-11-16T22:23:07.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.778+0000 7fb1337fe700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 0x7fb118057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.778+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 0x7fb118057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.586 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.778+0000 7fb1337fe700 1 -- 172.21.15.185:0/2710123261 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb12406d0b0 con 0x7fb13c0b3700 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.778+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 0x7fb118057520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fb12c006fd0 tx=0x7fb12c008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.778+0000 7fb15096f740 1 Processor -- start 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- start start 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2abd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.587 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b01610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x294a4b0 con 0x2b011b0 2020-11-16T22:23:07.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x29fe930 con 0x28a07b0 2020-11-16T22:23:07.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x28be6f0 con 0x2832bb0 2020-11-16T22:23:07.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.588 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57570/0 (socket says 172.21.15.185:57570) 2020-11-16T22:23:07.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 -- 172.21.15.185:0/1098150455 learned_addr learned my addr 172.21.15.185:0/1098150455 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:07.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2abd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb13bfff700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b01610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:07.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 -- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 msgr2=0x2abd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2abd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 -- 172.21.15.185:0/1098150455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 msgr2=0x2b01610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b01610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 -- 172.21.15.185:0/1098150455 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28b95a0 con 0x2832bb0 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb13bfff700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b01610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:07.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb133fff700 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb1240469e0 tx=0x7fb124013000).ready entity=mon.2 client_cookie=94c8624d8c3bf3d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:07.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb130ff9700 1 -- 172.21.15.185:0/1098150455 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124001ea0 con 0x2832bb0 2020-11-16T22:23:07.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb130ff9700 1 -- 172.21.15.185:0/1098150455 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240464c0 con 0x2832bb0 2020-11-16T22:23:07.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb130ff9700 1 -- 172.21.15.185:0/1098150455 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124042dc0 con 0x2832bb0 2020-11-16T22:23:07.792 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 msgr2=0x2abea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:07.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fb1240469e0 tx=0x7fb124013000).stop 2020-11-16T22:23:07.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- 172.21.15.185:0/1098150455 shutdown_connections 2020-11-16T22:23:07.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2abd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b01610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 --2- 172.21.15.185:0/1098150455 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2832bb0 0x2abea90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- 172.21.15.185:0/1098150455 >> 172.21.15.185:0/1098150455 conn(0x2a6ee40 msgr2=0x2abbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- 172.21.15.185:0/1098150455 shutdown_connections 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- 172.21.15.185:0/1098150455 wait complete. 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 -- start start 2020-11-16T22:23:07.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.782+0000 7fb15096f740 1 Processor -- start 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2af1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b0a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29eb6e0 con 0x2b011b0 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x29eb820 con 0x2832bb0 2020-11-16T22:23:07.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29eca90 con 0x28a07b0 2020-11-16T22:23:07.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 10 client.0 ll_register_callbacks cb 0x29e1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:07.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35898/0 (socket says 172.21.15.185:35898) 2020-11-16T22:23:08.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 -- 172.21.15.185:0/3844903748 learned_addr learned my addr 172.21.15.185:0/3844903748 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb133fff700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2af1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13bfff700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b0a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 -- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 msgr2=0x2af1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2af1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 -- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 msgr2=0x2b0a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b0a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 -- 172.21.15.185:0/3844903748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x29dc280 con 0x2832bb0 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fb12c012ae0 tx=0x7fb12c014000).ready entity=mon.1 client_cookie=1229e33d2f643aea server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c013850 con 0x2832bb0 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb12c0116b0 con 0x2832bb0 2020-11-16T22:23:08.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2a9f6c0 con 0x2832bb0 2020-11-16T22:23:08.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb12c011a80 con 0x2832bb0 2020-11-16T22:23:08.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 10 client.8206 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:08.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x2811fd0 con 0x2832bb0 2020-11-16T22:23:08.402 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2a9f6c0 con 0x2832bb0 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb12c011be0 con 0x2832bb0 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 10 client.8206 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 10 client.8206 resolve_mds: resolved ID 'c' to GID 7830 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 20 client.8206 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2af1a20 0x2b0aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 4 client.8206 mds_command: new command op to 7830 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:08.403 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 --> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x28dcc50 con 0x2af1a20 2020-11-16T22:23:08.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb133fff700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2af1a20 0x2b0aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb133fff700 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2af1a20 0x2b0aaf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=17784dee58a7ec2b server_cookie=a79fbf0332ee78ff in_seq=0 out_seq=0 2020-11-16T22:23:08.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.786+0000 7fb111ffb700 10 client.8206 ms_handle_connect on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:08.404 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb111ffb700 1 -- 172.21.15.185:0/3844903748 <== mds.1 v2:172.21.15.96:6834/477883541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x28dcc50 con 0x2af1a20 2020-11-16T22:23:08.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb111ffb700 10 client.8206 handle_command_reply: tid=0 2020-11-16T22:23:08.606 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 client.8206 shutdown 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 msgr2=0x2af1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fb12c012ae0 tx=0x7fb12c014000).stop 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 shutdown_connections 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2af1a20 0x2b0aaf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2832bb0 0x2af1040 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb111ffb700 0 client.8206 ms_handle_reset on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:08.607 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2b011b0 0x2b0a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 --2- 172.21.15.185:0/3844903748 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2af1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 >> 172.21.15.185:0/3844903748 conn(0x2a6ee40 msgr2=0x2abea90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 shutdown_connections 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 1 -- 172.21.15.185:0/3844903748 wait complete. 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.790+0000 7fb15096f740 20 client.8206 trim_cache size 0 max 16384 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 Processor -- start 2020-11-16T22:23:08.608 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- start start 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a6ee40 0x2abd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2aba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x29c3e40 con 0x2832bb0 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ab8bc0 con 0x28a07b0 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2aec640 con 0x2a6ee40 2020-11-16T22:23:08.609 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13b7fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a6ee40 0x2abd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35906/0 (socket says 172.21.15.185:35906) 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 -- 172.21.15.185:0/2205881441 learned_addr learned my addr 172.21.15.185:0/2205881441 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb133fff700 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2aba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 -- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a6ee40 msgr2=0x2abd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a6ee40 0x2abd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 -- 172.21.15.185:0/2205881441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 msgr2=0x2aba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2aba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 -- 172.21.15.185:0/2205881441 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080081d0 con 0x28a07b0 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb13bfff700 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fb134006e70 tx=0x7fb134016000).ready entity=mon.1 client_cookie=dc2ac83408d3ed5b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb1317fa700 1 -- 172.21.15.185:0/2205881441 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134022070 con 0x28a07b0 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb1317fa700 1 -- 172.21.15.185:0/2205881441 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400be90 con 0x28a07b0 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb1317fa700 1 -- 172.21.15.185:0/2205881441 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bbc0 con 0x28a07b0 2020-11-16T22:23:08.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 msgr2=0x2ae7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fb134006e70 tx=0x7fb134016000).stop 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- 172.21.15.185:0/2205881441 shutdown_connections 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x28a07b0 0x2ae7db0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2a6ee40 0x2abd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 --2- 172.21.15.185:0/2205881441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2aba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- 172.21.15.185:0/2205881441 >> 172.21.15.185:0/2205881441 conn(0x7fb108002320 msgr2=0x2aec800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- 172.21.15.185:0/2205881441 shutdown_connections 2020-11-16T22:23:08.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.810+0000 7fb15096f740 1 -- 172.21.15.185:0/2205881441 wait complete. 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 -- start start 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 Processor -- start 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2ae8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2b596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2af1fa0 con 0x2832bb0 2020-11-16T22:23:08.613 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2abf4c0 con 0x2a6ee40 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2abec10 con 0x28a07b0 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 10 client.0 ll_register_callbacks cb 0x2abe5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2b596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb133fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2b596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57588/0 (socket says 172.21.15.185:57588) 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35914/0 (socket says 172.21.15.185:35914) 2020-11-16T22:23:08.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb133fff700 1 -- 172.21.15.185:0/3520135092 learned_addr learned my addr 172.21.15.185:0/3520135092 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2ae8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 -- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 msgr2=0x2b596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2b596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 -- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 msgr2=0x2ae8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2ae8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 -- 172.21.15.185:0/3520135092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2850720 con 0x2a6ee40 2020-11-16T22:23:08.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb13bfff700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7fb134023000 tx=0x7fb134022000).ready entity=mon.1 client_cookie=4d781ca45b61cb1d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bbc0 con 0x2a6ee40 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb13401d690 con 0x2a6ee40 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1240464c0 con 0x2a6ee40 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb13401da60 con 0x2a6ee40 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.814+0000 7fb15096f740 10 client.8212 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x285be00 con 0x2a6ee40 2020-11-16T22:23:08.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240464c0 con 0x2a6ee40 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb134030070 con 0x2a6ee40 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 10 client.8212 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 10 client.8212 resolve_mds: resolved ID 'c' to GID 7830 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 20 client.8212 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2b011b0 0x2b5c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 4 client.8212 mds_command: new command op to 7830 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:23:08.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 --> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x28e3f50 con 0x2b011b0 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2b011b0 0x2b5c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb111ffb700 10 client.8212 ms_handle_connect on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.818+0000 7fb13b7fe700 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2b011b0 0x2b5c970 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=7b8bae583099760e server_cookie=d651fb7b67665087 in_seq=0 out_seq=0 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb111ffb700 1 -- 172.21.15.185:0/3520135092 <== mds.1 v2:172.21.15.96:6834/477883541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x28e3f50 con 0x2b011b0 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb111ffb700 10 client.8212 handle_command_reply: tid=0 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 client.8212 shutdown 2020-11-16T22:23:08.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 msgr2=0x2b59ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7fb134023000 tx=0x7fb134022000).stop 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 shutdown_connections 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2b011b0 0x2b5c970 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2a6ee40 0x2b59ba0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2832bb0 0x2ae8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb111ffb700 0 client.8212 ms_handle_reset on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:08.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 --2- 172.21.15.185:0/3520135092 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x28a07b0 0x2b596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 >> 172.21.15.185:0/3520135092 conn(0x7fb108002320 msgr2=0x2aba200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 shutdown_connections 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 1 -- 172.21.15.185:0/3520135092 wait complete. 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.834+0000 7fb15096f740 20 client.8212 trim_cache size 0 max 16384 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 msgr2=0x7fb118057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 0x7fb118057520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fb12c006fd0 tx=0x7fb12c008040).stop 2020-11-16T22:23:08.620 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 msgr2=0x7fb13c13d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fb12400c710 tx=0x7fb124007c50).stop 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 shutdown_connections 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb118055060 0x7fb118057520 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb13c13d680 0x7fb13c144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb13c061560 0x7fb13c147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 --2- 172.21.15.185:0/2710123261 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb13c0b3700 0x7fb13c13d190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 >> 172.21.15.185:0/2710123261 conn(0x7fb13c060a10 msgr2=0x7fb13c0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 shutdown_connections 2020-11-16T22:23:08.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:06.838+0000 7fb140c7c700 1 -- 172.21.15.185:0/2710123261 wait complete. 2020-11-16T22:23:08.622 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:08.623 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:23:08.623 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:23:08.623 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:08.623 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:08.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 Processor -- start 2020-11-16T22:23:08.753 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 -- start start 2020-11-16T22:23:08.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc50061640 0x7efc500b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc50062700 con 0x7efc50061cf0 2020-11-16T22:23:08.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc500629a0 con 0x7efc500b0300 2020-11-16T22:23:08.754 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.762+0000 7efc55dd4700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50062c40 con 0x7efc50061640 2020-11-16T22:23:08.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4effd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc50061640 0x7efc500b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4f7fe700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7efc500b0300 0x7efc500b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57588/0 (socket says 172.21.15.185:57588) 2020-11-16T22:23:08.755 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4f7fe700 1 -- 172.21.15.185:0/908791647 learned_addr learned my addr 172.21.15.185:0/908791647 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:23:08.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4effd700 1 -- 172.21.15.185:0/908791647 >> v1:172.21.15.185:6789/0 conn(0x7efc500b0300 legacy=0x7efc500b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4effd700 1 -- 172.21.15.185:0/908791647 >> v1:172.21.15.96:6789/0 conn(0x7efc50061cf0 legacy=0x7efc5013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4effd700 1 -- 172.21.15.185:0/908791647 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50062ee0 con 0x7efc50061640 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4effd700 1 --2- 172.21.15.185:0/908791647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc50061640 0x7efc500b06e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7efc4000b9a0 tx=0x7efc40006df0).ready entity=mon.1 client_cookie=5287fbc87127bcb6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4e7fc700 1 -- 172.21.15.185:0/908791647 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400097b0 con 0x7efc50061640 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4e7fc700 1 -- 172.21.15.185:0/908791647 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40009910 con 0x7efc50061640 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc4e7fc700 1 -- 172.21.15.185:0/908791647 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40009c90 con 0x7efc50061640 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- 172.21.15.185:0/908791647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc50061640 msgr2=0x7efc500b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 --2- 172.21.15.185:0/908791647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc50061640 0x7efc500b06e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7efc4000b9a0 tx=0x7efc40006df0).stop 2020-11-16T22:23:08.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- 172.21.15.185:0/908791647 shutdown_connections 2020-11-16T22:23:08.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 --2- 172.21.15.185:0/908791647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc50061640 0x7efc500b06e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- 172.21.15.185:0/908791647 >> 172.21.15.185:0/908791647 conn(0x7efc50060130 msgr2=0x7efc50060510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- 172.21.15.185:0/908791647 shutdown_connections 2020-11-16T22:23:08.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- 172.21.15.185:0/908791647 wait complete. 2020-11-16T22:23:08.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 Processor -- start 2020-11-16T22:23:08.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.766+0000 7efc55dd4700 1 -- start start 2020-11-16T22:23:08.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50061640 0x7efc501534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc50061cf0 0x7efc5013e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5013fc70 con 0x7efc50061640 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5013ff10 con 0x7efc500b0300 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc501401b0 con 0x7efc50061cf0 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4effd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc50061cf0 0x7efc5013e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35926/0 (socket says 172.21.15.185:35926) 2020-11-16T22:23:08.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 -- 172.21.15.185:0/1467883853 learned_addr learned my addr 172.21.15.185:0/1467883853 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:08.761 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4f7fe700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50061640 0x7efc501534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 -- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc50061cf0 msgr2=0x7efc5013e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc50061cf0 0x7efc5013e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 -- 172.21.15.185:0/1467883853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50061640 msgr2=0x7efc501534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50061640 0x7efc501534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 -- 172.21.15.185:0/1467883853 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50140450 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc4ffff700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7efc4800ca60 tx=0x7efc48007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800a6d0 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800a830 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5013eda0 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50140df0 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800abb0 con 0x7efc500b0300 2020-11-16T22:23:08.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.770+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5005d4e0 con 0x7efc500b0300 2020-11-16T22:23:08.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.774+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efc48019030 con 0x7efc500b0300 2020-11-16T22:23:08.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.774+0000 7efc477fe700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 0x7efc2c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:08.764 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.774+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc4806d5a0 con 0x7efc500b0300 2020-11-16T22:23:08.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.774+0000 7efc4f7fe700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 0x7efc2c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:08.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.774+0000 7efc4f7fe700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 0x7efc2c057520 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7efc38007250 tx=0x7efc38007af0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:08.771 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.782+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efc480436f0 con 0x7efc500b0300 2020-11-16T22:23:08.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc50063850 con 0x7efc500b0300 2020-11-16T22:23:08.971 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc477fe700 1 -- 172.21.15.185:0/1467883853 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+2528 (secure 0 0 0) 0x7efc48046d10 con 0x7efc500b0300 2020-11-16T22:23:08.972 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:08.972 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:05.291578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:08.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 msgr2=0x7efc2c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 0x7efc2c057520 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7efc38007250 tx=0x7efc38007af0).stop 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 msgr2=0x7efc501463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7efc4800ca60 tx=0x7efc48007eb0).stop 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 shutdown_connections 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc2c055060 0x7efc2c057520 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc500b0300 0x7efc501463f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50061640 0x7efc501534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 --2- 172.21.15.185:0/1467883853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc50061cf0 0x7efc5013e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:08.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 >> 172.21.15.185:0/1467883853 conn(0x7efc50060130 msgr2=0x7efc5013d650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:08.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 shutdown_connections 2020-11-16T22:23:08.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:08.982+0000 7efc55dd4700 1 -- 172.21.15.185:0/1467883853 wait complete. 2020-11-16T22:23:08.975 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 109 2020-11-16T22:23:08.986 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b flush journal 2020-11-16T22:23:09.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.122+0000 7f259f59e700 1 Processor -- start 2020-11-16T22:23:09.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.122+0000 7f259f59e700 1 -- start start 2020-11-16T22:23:09.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 0x7f25a0067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00b4e80 con 0x7f25a00b74f0 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a0141700 con 0x7f25a00670b0 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00b3e10 con 0x7f25a0068030 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259e59c700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7f25a00670b0 0x7f25a00b4770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57604/0 (socket says 172.21.15.185:57604) 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259e59c700 1 -- 172.21.15.185:0/1845039853 learned_addr learned my addr 172.21.15.185:0/1845039853 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:23:09.116 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259ed9d700 1 --2- 172.21.15.185:0/1845039853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 0x7f25a0067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259ed9d700 1 -- 172.21.15.185:0/1845039853 >> v1:172.21.15.185:6789/0 conn(0x7f25a00670b0 legacy=0x7f25a00b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259ed9d700 1 -- 172.21.15.185:0/1845039853 >> v1:172.21.15.96:6789/0 conn(0x7f25a00b74f0 legacy=0x7f25a013aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259dd9b700 1 -- 172.21.15.185:0/1845039853 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135463859 0 0) 0x7f25a0141700 con 0x7f25a00670b0 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259ed9d700 1 -- 172.21.15.185:0/1845039853 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a01419a0 con 0x7f25a0068030 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259ed9d700 1 --2- 172.21.15.185:0/1845039853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 0x7f25a0067490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f2598009cc0 tx=0x7f25980091d0).ready entity=mon.2 client_cookie=362b3023c21f1145 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259dd9b700 1 -- 172.21.15.185:0/1845039853 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800bbc0 con 0x7f25a0068030 2020-11-16T22:23:09.117 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259dd9b700 1 -- 172.21.15.185:0/1845039853 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800bd20 con 0x7f25a0068030 2020-11-16T22:23:09.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259dd9b700 1 -- 172.21.15.185:0/1845039853 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598010680 con 0x7f25a0068030 2020-11-16T22:23:09.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- 172.21.15.185:0/1845039853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 msgr2=0x7f25a0067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.118 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 --2- 172.21.15.185:0/1845039853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 0x7f25a0067490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f2598009cc0 tx=0x7f25980091d0).stop 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- 172.21.15.185:0/1845039853 shutdown_connections 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 --2- 172.21.15.185:0/1845039853 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a0068030 0x7f25a0067490 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- 172.21.15.185:0/1845039853 >> 172.21.15.185:0/1845039853 conn(0x7f25a0064b90 msgr2=0x7f25a0063790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- 172.21.15.185:0/1845039853 shutdown_connections 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.126+0000 7f259f59e700 1 -- 172.21.15.185:0/1845039853 wait complete. 2020-11-16T22:23:09.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 Processor -- start 2020-11-16T22:23:09.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- start start 2020-11-16T22:23:09.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 0x7f25a013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 0x7f25a0136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.121 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0142080 con 0x7f25a0068030 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0142320 con 0x7f25a00b74f0 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01425c0 con 0x7f25a00670b0 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 0x7f25a013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35942/0 (socket says 172.21.15.185:35942) 2020-11-16T22:23:09.122 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 0x7f25a013fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57614/0 (socket says 172.21.15.185:57614) 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 -- 172.21.15.185:0/2790664647 learned_addr learned my addr 172.21.15.185:0/2790664647 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f2597fff700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 0x7f25a0136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 -- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 msgr2=0x7f25a013fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 0x7f25a013fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 -- 172.21.15.185:0/2790664647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 msgr2=0x7f25a0136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 0x7f25a0136f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.123 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 -- 172.21.15.185:0/2790664647 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0142860 con 0x7f25a00b74f0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f2597fff700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 0x7f25a0136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259ed9d700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f2598009ea0 tx=0x7f2598018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f25967fc700 1 -- 172.21.15.185:0/2790664647 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259801f070 con 0x7f25a00b74f0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f25967fc700 1 -- 172.21.15.185:0/2790664647 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800be90 con 0x7f25a00b74f0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0140180 con 0x7f25a00b74f0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f25967fc700 1 -- 172.21.15.185:0/2790664647 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598006ea0 con 0x7f25a00b74f0 2020-11-16T22:23:09.124 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.130+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a0060d50 con 0x7f25a00b74f0 2020-11-16T22:23:09.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f25967fc700 1 -- 172.21.15.185:0/2790664647 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f259802b070 con 0x7f25a00b74f0 2020-11-16T22:23:09.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f25967fc700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 0x7f257c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.125 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f25967fc700 1 -- 172.21.15.185:0/2790664647 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25980714d0 con 0x7f25a00b74f0 2020-11-16T22:23:09.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f259e59c700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 0x7f257c057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f259e59c700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 0x7f257c057520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f25900011a0 tx=0x7f25900079e0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f25b3474740 1 Processor -- start 2020-11-16T22:23:09.126 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.134+0000 7f25b3474740 1 -- start start 2020-11-16T22:23:09.127 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b2eda0 0x1b2d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ba4b0 con 0x19107b0 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6e930 con 0x18a2bb0 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x192e6f0 con 0x1b2eda0 2020-11-16T22:23:09.128 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f259ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b2eda0 0x1b2d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35946/0 (socket says 172.21.15.185:35946) 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f259e59c700 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 -- 172.21.15.185:0/730990301 learned_addr learned my addr 172.21.15.185:0/730990301 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 -- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b2eda0 msgr2=0x1b2d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b2eda0 0x1b2d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 -- 172.21.15.185:0/730990301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 msgr2=0x1b2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.129 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 -- 172.21.15.185:0/730990301 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19295a0 con 0x18a2bb0 2020-11-16T22:23:09.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f2597fff700 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f25880018e0 tx=0x7f2588009040).ready entity=mon.1 client_cookie=c1cfa6454821ca1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25777fe700 1 -- 172.21.15.185:0/730990301 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800e7d0 con 0x18a2bb0 2020-11-16T22:23:09.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25777fe700 1 -- 172.21.15.185:0/730990301 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800e930 con 0x18a2bb0 2020-11-16T22:23:09.130 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25777fe700 1 -- 172.21.15.185:0/730990301 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800ecb0 con 0x18a2bb0 2020-11-16T22:23:09.131 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 msgr2=0x1b2ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f25880018e0 tx=0x7f2588009040).stop 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- 172.21.15.185:0/730990301 shutdown_connections 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1b2eda0 0x1b2d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 --2- 172.21.15.185:0/730990301 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18a2bb0 0x1b2ce90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- 172.21.15.185:0/730990301 >> 172.21.15.185:0/730990301 conn(0x1adee40 msgr2=0x1b2bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.132 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- 172.21.15.185:0/730990301 shutdown_connections 2020-11-16T22:23:09.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.138+0000 7f25b3474740 1 -- 172.21.15.185:0/730990301 wait complete. 2020-11-16T22:23:09.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- start start 2020-11-16T22:23:09.133 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 Processor -- start 2020-11-16T22:23:09.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18a2bb0 0x1b2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b2eda0 0x1b2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.135 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a5b6e0 con 0x18a2bb0 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a5b820 con 0x1b2eda0 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a5ca90 con 0x19107b0 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 10 client.0 ll_register_callbacks cb 0x1a51ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57630/0 (socket says 172.21.15.185:57630) 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 -- 172.21.15.185:0/2904463266 learned_addr learned my addr 172.21.15.185:0/2904463266 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:09.136 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f259ed9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b2eda0 0x1b2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f259e59c700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18a2bb0 0x1b2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 -- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b2eda0 msgr2=0x1b2f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b2eda0 0x1b2f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 -- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18a2bb0 msgr2=0x1b2d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18a2bb0 0x1b2d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 -- 172.21.15.185:0/2904463266 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1a4c280 con 0x19107b0 2020-11-16T22:23:09.137 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2597fff700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f2588007ed0 tx=0x7f2588009040).ready entity=mon.2 client_cookie=90e29db95839e448 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588007560 con 0x19107b0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2588014850 con 0x19107b0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1b0f6c0 con 0x19107b0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2588014c60 con 0x19107b0 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 10 client.7322 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:09.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x1881fd0 con 0x19107b0 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1b0f6c0 con 0x19107b0 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f2588006b40 con 0x19107b0 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 10 client.7322 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 10 client.7322 resolve_mds: resolved ID 'b' to GID 7821 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 20 client.7322 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b648a0 0x1b5d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 4 client.7322 mds_command: new command op to 7821 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:09.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 --> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x194cc50 con 0x1b648a0 2020-11-16T22:23:09.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.142+0000 7f259e59c700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b648a0 0x1b5d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f2574ff9700 10 client.7322 ms_handle_connect on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:09.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f259e59c700 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b648a0 0x1b5d0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=62f6c403e1a42144 server_cookie=c2c96a4c5fa0f642 in_seq=0 out_seq=0 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f2574ff9700 1 -- 172.21.15.185:0/2904463266 <== mds.2 v2:172.21.15.96:6832/2921285835 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x194cc50 con 0x1b648a0 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f2574ff9700 10 client.7322 handle_command_reply: tid=0 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 client.7322 shutdown 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 msgr2=0x1b73050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f2588007ed0 tx=0x7f2588009040).stop 2020-11-16T22:23:09.141 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 shutdown_connections 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b648a0 0x1b5d0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1b2eda0 0x1b2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f2574ff9700 0 client.7322 ms_handle_reset on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18a2bb0 0x1b2d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 --2- 172.21.15.185:0/2904463266 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x19107b0 0x1b73050 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 >> 172.21.15.185:0/2904463266 conn(0x1adee40 msgr2=0x1b60f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.142 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 shutdown_connections 2020-11-16T22:23:09.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 1 -- 172.21.15.185:0/2904463266 wait complete. 2020-11-16T22:23:09.143 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.146+0000 7f25b3474740 20 client.7322 trim_cache size 0 max 16384 2020-11-16T22:23:09.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.162+0000 7f25b3474740 1 Processor -- start 2020-11-16T22:23:09.154 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- start start 2020-11-16T22:23:09.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b61380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a33e40 con 0x19107b0 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b10200 con 0x1adee40 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b5d870 con 0x18a2bb0 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35960/0 (socket says 172.21.15.185:35960) 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57640/0 (socket says 172.21.15.185:57640) 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 -- 172.21.15.185:0/2858502568 learned_addr learned my addr 172.21.15.185:0/2858502568 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259ed9d700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b61380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 -- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 msgr2=0x1b60e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 -- 172.21.15.185:0/2858502568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 msgr2=0x1b61380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b61380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 -- 172.21.15.185:0/2858502568 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568008160 con 0x1adee40 2020-11-16T22:23:09.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259ed9d700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b61380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:09.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2597fff700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:09.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f259e59c700 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f2590023810 tx=0x7f2590017040).ready entity=mon.1 client_cookie=be4794d721de1d2a server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2577fff700 1 -- 172.21.15.185:0/2858502568 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259001a040 con 0x1adee40 2020-11-16T22:23:09.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2577fff700 1 -- 172.21.15.185:0/2858502568 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259001b040 con 0x1adee40 2020-11-16T22:23:09.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f2577fff700 1 -- 172.21.15.185:0/2858502568 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900230e0 con 0x1adee40 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 msgr2=0x1b71bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f2590023810 tx=0x7f2590017040).stop 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- 172.21.15.185:0/2858502568 shutdown_connections 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1adee40 0x1b71bb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x19107b0 0x1b61380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- 172.21.15.185:0/2858502568 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b60e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- 172.21.15.185:0/2858502568 >> 172.21.15.185:0/2858502568 conn(0x7f2568002320 msgr2=0x1b2e010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- 172.21.15.185:0/2858502568 shutdown_connections 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- 172.21.15.185:0/2858502568 wait complete. 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- start start 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 Processor -- start 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 0x1b79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b2eda0 0x1bc3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b28770 con 0x1b2eda0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bc9e40 con 0x19107b0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.166+0000 7f25b3474740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b5c500 con 0x18a2bb0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 10 client.0 ll_register_callbacks cb 0x1b28ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 0x1b79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2597fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 0x1b79220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35966/0 (socket says 172.21.15.185:35966) 2020-11-16T22:23:09.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57644/0 (socket says 172.21.15.185:57644) 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2597fff700 1 -- 172.21.15.185:0/811231785 learned_addr learned my addr 172.21.15.185:0/811231785 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259ed9d700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b2eda0 0x1bc3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 -- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 msgr2=0x1b79220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 0x1b79220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 -- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b2eda0 msgr2=0x1bc3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b2eda0 0x1bc3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 -- 172.21.15.185:0/811231785 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x18c0720 con 0x18a2bb0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f259e59c700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x1b2db10 tx=0x7f259000b000).ready entity=mon.2 client_cookie=75c77c35203ac760 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259001a040 con 0x18a2bb0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25900062d0 con 0x18a2bb0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258800e930 con 0x18a2bb0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25900159c0 con 0x18a2bb0 2020-11-16T22:23:09.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 10 client.7328 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x18cbe00 con 0x18a2bb0 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258800e930 con 0x18a2bb0 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f2590015b20 con 0x18a2bb0 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 10 client.7328 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 10 client.7328 resolve_mds: resolved ID 'b' to GID 7821 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 20 client.7328 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:09.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b79710 0x1bc3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:09.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 4 client.7328 mds_command: new command op to 7821 tid=0[{,",p,r,e,f,i,x,",:, ,",f,l,u,s,h, ,j,o,u,r,n,a,l,",}] 2020-11-16T22:23:09.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 --> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] -- command(tid 0: { " p r e f i x " : " f l u s h j o u r n a l " }) v1 -- 0x1953f50 con 0x1b79710 2020-11-16T22:23:09.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2597fff700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b79710 0x1bc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:09.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2597fff700 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b79710 0x1bc3700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=b313919cc72e3343 server_cookie=233f1d647c499401 in_seq=0 out_seq=0 2020-11-16T22:23:09.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.170+0000 7f2574ff9700 10 client.7328 ms_handle_connect on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:09.182 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.190+0000 7f2574ff9700 1 -- 172.21.15.185:0/811231785 <== mds.2 v2:172.21.15.96:6832/2921285835 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x1953f50 con 0x1b79710 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.190+0000 7f2574ff9700 10 client.7328 handle_command_reply: tid=0 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.190+0000 7f25b3474740 1 client.7328 shutdown 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 msgr2=0x1b7a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x1b2db10 tx=0x7f259000b000).stop 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 shutdown_connections 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1b2eda0 0x1bc3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.183 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18a2bb0 0x1b7a630 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x1b79710 0x1bc3700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 --2- 172.21.15.185:0/811231785 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x19107b0 0x1b79220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f2574ff9700 0 client.7328 ms_handle_reset on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 >> 172.21.15.185:0/811231785 conn(0x7f2568002320 msgr2=0x1b236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 shutdown_connections 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 1 -- 172.21.15.185:0/811231785 wait complete. 2020-11-16T22:23:09.184 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f25b3474740 20 client.7328 trim_cache size 0 max 16384 2020-11-16T22:23:09.185 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:09.185 INFO:teuthology.orchestra.run.smithi185.stdout: "message": "", 2020-11-16T22:23:09.185 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:23:09.185 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:09.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 msgr2=0x7f257c057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 0x7f257c057520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f25900011a0 tx=0x7f25900079e0).stop 2020-11-16T22:23:09.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 msgr2=0x7f25a0137450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:09.186 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f2598009ea0 tx=0x7f2598018000).stop 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 shutdown_connections 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f257c055060 0x7f257c057520 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f25a00b74f0 0x7f25a0137450 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f25a00670b0 0x7f25a013fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 --2- 172.21.15.185:0/2790664647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25a0068030 0x7f25a0136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 >> 172.21.15.185:0/2790664647 conn(0x7f25a0064b90 msgr2=0x7f25a0060130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:09.187 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 shutdown_connections 2020-11-16T22:23:09.188 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:09.194+0000 7f259f59e700 1 -- 172.21.15.185:0/2790664647 wait complete. 2020-11-16T22:23:09.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-16T22:23:09.200 INFO:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo find d1/d2/d3/) 2020-11-16T22:23:09.259 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/ 2020-11-16T22:23:09.260 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4 2020-11-16T22:23:09.260 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5 2020-11-16T22:23:09.260 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6 2020-11-16T22:23:09.260 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7 2020-11-16T22:23:09.260 INFO:teuthology.orchestra.run.smithi062.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2020-11-16T22:23:09.265 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:09.265 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:09.265 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:09.266 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:09.266 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2020-11-16T22:23:09.266 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:09.382 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627778 2020-11-16T22:23:09.572 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:09.572 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:09.573 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:09.573 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:09.573 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2020-11-16T22:23:09.574 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627779 2020-11-16T22:23:09.877 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:09.877 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:09.877 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:09.878 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:09.878 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2020-11-16T22:23:09.878 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:10.034 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627780 2020-11-16T22:23:10.225 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:10.226 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:10.226 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:10.226 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:10.226 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2020-11-16T22:23:10.227 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:10.337 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627781 2020-11-16T22:23:10.525 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:10.526 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:10.526 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:10.526 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:10.527 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2020-11-16T22:23:10.527 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:10.642 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627782 2020-11-16T22:23:10.829 INFO:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-16T22:23:10.830 INFO:teuthology.orchestra.run.smithi062:> import os 2020-11-16T22:23:10.830 INFO:teuthology.orchestra.run.smithi062:> import stat 2020-11-16T22:23:10.830 INFO:teuthology.orchestra.run.smithi062:> 2020-11-16T22:23:10.831 INFO:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2020-11-16T22:23:10.831 INFO:teuthology.orchestra.run.smithi062:> ' 2020-11-16T22:23:10.946 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627783 2020-11-16T22:23:10.971 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:23:10.977 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:23:10.979 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:23:11.136 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000002.00000000 parent 2020-11-16T22:23:11.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 Processor -- start 2020-11-16T22:23:11.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 -- start start 2020-11-16T22:23:11.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 0x557d4c4bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 0x557d4c42f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c42c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4c39e1f0 con 0x557d4c3f01e0 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4c36cbe0 con 0x557d4c424a80 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d4c3efce0 con 0x557d4c428620 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2696700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 0x557d4c42f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 0x557d4c4bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2696700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 0x557d4c42f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35980/0 (socket says 172.21.15.185:35980) 2020-11-16T22:23:11.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2696700 1 -- 172.21.15.185:0/4030805842 learned_addr learned my addr 172.21.15.185:0/4030805842 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:11.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb3698700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c42c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 -- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 msgr2=0x557d4c42c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c42c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.172 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 -- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 msgr2=0x557d4c42f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 0x557d4c42f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 -- 172.21.15.185:0/4030805842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4c3968a0 con 0x557d4c3f01e0 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.178+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 0x557d4c4bde80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fdc9c009ba0 tx=0x7fdc9c00c040).ready entity=mon.0 client_cookie=4de1c04f88ab9bf7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb1694700 1 -- 172.21.15.185:0/4030805842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00f040 con 0x557d4c3f01e0 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb1694700 1 -- 172.21.15.185:0/4030805842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c008b10 con 0x557d4c3f01e0 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb1694700 1 -- 172.21.15.185:0/4030805842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c005e50 con 0x557d4c3f01e0 2020-11-16T22:23:11.173 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- 172.21.15.185:0/4030805842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 msgr2=0x557d4c4bde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 0x557d4c4bde80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fdc9c009ba0 tx=0x7fdc9c00c040).stop 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- 172.21.15.185:0/4030805842 shutdown_connections 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c42c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c3f01e0 0x557d4c4bde80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/4030805842 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c424a80 0x557d4c42f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- 172.21.15.185:0/4030805842 >> 172.21.15.185:0/4030805842 conn(0x557d4c430be0 msgr2=0x557d4c4302c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:11.174 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- 172.21.15.185:0/4030805842 shutdown_connections 2020-11-16T22:23:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- 172.21.15.185:0/4030805842 wait complete. 2020-11-16T22:23:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 Processor -- start 2020-11-16T22:23:11.175 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- start start 2020-11-16T22:23:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 0x557d4c4cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.176 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c424a80 0x557d4c4269a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c4bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4c4bbfc0 con 0x557d4c424a80 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4c4ce8c0 con 0x557d4c3f01e0 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcc1891c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d4c4ca970 con 0x557d4c428620 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 0x557d4c4cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.177 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb3698700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c4bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 0x557d4c4cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:35990/0 (socket says 172.21.15.185:35990) 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 -- 172.21.15.185:0/2908783407 learned_addr learned my addr 172.21.15.185:0/2908783407 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2696700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c424a80 0x557d4c4269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 -- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 msgr2=0x557d4c4bc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x557d4c428620 0x557d4c4bc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 -- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c424a80 msgr2=0x557d4c4269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.178 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d4c424a80 0x557d4c4269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.182+0000 7fdcb2e97700 1 -- 172.21.15.185:0/2908783407 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4c392280 con 0x557d4c3f01e0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcb2e97700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 0x557d4c4cb0b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7fdc9c014d50 tx=0x7fdc9c018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 -- 172.21.15.185:0/2908783407 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00f070 con 0x557d4c3f01e0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 -- 172.21.15.185:0/2908783407 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c0135f0 con 0x557d4c3f01e0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d4c370620 con 0x557d4c3f01e0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 -- 172.21.15.185:0/2908783407 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0098e0 con 0x557d4c3f01e0 2020-11-16T22:23:11.179 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4c426e90 con 0x557d4c3f01e0 2020-11-16T22:23:11.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 -- 172.21.15.185:0/2908783407 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fdc9c025030 con 0x557d4c3f01e0 2020-11-16T22:23:11.180 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc98053ce0 0x7fdc980561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcab7fe700 1 -- 172.21.15.185:0/2908783407 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdc9c06e5d0 con 0x557d4c3f01e0 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcb2696700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc98053ce0 0x7fdc980561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x557d4c42b450 0x557d4c4d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- osd_op(unknown.0.0:1 15.2 15:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x557d4c4bb160 con 0x557d4c42b450 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcb2696700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc98053ce0 0x7fdc980561a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x557d4c4bc500 tx=0x7fdcac006980).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:11.181 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.186+0000 7fdcb3698700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x557d4c42b450 0x557d4c4d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.382 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.190+0000 7fdcb3698700 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x557d4c42b450 0x557d4c4d6e80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcb3698700 1 -- 172.21.15.185:0/2908783407 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v84'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdca4005420 con 0x557d4c42b450 2020-11-16T22:23:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x557d4c42b450 msgr2=0x557d4c4d6e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x557d4c42b450 0x557d4c4d6e80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc98053ce0 msgr2=0x7fdc980561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.384 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fdc98053ce0 0x7fdc980561a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x557d4c4bc500 tx=0x7fdcac006980).stop 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 msgr2=0x557d4c4cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 --2- 172.21.15.185:0/2908783407 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x557d4c3f01e0 0x557d4c4cb0b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7fdc9c014d50 tx=0x7fdc9c018000).stop 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcb3698700 1 -- 172.21.15.185:0/2908783407 reap_dead start 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 shutdown_connections 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 >> 172.21.15.185:0/2908783407 conn(0x557d4c430be0 msgr2=0x557d4c42dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:11.385 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 shutdown_connections 2020-11-16T22:23:11.386 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.194+0000 7fdcc1891c40 1 -- 172.21.15.185:0/2908783407 wait complete. 2020-11-16T22:23:11.386 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000003.00000000 parent 2020-11-16T22:23:11.459 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 Processor -- start 2020-11-16T22:23:11.460 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 -- start start 2020-11-16T22:23:11.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89d1620 0x5567e8a4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:11.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e89471f0 con 0x5567e89d1620 2020-11-16T22:23:11.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e8915be0 con 0x5567e89991e0 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567e8a60af0 con 0x5567e89cda80 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb67dc92700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb675491700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb67dc92700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36002/0 (socket says 172.21.15.185:36002) 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb67dc92700 1 -- 172.21.15.185:0/80283733 learned_addr learned my addr 172.21.15.185:0/80283733 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb675491700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57680/0 (socket says 172.21.15.185:57680) 2020-11-16T22:23:11.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.470+0000 7fb67e493700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89d1620 0x5567e8a4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:11.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 -- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 msgr2=0x5567e89d93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:11.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 -- 172.21.15.185:0/80283733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89d1620 msgr2=0x5567e8a4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:11.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89d1620 0x5567e8a4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 -- 172.21.15.185:0/80283733 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e893f8a0 con 0x5567e89cda80 2020-11-16T22:23:11.870 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb67dc92700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:11.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb675491700 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fb66800b500 tx=0x7fb6680017b0).ready entity=mon.2 client_cookie=fa07c06b3db8742e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:11.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb67cc90700 1 -- 172.21.15.185:0/80283733 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800d490 con 0x5567e89cda80 2020-11-16T22:23:11.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb67cc90700 1 -- 172.21.15.185:0/80283733 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800d5f0 con 0x5567e89cda80 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb67cc90700 1 -- 172.21.15.185:0/80283733 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800d990 con 0x5567e89cda80 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 msgr2=0x5567e8a4c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fb66800b500 tx=0x7fb6680017b0).stop 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- 172.21.15.185:0/80283733 shutdown_connections 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89991e0 0x5567e89d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89d1620 0x5567e8a4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/80283733 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89cda80 0x5567e8a4c3c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:11.876 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- 172.21.15.185:0/80283733 >> 172.21.15.185:0/80283733 conn(0x5567e89d9be0 msgr2=0x5567e8a667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- 172.21.15.185:0/80283733 shutdown_connections 2020-11-16T22:23:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- 172.21.15.185:0/80283733 wait complete. 2020-11-16T22:23:12.078 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 Processor -- start 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.474+0000 7fb68c68cc40 1 -- start start 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 0x5567e8a66960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89cda80 0x5567e8a64da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89d1620 0x5567e8a65290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e8a4c510 con 0x5567e89cda80 2020-11-16T22:23:12.079 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e8a616b0 con 0x5567e89d1620 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567e8a61950 con 0x5567e89991e0 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 0x5567e8a66960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 0x5567e8a66960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57684/0 (socket says 172.21.15.185:57684) 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 -- 172.21.15.185:0/921859715 learned_addr learned my addr 172.21.15.185:0/921859715 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67e493700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89d1620 0x5567e8a65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb675491700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89cda80 0x5567e8a64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.080 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 -- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89d1620 msgr2=0x5567e8a65290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x5567e89d1620 0x5567e8a65290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:12.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 -- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89cda80 msgr2=0x5567e8a64da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.282 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567e89cda80 0x5567e8a64da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 -- 172.21.15.185:0/921859715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e893b280 con 0x5567e89991e0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb67dc92700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 0x5567e8a66960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fb670008fd0 tx=0x7fb67000c040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb676ffd700 1 -- 172.21.15.185:0/921859715 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x5567e89991e0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb676ffd700 1 -- 172.21.15.185:0/921859715 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670010c70 con 0x5567e89991e0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5567e8919620 con 0x5567e89991e0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb676ffd700 1 -- 172.21.15.185:0/921859715 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670015470 con 0x5567e89991e0 2020-11-16T22:23:12.283 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.478+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567e8a63870 con 0x5567e89991e0 2020-11-16T22:23:12.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb676ffd700 1 -- 172.21.15.185:0/921859715 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb6700158a0 con 0x5567e89991e0 2020-11-16T22:23:12.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb676ffd700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb660053ce0 0x7fb6600561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb676ffd700 1 -- 172.21.15.185:0/921859715 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb67006db30 con 0x5567e89991e0 2020-11-16T22:23:12.284 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb675491700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb660053ce0 0x7fb6600561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5567e8a6ad30 0x5567e8a80110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 --> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] -- osd_op(unknown.0.0:1 15.2 15:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x5567e8a80600 con 0x5567e8a6ad30 2020-11-16T22:23:12.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb675491700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb660053ce0 0x7fb6600561a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x5567e89d8f10 tx=0x7fb668014000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:12.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb67e493700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5567e8a6ad30 0x5567e8a80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.482+0000 7fb67e493700 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5567e8a6ad30 0x5567e8a80110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.486+0000 7fb67e493700 1 -- 172.21.15.185:0/921859715 <== osd.5 v2:172.21.15.96:6809/15269 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v84'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb6780057a0 con 0x5567e8a6ad30 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5567e8a6ad30 msgr2=0x5567e8a80110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.96:6809/15269,v1:172.21.15.96:6812/15269] conn(0x5567e8a6ad30 0x5567e8a80110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb660053ce0 msgr2=0x7fb6600561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb660053ce0 0x7fb6600561a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x5567e89d8f10 tx=0x7fb668014000).stop 2020-11-16T22:23:12.487 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 msgr2=0x5567e8a66960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 --2- 172.21.15.185:0/921859715 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x5567e89991e0 0x5567e8a66960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fb670008fd0 tx=0x7fb67000c040).stop 2020-11-16T22:23:12.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb67e493700 1 -- 172.21.15.185:0/921859715 reap_dead start 2020-11-16T22:23:12.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 shutdown_connections 2020-11-16T22:23:12.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 >> 172.21.15.185:0/921859715 conn(0x5567e89d9be0 msgr2=0x5567e8a6c660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:12.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 shutdown_connections 2020-11-16T22:23:12.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:11.490+0000 7fb68c68cc40 1 -- 172.21.15.185:0/921859715 wait complete. 2020-11-16T22:23:12.691 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000004.00000000 parent 2020-11-16T22:23:12.724 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.734+0000 7f82804dec40 1 Processor -- start 2020-11-16T22:23:12.725 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.734+0000 7f82804dec40 1 -- start start 2020-11-16T22:23:12.726 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a760a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a75a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:12.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565a613be0 con 0x55565a6ce850 2020-11-16T22:23:12.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565a761050 con 0x55565a75ba80 2020-11-16T22:23:12.727 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55565a760570 con 0x55565a6d3440 2020-11-16T22:23:12.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82722e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a75a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57702/0 (socket says 172.21.15.185:57702) 2020-11-16T22:23:12.728 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82722e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a75a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36028/0 (socket says 172.21.15.185:36028) 2020-11-16T22:23:12.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 -- 172.21.15.185:0/563528025 learned_addr learned my addr 172.21.15.185:0/563528025 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:12.930 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f8271ae4700 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a760a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:12.933 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 -- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 msgr2=0x55565a75a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:12.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a75a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:12.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 -- 172.21.15.185:0/563528025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 msgr2=0x55565a760a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:12.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a760a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:12.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 -- 172.21.15.185:0/563528025 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55565a63c480 con 0x55565a6d3440 2020-11-16T22:23:12.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82712e3700 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f825c009af0 tx=0x7f825c00c040).ready entity=mon.2 client_cookie=2386f6b9d544e5dc server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:12.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f826bfff700 1 -- 172.21.15.185:0/563528025 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00f040 con 0x55565a6d3440 2020-11-16T22:23:12.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f826bfff700 1 -- 172.21.15.185:0/563528025 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c005da0 con 0x55565a6d3440 2020-11-16T22:23:12.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f826bfff700 1 -- 172.21.15.185:0/563528025 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c008c60 con 0x55565a6d3440 2020-11-16T22:23:12.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 msgr2=0x55565a6d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:12.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f825c009af0 tx=0x7f825c00c040).stop 2020-11-16T22:23:13.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- 172.21.15.185:0/563528025 shutdown_connections 2020-11-16T22:23:13.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a75a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a760a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 --2- 172.21.15.185:0/563528025 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a6d2320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.138 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- 172.21.15.185:0/563528025 >> 172.21.15.185:0/563528025 conn(0x55565a6d4670 msgr2=0x55565a6d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- 172.21.15.185:0/563528025 shutdown_connections 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.738+0000 7f82804dec40 1 -- 172.21.15.185:0/563528025 wait complete. 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 Processor -- start 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- start start 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a772d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a75c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.139 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a775d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565a7731f0 con 0x55565a6ce850 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565a75ac70 con 0x55565a75ba80 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55565a75c640 con 0x55565a6d3440 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82722e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a775d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a75c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.140 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82722e5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a775d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36030/0 (socket says 172.21.15.185:36030) 2020-11-16T22:23:13.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a75c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57706/0 (socket says 172.21.15.185:57706) 2020-11-16T22:23:13.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82722e5700 1 -- 172.21.15.185:0/157166923 learned_addr learned my addr 172.21.15.185:0/157166923 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:13.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f8271ae4700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a772d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 -- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 msgr2=0x55565a775d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.342 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55565a75ba80 0x55565a775d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 -- 172.21.15.185:0/157166923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 msgr2=0x55565a772d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55565a6ce850 0x55565a772d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 -- 172.21.15.185:0/157166923 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55565a63d8a0 con 0x55565a6d3440 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82712e3700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a75c150 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f825c009af0 tx=0x7f825c018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f826a7fc700 1 -- 172.21.15.185:0/157166923 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00f040 con 0x55565a6d3440 2020-11-16T22:23:13.343 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55565a639280 con 0x55565a6d3440 2020-11-16T22:23:13.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f826a7fc700 1 -- 172.21.15.185:0/157166923 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c014d30 con 0x55565a6d3440 2020-11-16T22:23:13.344 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f826a7fc700 1 -- 172.21.15.185:0/157166923 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c014ea0 con 0x55565a6d3440 2020-11-16T22:23:13.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.742+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55565a617620 con 0x55565a6d3440 2020-11-16T22:23:13.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f826a7fc700 1 -- 172.21.15.185:0/157166923 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f825c01e8c0 con 0x55565a6d3440 2020-11-16T22:23:13.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f826a7fc700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8254053ce0 0x7f82540561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f826a7fc700 1 -- 172.21.15.185:0/157166923 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f825c06e520 con 0x55565a6d3440 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f8271ae4700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8254053ce0 0x7f82540561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f82804dec40 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55565a759740 0x55565a759b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 15.5 15:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x55565a77ee30 con 0x55565a759740 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f82722e5700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55565a759740 0x55565a759b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f82722e5700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55565a759740 0x55565a759b60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:13.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.746+0000 7f8271ae4700 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8254053ce0 0x7f82540561a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f826c0082e0 tx=0x7f826c012000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:13.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82722e5700 1 -- 172.21.15.185:0/157166923 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v84'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8264012040 con 0x55565a759740 2020-11-16T22:23:13.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55565a759740 msgr2=0x55565a759b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55565a759740 0x55565a759b60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8254053ce0 msgr2=0x7f82540561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f8254053ce0 0x7f82540561a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f826c0082e0 tx=0x7f826c012000).stop 2020-11-16T22:23:13.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 msgr2=0x55565a75c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 --2- 172.21.15.185:0/157166923 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55565a6d3440 0x55565a75c150 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f825c009af0 tx=0x7f825c018000).stop 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82722e5700 1 -- 172.21.15.185:0/157166923 reap_dead start 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 shutdown_connections 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 >> 172.21.15.185:0/157166923 conn(0x55565a6d4670 msgr2=0x55565a6d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 shutdown_connections 2020-11-16T22:23:13.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:12.750+0000 7f82804dec40 1 -- 172.21.15.185:0/157166923 wait complete. 2020-11-16T22:23:13.752 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000005.00000000 parent 2020-11-16T22:23:13.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.794+0000 7f6ff017ec40 1 Processor -- start 2020-11-16T22:23:13.785 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.794+0000 7f6ff017ec40 1 -- start start 2020-11-16T22:23:13.787 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d2a80 0x563bcd151330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcd04c1f0 con 0x563bcd0d2a80 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcd01abe0 con 0x563bcd09e1e0 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bcd0d7320 con 0x563bcd0d6620 2020-11-16T22:23:13.788 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1f85700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1f85700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57726/0 (socket says 172.21.15.185:57726) 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36046/0 (socket says 172.21.15.185:36046) 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1f85700 1 -- 172.21.15.185:0/1610194764 learned_addr learned my addr 172.21.15.185:0/1610194764 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe0f83700 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d2a80 0x563bcd151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:13.789 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 -- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 msgr2=0x563bcd151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:13.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 -- 172.21.15.185:0/1610194764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d2a80 msgr2=0x563bcd151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:13.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d2a80 0x563bcd151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 -- 172.21.15.185:0/1610194764 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcd0448a0 con 0x563bcd09e1e0 2020-11-16T22:23:13.790 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1f85700 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:13.791 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6fcc009b70 tx=0x7f6fcc00c040).ready entity=mon.1 client_cookie=89fa440b732be6e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:13.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fdb7fe700 1 -- 172.21.15.185:0/1610194764 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00f040 con 0x563bcd09e1e0 2020-11-16T22:23:13.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fdb7fe700 1 -- 172.21.15.185:0/1610194764 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc008b10 con 0x563bcd09e1e0 2020-11-16T22:23:13.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6fdb7fe700 1 -- 172.21.15.185:0/1610194764 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc005e50 con 0x563bcd09e1e0 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 msgr2=0x563bcd0dad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6fcc009b70 tx=0x7f6fcc00c040).stop 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1610194764 shutdown_connections 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd0dad50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d6620 0x563bcd151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1610194764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d2a80 0x563bcd151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:13.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1610194764 >> 172.21.15.185:0/1610194764 conn(0x563bcd0debe0 msgr2=0x563bcd0d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:13.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1610194764 shutdown_connections 2020-11-16T22:23:13.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.798+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1610194764 wait complete. 2020-11-16T22:23:13.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 Processor -- start 2020-11-16T22:23:13.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 -- start start 2020-11-16T22:23:13.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 0x563bcd0d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d6620 0x563bcd149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcd14c2a0 con 0x563bcd0d6620 2020-11-16T22:23:14.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcd14ffe0 con 0x563bcd09e1e0 2020-11-16T22:23:14.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6ff017ec40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bcd150240 con 0x563bcd0d2a80 2020-11-16T22:23:14.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe1784700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe0f83700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 0x563bcd0d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe1784700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd14bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36052/0 (socket says 172.21.15.185:36052) 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe1784700 1 -- 172.21.15.185:0/1965685332 learned_addr learned my addr 172.21.15.185:0/1965685332 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe0f83700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 0x563bcd0d82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57730/0 (socket says 172.21.15.185:57730) 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.802+0000 7f6fe1f85700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d6620 0x563bcd149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.203 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 -- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 msgr2=0x563bcd14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:14.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:14.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 -- 172.21.15.185:0/1965685332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d6620 msgr2=0x563bcd149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:14.204 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bcd0d6620 0x563bcd149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:14.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 -- 172.21.15.185:0/1965685332 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcd040280 con 0x563bcd0d2a80 2020-11-16T22:23:14.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x563bcd09e1e0 0x563bcd14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:14.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe0f83700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 0x563bcd0d82e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x563bcd16b4f0 tx=0x7f6fd40017f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:14.406 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 -- 172.21.15.185:0/1965685332 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4001d50 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 -- 172.21.15.185:0/1965685332 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd4002030 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563bcd01e620 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 -- 172.21.15.185:0/1965685332 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40023b0 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bcd150780 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 -- 172.21.15.185:0/1965685332 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f6fd4019030 con 0x563bcd0d2a80 2020-11-16T22:23:14.407 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fc4053ce0 0x7f6fc40561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fd9ffb700 1 -- 172.21.15.185:0/1965685332 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fd400b070 con 0x563bcd0d2a80 2020-11-16T22:23:14.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.806+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fc4053ce0 0x7f6fc40561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.408 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.810+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x563bcd17c930 0x563bcd17cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.810+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 --> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] -- osd_op(unknown.0.0:1 15.1 15:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x563bcd185570 con 0x563bcd17c930 2020-11-16T22:23:14.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.810+0000 7f6fe1784700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fc4053ce0 0x7f6fc40561a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x563bcd16b000 tx=0x7f6fcc018000).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:14.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.810+0000 7f6fe1f85700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x563bcd17c930 0x563bcd17cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.810+0000 7f6fe1f85700 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x563bcd17c930 0x563bcd17cd30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:14.610 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6fe1f85700 1 -- 172.21.15.185:0/1965685332 <== osd.1 v2:172.21.15.185:6812/15205 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v84'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6fdc00a560 con 0x563bcd17c930 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x563bcd17c930 msgr2=0x563bcd17cd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6812/15205,v1:172.21.15.185:6814/15205] conn(0x563bcd17c930 0x563bcd17cd30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fc4053ce0 msgr2=0x7f6fc40561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f6fc4053ce0 0x7f6fc40561a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x563bcd16b000 tx=0x7f6fcc018000).stop 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 msgr2=0x563bcd0d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:14.611 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 --2- 172.21.15.185:0/1965685332 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x563bcd0d2a80 0x563bcd0d82e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x563bcd16b4f0 tx=0x7f6fd40017f0).stop 2020-11-16T22:23:14.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6fe1f85700 1 -- 172.21.15.185:0/1965685332 reap_dead start 2020-11-16T22:23:14.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 shutdown_connections 2020-11-16T22:23:14.612 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 >> 172.21.15.185:0/1965685332 conn(0x563bcd0debe0 msgr2=0x563bcd0dad50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:14.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 shutdown_connections 2020-11-16T22:23:14.814 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:13.814+0000 7f6ff017ec40 1 -- 172.21.15.185:0/1965685332 wait complete. 2020-11-16T22:23:14.814 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000006.00000000 parent 2020-11-16T22:23:14.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 Processor -- start 2020-11-16T22:23:14.847 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 -- start start 2020-11-16T22:23:14.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.848 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2885ae60 0x55db288fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db287d41f0 con 0x55db2885ae60 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db287a2be0 con 0x55db288261e0 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db288f45d0 con 0x55db2885aa80 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.849 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d37fb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36068/0 (socket says 172.21.15.185:36068) 2020-11-16T22:23:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/2831657983 learned_addr learned my addr 172.21.15.185:0/2831657983 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:14.850 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d47fd700 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2885ae60 0x55db288fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:14.851 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 msgr2=0x55db288d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:15.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/2831657983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2885ae60 msgr2=0x55db288fcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:15.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2885ae60 0x55db288fcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.054 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/2831657983 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db287cc8a0 con 0x55db288261e0 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fc4c4001e70 tx=0x7fc4c40016f0).ready entity=mon.1 client_cookie=454593e7a0cd8512 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d27f9700 1 -- 172.21.15.185:0/2831657983 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400e430 con 0x55db288261e0 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d27f9700 1 -- 172.21.15.185:0/2831657983 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400e590 con 0x55db288261e0 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.858+0000 7fc4d27f9700 1 -- 172.21.15.185:0/2831657983 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400e910 con 0x55db288261e0 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 msgr2=0x55db288d93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.055 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fc4c4001e70 tx=0x7fc4c40016f0).stop 2020-11-16T22:23:15.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/2831657983 shutdown_connections 2020-11-16T22:23:15.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db288261e0 0x55db288d93c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db2885ae60 0x55db288fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.056 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/2831657983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/2831657983 >> 172.21.15.185:0/2831657983 conn(0x55db28866be0 msgr2=0x55db28865f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:15.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/2831657983 shutdown_connections 2020-11-16T22:23:15.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/2831657983 wait complete. 2020-11-16T22:23:15.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 Processor -- start 2020-11-16T22:23:15.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.862+0000 7fc4e29f6c40 1 -- start start 2020-11-16T22:23:15.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db288261e0 0x55db288d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.262 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 0x55db288d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db288ed2e0 con 0x55db288261e0 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db288ed580 con 0x55db2885ae60 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db288d6a80 con 0x55db2885aa80 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d47fd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 0x55db288d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d37fb700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.263 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db288261e0 0x55db288d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d47fd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 0x55db288d8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36076/0 (socket says 172.21.15.185:36076) 2020-11-16T22:23:15.264 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d47fd700 1 -- 172.21.15.185:0/1470089895 learned_addr learned my addr 172.21.15.185:0/1470089895 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:15.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 msgr2=0x55db288d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 msgr2=0x55db288d8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 0x55db288d8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 -- 172.21.15.185:0/1470089895 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db287c8280 con 0x55db288261e0 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d47fd700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55db2885ae60 0x55db288d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d37fb700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55db2885aa80 0x55db288d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d3ffc700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db288261e0 0x55db288d57d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc4c4011000 tx=0x7fc4c40016f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:15.467 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d0ff6700 1 -- 172.21.15.185:0/1470089895 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401a830 con 0x55db288261e0 2020-11-16T22:23:15.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d0ff6700 1 -- 172.21.15.185:0/1470089895 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c401a990 con 0x55db288261e0 2020-11-16T22:23:15.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55db287a6620 con 0x55db288261e0 2020-11-16T22:23:15.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4d0ff6700 1 -- 172.21.15.185:0/1470089895 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401ad10 con 0x55db288261e0 2020-11-16T22:23:15.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.866+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db288d5cc0 con 0x55db288261e0 2020-11-16T22:23:15.468 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d0ff6700 1 -- 172.21.15.185:0/1470089895 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc4c401b860 con 0x55db288261e0 2020-11-16T22:23:15.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d0ff6700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4c0053ce0 0x7fc4c00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d37fb700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4c0053ce0 0x7fc4c00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d0ff6700 1 -- 172.21.15.185:0/1470089895 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4c406ef30 con 0x55db288261e0 2020-11-16T22:23:15.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55db288eb470 0x55db288d4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 15.5 15:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x55db2890d390 con 0x55db288eb470 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d37fb700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4c0053ce0 0x7fc4c00561a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x55db288f11f0 tx=0x7fc4c800c040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d47fd700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55db288eb470 0x55db288d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.870+0000 7fc4d47fd700 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55db288eb470 0x55db288d4910 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4d47fd700 1 -- 172.21.15.185:0/1470089895 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v84'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc4cc00a500 con 0x55db288eb470 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55db288eb470 msgr2=0x55db288d4910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55db288eb470 0x55db288d4910 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:15.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4c0053ce0 msgr2=0x7fc4c00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc4c0053ce0 0x7fc4c00561a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x55db288f11f0 tx=0x7fc4c800c040).stop 2020-11-16T22:23:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db288261e0 msgr2=0x55db288d57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 --2- 172.21.15.185:0/1470089895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55db288261e0 0x55db288d57d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc4c4011000 tx=0x7fc4c40016f0).stop 2020-11-16T22:23:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4d47fd700 1 -- 172.21.15.185:0/1470089895 reap_dead start 2020-11-16T22:23:15.874 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 shutdown_connections 2020-11-16T22:23:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 >> 172.21.15.185:0/1470089895 conn(0x55db28866be0 msgr2=0x55db288d9040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 shutdown_connections 2020-11-16T22:23:15.875 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:14.874+0000 7fc4e29f6c40 1 -- 172.21.15.185:0/1470089895 wait complete. 2020-11-16T22:23:15.875 INFO:teuthology.orchestra.run.smithi185:> rados -p cephfs_metadata rmxattr 10000000007.00000000 parent 2020-11-16T22:23:15.907 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 Processor -- start 2020-11-16T22:23:15.908 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 -- start start 2020-11-16T22:23:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986fc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a398662e60 0x55a3986de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:15.909 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3985dc1f0 con 0x55a398662e60 2020-11-16T22:23:15.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3985aabe0 con 0x55a39862e1e0 2020-11-16T22:23:15.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a398666420 con 0x55a398662a80 2020-11-16T22:23:15.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:15.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36092/0 (socket says 172.21.15.185:36092) 2020-11-16T22:23:16.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 -- 172.21.15.185:0/3434101974 learned_addr learned my addr 172.21.15.185:0/3434101974 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:16.114 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d02bc700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.115 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d12be700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a398662e60 0x55a3986de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 -- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 msgr2=0x55a3986fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 -- 172.21.15.185:0/3434101974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a398662e60 msgr2=0x55a3986de1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a398662e60 0x55a3986de1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 -- 172.21.15.185:0/3434101974 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3985d48a0 con 0x55a39862e1e0 2020-11-16T22:23:16.119 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d02bc700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:16.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.918+0000 7f21d0abd700 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f21c000b640 tx=0x7f21c0001700).ready entity=mon.1 client_cookie=c117d06300883c2c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:16.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cf2ba700 1 -- 172.21.15.185:0/3434101974 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000e430 con 0x55a39862e1e0 2020-11-16T22:23:16.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cf2ba700 1 -- 172.21.15.185:0/3434101974 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000e590 con 0x55a39862e1e0 2020-11-16T22:23:16.120 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cf2ba700 1 -- 172.21.15.185:0/3434101974 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000e910 con 0x55a39862e1e0 2020-11-16T22:23:16.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 msgr2=0x55a3987046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f21c000b640 tx=0x7f21c0001700).stop 2020-11-16T22:23:16.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/3434101974 shutdown_connections 2020-11-16T22:23:16.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a39862e1e0 0x55a3987046f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.322 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a398662e60 0x55a3986de1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/3434101974 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/3434101974 >> 172.21.15.185:0/3434101974 conn(0x55a39866ebe0 msgr2=0x55a39866df50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/3434101974 shutdown_connections 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/3434101974 wait complete. 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 Processor -- start 2020-11-16T22:23:16.323 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- start start 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a39862e1e0 0x55a39866de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a398662e60 0x55a3986f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a398703790 con 0x55a39862e1e0 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a398703a30 con 0x55a398662e60 2020-11-16T22:23:16.324 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a398703cd0 con 0x55a398662a80 2020-11-16T22:23:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57774/0 (socket says 172.21.15.185:57774) 2020-11-16T22:23:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 -- 172.21.15.185:0/961226042 learned_addr learned my addr 172.21.15.185:0/961226042 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d12be700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a398662e60 0x55a3986f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.526 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d0abd700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a39862e1e0 0x55a39866de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 -- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a398662e60 msgr2=0x55a3986f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x55a398662e60 0x55a3986f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 -- 172.21.15.185:0/961226042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a39862e1e0 msgr2=0x55a39866de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a39862e1e0 0x55a39866de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 -- 172.21.15.185:0/961226042 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3985d0280 con 0x55a398662a80 2020-11-16T22:23:16.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d0abd700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a39862e1e0 0x55a39866de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:16.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21d02bc700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986f85d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f21b8008e50 tx=0x7f21b800c040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:16.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cdab7700 1 -- 172.21.15.185:0/961226042 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800f040 con 0x55a398662a80 2020-11-16T22:23:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cdab7700 1 -- 172.21.15.185:0/961226042 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8010cd0 con 0x55a398662a80 2020-11-16T22:23:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21cdab7700 1 -- 172.21.15.185:0/961226042 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8015680 con 0x55a398662a80 2020-11-16T22:23:16.730 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3985ae620 con 0x55a398662a80 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.922+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a398704210 con 0x55a398662a80 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21cdab7700 1 -- 172.21.15.185:0/961226042 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f21b8015ab0 con 0x55a398662a80 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21cdab7700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f21b0053ce0 0x7f21b00561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21cdab7700 1 -- 172.21.15.185:0/961226042 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f21b806d9f0 con 0x55a398662a80 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21d0abd700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f21b0053ce0 0x7f21b00561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.731 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55a3986fc300 0x55a3986d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:16.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 --> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] -- osd_op(unknown.0.0:1 15.5 15:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e84) v8 -- 0x55a3986f8fb0 con 0x55a3986fc300 2020-11-16T22:23:16.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21d12be700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55a3986fc300 0x55a3986d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:16.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.926+0000 7f21d0abd700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f21b0053ce0 0x7f21b00561a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f21c00016d0 tx=0x7f21c0001700).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:16.732 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21d12be700 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55a3986fc300 0x55a3986d78c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21d12be700 1 -- 172.21.15.185:0/961226042 <== osd.3 v2:172.21.15.185:6824/15203 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v84'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f21c800a500 con 0x55a3986fc300 2020-11-16T22:23:16.934 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55a3986fc300 msgr2=0x55a3986d78c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6824/15203,v1:172.21.15.185:6825/15203] conn(0x55a3986fc300 0x55a3986d78c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f21b0053ce0 msgr2=0x7f21b00561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f21b0053ce0 0x7f21b00561a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f21c00016d0 tx=0x7f21c0001700).stop 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 msgr2=0x55a3986f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 --2- 172.21.15.185:0/961226042 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x55a398662a80 0x55a3986f85d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f21b8008e50 tx=0x7f21b800c040).stop 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21d12be700 1 -- 172.21.15.185:0/961226042 reap_dead start 2020-11-16T22:23:16.935 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 shutdown_connections 2020-11-16T22:23:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 >> 172.21.15.185:0/961226042 conn(0x55a39866ebe0 msgr2=0x55a3987046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 shutdown_connections 2020-11-16T22:23:16.936 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:15.930+0000 7f21df4b7c40 1 -- 172.21.15.185:0/961226042 wait complete. 2020-11-16T22:23:16.936 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:17.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 Processor -- start 2020-11-16T22:23:17.068 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 -- start start 2020-11-16T22:23:17.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb830063050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8300b1540 0x7fb8300b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8300b1e90 0x7fb830143e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.070 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb830096a20 con 0x7fb8300b1540 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb830068460 con 0x7fb8300b1e90 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb834f08700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300964e0 con 0x7fb830062c70 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8300b1e90 0x7fb830143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb830063050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb8300b1e90 0x7fb830143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36110/0 (socket says 172.21.15.185:36110) 2020-11-16T22:23:17.071 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 -- 172.21.15.185:0/293019707 learned_addr learned my addr 172.21.15.185:0/293019707 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82dd9b700 1 --2- 172.21.15.185:0/293019707 >> v2:172.21.15.96:3300/0 conn(0x7fb8300b1540 0x7fb8300b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 -- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 msgr2=0x7fb830063050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 --2- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb830063050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 -- 172.21.15.185:0/293019707 >> v2:172.21.15.96:3300/0 conn(0x7fb8300b1540 msgr2=0x7fb8300b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 --2- 172.21.15.185:0/293019707 >> v2:172.21.15.96:3300/0 conn(0x7fb8300b1540 0x7fb8300b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.072 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.078+0000 7fb82ed9d700 1 -- 172.21.15.185:0/293019707 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830096780 con 0x7fb8300b1e90 2020-11-16T22:23:17.073 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb82ed9d700 1 --2- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb830143e60 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fb8300b2780 tx=0x7fb828006eb0).ready entity=mon.1 client_cookie=e2b294166969cb07 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:17.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb82d59a700 1 -- 172.21.15.185:0/293019707 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828009720 con 0x7fb8300b1e90 2020-11-16T22:23:17.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb82d59a700 1 -- 172.21.15.185:0/293019707 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828009880 con 0x7fb8300b1e90 2020-11-16T22:23:17.274 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb82d59a700 1 -- 172.21.15.185:0/293019707 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828009c00 con 0x7fb8300b1e90 2020-11-16T22:23:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 msgr2=0x7fb830143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 --2- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb830143e60 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fb8300b2780 tx=0x7fb828006eb0).stop 2020-11-16T22:23:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- 172.21.15.185:0/293019707 shutdown_connections 2020-11-16T22:23:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 --2- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb830143e60 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.277 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 --2- 172.21.15.185:0/293019707 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb830063050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 --2- 172.21.15.185:0/293019707 >> v2:172.21.15.96:3300/0 conn(0x7fb8300b1540 0x7fb8300b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- 172.21.15.185:0/293019707 >> 172.21.15.185:0/293019707 conn(0x7fb8300604a0 msgr2=0x7fb8300608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- 172.21.15.185:0/293019707 shutdown_connections 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- 172.21.15.185:0/293019707 wait complete. 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 Processor -- start 2020-11-16T22:23:17.278 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.082+0000 7fb834f08700 1 -- start start 2020-11-16T22:23:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.279 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8300b1540 0x7fb83013c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb83013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83013e170 con 0x7fb8300b1540 2020-11-16T22:23:17.482 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83013d990 con 0x7fb8300b1e90 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83013dc30 con 0x7fb830062c70 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57788/0 (socket says 172.21.15.185:57788) 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 -- 172.21.15.185:0/3304450613 learned_addr learned my addr 172.21.15.185:0/3304450613 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82ed9d700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb83013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.483 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82dd9b700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8300b1540 0x7fb83013c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 -- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 msgr2=0x7fb83013cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb83013cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 -- 172.21.15.185:0/3304450613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8300b1540 msgr2=0x7fb83013c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8300b1540 0x7fb83013c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 -- 172.21.15.185:0/3304450613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83013ded0 con 0x7fb830062c70 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb82e59c700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fb82000ed00 tx=0x7fb8200091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:17.484 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820009560 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000cb00 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8300636e0 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000ce80 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.086+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb830143550 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb820022070 con 0x7fb830062c70 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb81e7fc700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 0x7fb804057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.485 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83005d4e0 con 0x7fb830062c70 2020-11-16T22:23:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb82dd9b700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 0x7fb804057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb82006e400 con 0x7fb830062c70 2020-11-16T22:23:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.090+0000 7fb82dd9b700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 0x7fb804057520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fb83014a3b0 tx=0x7fb824008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:17.486 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.098+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fb82005ee60 con 0x7fb830062c70 2020-11-16T22:23:17.488 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.298+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb830144cc0 con 0x7fb830062c70 2020-11-16T22:23:17.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.298+0000 7fb81e7fc700 1 -- 172.21.15.185:0/3304450613 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+2528 (secure 0 0 0) 0x7fb8200169f0 con 0x7fb830062c70 2020-11-16T22:23:17.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 msgr2=0x7fb804057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 0x7fb804057520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fb83014a3b0 tx=0x7fb824008040).stop 2020-11-16T22:23:17.690 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 msgr2=0x7fb8300631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fb82000ed00 tx=0x7fb8200091d0).stop 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 shutdown_connections 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb804055060 0x7fb804057520 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb8300b1e90 0x7fb83013cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb830062c70 0x7fb8300631f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 --2- 172.21.15.185:0/3304450613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8300b1540 0x7fb83013c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.691 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 >> 172.21.15.185:0/3304450613 conn(0x7fb8300604a0 msgr2=0x7fb830141280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:17.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 shutdown_connections 2020-11-16T22:23:17.692 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.302+0000 7fb834f08700 1 -- 172.21.15.185:0/3304450613 wait complete. 2020-11-16T22:23:17.692 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 109 2020-11-16T22:23:17.692 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:17.692 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:05.291578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:17.693 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 recursive force 2020-11-16T22:23:17.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 Processor -- start 2020-11-16T22:23:17.831 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 -- start start 2020-11-16T22:23:17.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4f40b7550 0x7fa4f4148160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f4144170 con 0x7fa4f40b7170 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f4144f80 con 0x7fa4f40b7550 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40ad390 con 0x7fa4f40acee0 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57800/0 (socket says 172.21.15.185:57800) 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 -- 172.21.15.185:0/644842662 learned_addr learned my addr 172.21.15.185:0/644842662 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:17.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/644842662 >> v2:172.21.15.96:3300/0 conn(0x7fa4f40b7550 0x7fa4f4148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 -- 172.21.15.185:0/644842662 >> v2:172.21.15.96:3300/0 conn(0x7fa4f40b7550 msgr2=0x7fa4f4148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/644842662 >> v2:172.21.15.96:3300/0 conn(0x7fa4f40b7550 0x7fa4f4148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 -- 172.21.15.185:0/644842662 >> v1:172.21.15.96:6789/0 conn(0x7fa4f40b7170 legacy=0x7fa4f4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 -- 172.21.15.185:0/644842662 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40b6ed0 con 0x7fa4f40acee0 2020-11-16T22:23:17.835 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/644842662 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fa4ec009d50 tx=0x7fa4ec009290).ready entity=mon.2 client_cookie=71507079ead0adbe server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:17.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f1ffb700 1 -- 172.21.15.185:0/644842662 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00bbc0 con 0x7fa4f40acee0 2020-11-16T22:23:17.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f1ffb700 1 -- 172.21.15.185:0/644842662 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00bd20 con 0x7fa4f40acee0 2020-11-16T22:23:17.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.842+0000 7fa4f1ffb700 1 -- 172.21.15.185:0/644842662 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec011610 con 0x7fa4f40acee0 2020-11-16T22:23:17.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- 172.21.15.185:0/644842662 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 msgr2=0x7fa4f40b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:17.836 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/644842662 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7fa4ec009d50 tx=0x7fa4ec009290).stop 2020-11-16T22:23:18.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- 172.21.15.185:0/644842662 shutdown_connections 2020-11-16T22:23:18.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/644842662 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40acee0 0x7fa4f40b64b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.038 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/644842662 >> v2:172.21.15.96:3300/0 conn(0x7fa4f40b7550 0x7fa4f4148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- 172.21.15.185:0/644842662 >> 172.21.15.185:0/644842662 conn(0x7fa4f4067860 msgr2=0x7fa4f4067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:18.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- 172.21.15.185:0/644842662 shutdown_connections 2020-11-16T22:23:18.039 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- 172.21.15.185:0/644842662 wait complete. 2020-11-16T22:23:18.050 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:23:17.891+0000 7f1d21009700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1) n(v2 rc2020-11-16T22:22:44.682013+0000 4=0+4) (inest lock) (ifile excl) (iversion lock) caps={7974=pAsLsXsFsx/pAsLsXsFsx@4},l=7974 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 authpin=1 scrubqueue=0 0x55f9fc970c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:22:44.682013+0000 4=0+4)","memory_value.dirrstat":"f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1)","memory_value.rstat":"n(v2 rc2020-11-16T22:22:44.682013+0000 4=0+4)","error_str":""},"return_code":-61} 2020-11-16T22:23:18.051 INFO:tasks.ceph.mds.c.smithi096.stderr:2020-11-16T22:23:17.895+0000 7f1d27015700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v10 ap=2 f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1) n(v1 rc2020-11-16T22:22:44.682013+0000 3=0+3) (idft sync r=1) (inest sync r=1 scatter_wanted) (ifile sync r=1) (iversion lock) caps={7974=pAsLsXsFs/pAsLsXsFsx@7} | request=0 lock=3 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55f9fc971180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:22:44.490007+0000 3=0+3)","memory_value.dirrstat":"f(v0 m2020-11-16T22:22:44.486007+0000 1=0+1)","memory_value.rstat":"n(v1 rc2020-11-16T22:22:44.682013+0000 3=0+3)","error_str":""},"return_code":-61} 2020-11-16T22:23:18.051 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:23:17.891+0000 7fab5d21a700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2020-11-16T22:22:44.490007+0000 1=0+1) n(v0 rc2020-11-16T22:22:44.490007+0000 2=0+2) (inest lock) (ifile excl) (iversion lock) caps={7974=pAsLsXsFsx/pAsLsXsFsx@4},l=7974 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55bf571ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:22:44.490007+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:22:44.490007+0000 2=0+2)","memory_value.dirrstat":"f(v0 m2020-11-16T22:22:44.490007+0000 1=0+1)","memory_value.rstat":"n(v0 rc2020-11-16T22:22:44.490007+0000 2=0+2)","error_str":""},"return_code":-61} 2020-11-16T22:23:18.051 INFO:tasks.ceph.mds.b.smithi096.stderr:2020-11-16T22:23:17.891+0000 7fab5d21a700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={7974=pAsLsXsFsx/AsLsXsFsx@2},l=7974 | importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55bf571ad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v10>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v21>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirrstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2020-11-16T22:23:18.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 Processor -- start 2020-11-16T22:23:18.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- start start 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 0x7fa4f413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 0x7fa4f40ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40b3ba0 con 0x7fa4f40acee0 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40afc80 con 0x7fa4f40b7550 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f3fff700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f4141a30 con 0x7fa4f40b7170 2020-11-16T22:23:18.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 0x7fa4f40ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57802/0 (socket says 172.21.15.185:57802) 2020-11-16T22:23:18.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 0x7fa4f40ba6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36128/0 (socket says 172.21.15.185:36128) 2020-11-16T22:23:18.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 -- 172.21.15.185:0/1256632296 learned_addr learned my addr 172.21.15.185:0/1256632296 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 0x7fa4f413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 -- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 msgr2=0x7fa4f40ba6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 0x7fa4f40ba6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 -- 172.21.15.185:0/1256632296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 msgr2=0x7fa4f413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 0x7fa4f413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 -- 172.21.15.185:0/1256632296 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f413fd60 con 0x7fa4f40b7170 2020-11-16T22:23:18.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 0x7fa4f413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.846+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fa4e400ebd0 tx=0x7fa4e4006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 -- 172.21.15.185:0/1256632296 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400c8e0 con 0x7fa4f40b7170 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 -- 172.21.15.185:0/1256632296 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e400ca40 con 0x7fa4f40b7170 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 -- 172.21.15.185:0/1256632296 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400cdc0 con 0x7fa4f40b7170 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f4141590 con 0x7fa4f40b7170 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f4140080 con 0x7fa4f40b7170 2020-11-16T22:23:18.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 -- 172.21.15.185:0/1256632296 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa4e401a030 con 0x7fa4f40b7170 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 0x7fa4d0057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4eaffd700 1 -- 172.21.15.185:0/1256632296 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4e406da40 con 0x7fa4f40b7170 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 0x7fa4d0057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.850+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 0x7fa4d0057520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fa4dc001120 tx=0x7fa4dc009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 Processor -- start 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 -- start start 2020-11-16T22:23:18.257 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x1557960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1557e50 0x1556430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x13e3560 con 0x1339860 2020-11-16T22:23:18.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x14979e0 con 0x12cbc60 2020-11-16T22:23:18.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x13577a0 con 0x1557e50 2020-11-16T22:23:18.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36134/0 (socket says 172.21.15.185:36134) 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 -- 172.21.15.185:0/2812726647 learned_addr learned my addr 172.21.15.185:0/2812726647 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x1557960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1557e50 0x1556430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 -- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1557e50 msgr2=0x1556430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1557e50 0x1556430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 -- 172.21.15.185:0/2812726647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 msgr2=0x1557960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:18.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x1557960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 -- 172.21.15.185:0/2812726647 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1352650 con 0x12cbc60 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x1557960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fa4e40474a0 tx=0x7fa4e4009000).ready entity=mon.1 client_cookie=f541a2b8fb41822f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.854+0000 7fa4cbfff700 1 -- 172.21.15.185:0/2812726647 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4047780 con 0x12cbc60 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4cbfff700 1 -- 172.21.15.185:0/2812726647 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e40441f0 con 0x12cbc60 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4cbfff700 1 -- 172.21.15.185:0/2812726647 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4047110 con 0x12cbc60 2020-11-16T22:23:18.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 msgr2=0x1555f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fa4e40474a0 tx=0x7fa4e4009000).stop 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/2812726647 shutdown_connections 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1557e50 0x1556430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x1557960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- 172.21.15.185:0/2812726647 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x1555f40 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/2812726647 >> 172.21.15.185:0/2812726647 conn(0x1507ef0 msgr2=0x1554e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/2812726647 shutdown_connections 2020-11-16T22:23:18.465 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/2812726647 wait complete. 2020-11-16T22:23:18.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- start start 2020-11-16T22:23:18.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 Processor -- start 2020-11-16T22:23:18.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.466 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 0x1558a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1484790 con 0x1557e50 2020-11-16T22:23:18.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x14848d0 con 0x12cbc60 2020-11-16T22:23:18.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1485b40 con 0x1339860 2020-11-16T22:23:18.670 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 10 client.0 ll_register_callbacks cb 0x147ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36138/0 (socket says 172.21.15.185:36138) 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/119498432 learned_addr learned my addr 172.21.15.185:0/119498432 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 0x1558a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 msgr2=0x1558530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:18.671 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/119498432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 msgr2=0x1558a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 0x1558a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/119498432 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1475330 con 0x12cbc60 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 0x1558a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fa4dc017860 tx=0x7fa4dc003000).ready entity=mon.1 client_cookie=d906e6b0440e6fa0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:18.672 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc01c070 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4dc011b30 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.858+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1538770 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0146d0 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 10 client.8305 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x12ab080 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1538770 con 0x12cbc60 2020-11-16T22:23:18.673 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa4dc021400 con 0x12cbc60 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 10 client.8305 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 10 client.8305 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 20 client.8305 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x155b410 0x159ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 4 client.8305 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:18.674 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1375d00 con 0x155b410 2020-11-16T22:23:18.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x155b410 0x159ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 10 client.8305 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:18.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x155b410 0x159ee20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1fbf9f6dbbf9c204 server_cookie=2dc97172514914dd in_seq=0 out_seq=0 2020-11-16T22:23:18.878 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 1 -- 172.21.15.185:0/119498432 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1375d00 con 0x155b410 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 10 client.8305 handle_command_reply: tid=0 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 client.8305 shutdown 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 msgr2=0x155af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fa4dc017860 tx=0x7fa4dc003000).stop 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 shutdown_connections 2020-11-16T22:23:18.879 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x155b410 0x159ee20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x12cbc60 0x155af20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1557e50 0x1558a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 --2- 172.21.15.185:0/119498432 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 >> 172.21.15.185:0/119498432 conn(0x1507ef0 msgr2=0x1555ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa4c97fa700 0 client.8305 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 shutdown_connections 2020-11-16T22:23:18.880 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 1 -- 172.21.15.185:0/119498432 wait complete. 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.862+0000 7fa508215740 20 client.8305 trim_cache size 0 max 16384 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 Processor -- start 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 -- start start 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x1559060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x12cbc60 0x1555fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x145cef0 con 0x12cbc60 2020-11-16T22:23:18.881 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1587d10 con 0x1507ef0 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x158a4e0 con 0x1339860 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa4f37fe700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57822/0 (socket says 172.21.15.185:57822) 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa4f37fe700 1 -- 172.21.15.185:0/3033778865 learned_addr learned my addr 172.21.15.185:0/3033778865 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.882+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x1559060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.882 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x12cbc60 0x1555fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:18.883 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 -- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 msgr2=0x1559060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:19.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x1559060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 -- 172.21.15.185:0/3033778865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x12cbc60 msgr2=0x1555fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:19.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x12cbc60 0x1555fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.090 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 -- 172.21.15.185:0/3033778865 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x159b6e0 con 0x1339860 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fa4f4144300 tx=0x7fa4ec016000).ready entity=mon.2 client_cookie=c3aeb16cf54d69b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4e8ff9700 1 -- 172.21.15.185:0/3033778865 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00be90 con 0x1339860 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4e8ff9700 1 -- 172.21.15.185:0/3033778865 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec023040 con 0x1339860 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4e8ff9700 1 -- 172.21.15.185:0/3033778865 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec011c60 con 0x1339860 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 msgr2=0x1558230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:19.091 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fa4f4144300 tx=0x7fa4ec016000).stop 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- 172.21.15.185:0/3033778865 shutdown_connections 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x1559060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1339860 0x1558230 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- 172.21.15.185:0/3033778865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x12cbc60 0x1555fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- 172.21.15.185:0/3033778865 >> 172.21.15.185:0/3033778865 conn(0x7fa4c0002320 msgr2=0x155af80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- 172.21.15.185:0/3033778865 shutdown_connections 2020-11-16T22:23:19.092 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- 172.21.15.185:0/3033778865 wait complete. 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- start start 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 Processor -- start 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x15e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x15e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1594360 con 0x1339860 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1595bd0 con 0x1507ef0 2020-11-16T22:23:19.093 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1559570 con 0x12cbc60 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa508215740 10 client.0 ll_register_callbacks cb 0x158a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57828/0 (socket says 172.21.15.185:57828) 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/3151582488 learned_addr learned my addr 172.21.15.185:0/3151582488 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f37fe700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x15e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x15e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:19.094 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 msgr2=0x15e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:19.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x15e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/3151582488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 msgr2=0x15e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:19.298 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x15e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 -- 172.21.15.185:0/3151582488 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1551dc0 con 0x12cbc60 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4f2ffd700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa4dc017860 tx=0x7fa4dc01f000).ready entity=mon.2 client_cookie=d745f3c105f7a618 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.886+0000 7fa4c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc031040 con 0x12cbc60 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4dc021400 con 0x12cbc60 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4e40441f0 con 0x12cbc60 2020-11-16T22:23:19.299 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0148e0 con 0x12cbc60 2020-11-16T22:23:19.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 10 client.7403 fetch_fsmap learned FSMap version 109 2020-11-16T22:23:19.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x7fa4c0008160 con 0x12cbc60 2020-11-16T22:23:19.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e40441f0 con 0x12cbc60 2020-11-16T22:23:19.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 109) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fa4dc03e020 con 0x12cbc60 2020-11-16T22:23:19.300 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 10 client.7403 fetch_fsmap finished waiting for FSMap version 109 2020-11-16T22:23:19.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 10 client.7403 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:19.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 20 client.7403 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:19.502 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1557e50 0x1593480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 4 client.7403 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,r,t,",,, ,",p,a,t,h,",:, ,",/,d,1,/,d,2,/,d,3,",,, ,",s,c,r,u,b,o,p,s,",:, ,[,",r,e,c,u,r,s,i,v,e,",,, ,",f,o,r,c,e,",],}] 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " s c r u b s t a r t " , " p a t h " : " / d 1 / d 2 / d 3 " , " s c r u b o p s " : [ " r e c u r s i v e " , " f o r c e " ] }) v1 -- 0x137d000 con 0x1557e50 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1557e50 0x1593480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 10 client.7403 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4f27fc700 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1557e50 0x1593480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=15034572bf7c4373 server_cookie=74240dcf857eebdb in_seq=0 out_seq=0 2020-11-16T22:23:19.503 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 1 -- 172.21.15.185:0/3151582488 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x137d000 con 0x1557e50 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 10 client.7403 handle_command_reply: tid=0 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 client.7403 shutdown 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 msgr2=0x1558a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa4dc017860 tx=0x7fa4dc01f000).stop 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 shutdown_connections 2020-11-16T22:23:19.504 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1557e50 0x1593480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1507ef0 0x15e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa4c97fa700 0 client.7403 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:19.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x12cbc60 0x1558a50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.706 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 --2- 172.21.15.185:0/3151582488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1339860 0x15e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 >> 172.21.15.185:0/3151582488 conn(0x7fa4c0002320 msgr2=0x1556060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 shutdown_connections 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 1 -- 172.21.15.185:0/3151582488 wait complete. 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.890+0000 7fa508215740 20 client.7403 trim_cache size 0 max 16384 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 msgr2=0x7fa4d0057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:19.707 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 0x7fa4d0057520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fa4dc001120 tx=0x7fa4dc009040).stop 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 msgr2=0x7fa4f41476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fa4e400ebd0 tx=0x7fa4e4006df0).stop 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 shutdown_connections 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa4d0055060 0x7fa4d0057520 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa4f40b7550 0x7fa4f40ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f40acee0 0x7fa4f413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.708 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 --2- 172.21.15.185:0/1256632296 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa4f40b7170 0x7fa4f41476e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:19.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 >> 172.21.15.185:0/1256632296 conn(0x7fa4f4067860 msgr2=0x7fa4f4144790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:19.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 shutdown_connections 2020-11-16T22:23:19.910 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:17.894+0000 7fa4f3fff700 1 -- 172.21.15.185:0/1256632296 wait complete. 2020-11-16T22:23:19.910 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:19.911 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0, 2020-11-16T22:23:19.911 INFO:teuthology.orchestra.run.smithi185.stdout: "scrub_tag": "4a97bcea-0e01-4b36-9db4-20d2bef38a21", 2020-11-16T22:23:19.911 INFO:teuthology.orchestra.run.smithi185.stdout: "mode": "asynchronous" 2020-11-16T22:23:19.911 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:19.911 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:23:17.890+0000 7f5df92a7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v23 ap=1 f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1) n(v4 rc2020-11-16T22:22:44.682013+0000 6=0+6) (inest mix) (ifile excl) (iversion lock) caps={7974=pAsLsXsFsx/pAsLsXsFsx@5},l=7974 | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x562d3c2f6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:22:44.682013+0000 6=0+6)","memory_value.dirrstat":"f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1)","memory_value.rstat":"n(v4 rc2020-11-16T22:22:44.682013+0000 6=0+6)","error_str":""},"return_code":-61} 2020-11-16T22:23:19.912 INFO:tasks.ceph.mds.a.smithi185.stderr:2020-11-16T22:23:17.894+0000 7f5dff2b3700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1) n(v3 rc2020-11-16T22:22:44.682013+0000 5=0+5) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={7974=pAsLsXsFs/pAsLsXsFsx@9} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x562d3c2f6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v23>,<0x10000000000/d2 v25>,<0x1/d1 v22>]//","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2020-11-16T22:22:44.682013+0000 5=0+5)","memory_value.dirrstat":"f(v0 m2020-11-16T22:22:44.482007+0000 1=0+1)","memory_value.rstat":"n(v3 rc2020-11-16T22:22:44.682013+0000 5=0+5)","error_str":""},"return_code":-61} 2020-11-16T22:23:19.912 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:20.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 Processor -- start 2020-11-16T22:23:20.048 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 -- start start 2020-11-16T22:23:20.049 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fc98c144b90 0x7fc98c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c12cca0 0x7fc98c12d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c0b82e0 con 0x7fc98c0b4720 2020-11-16T22:23:20.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98c0b7650 con 0x7fc98c144b90 2020-11-16T22:23:20.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9920a2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0b78f0 con 0x7fc98c12cca0 2020-11-16T22:23:20.050 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9910a0700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fc98c0b4720 0x7fc98c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57830/0 (socket says 172.21.15.185:57830) 2020-11-16T22:23:20.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9910a0700 1 -- 172.21.15.185:0/1529511440 learned_addr learned my addr 172.21.15.185:0/1529511440 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:23:20.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc99089f700 1 --2- 172.21.15.185:0/1529511440 >> v2:172.21.15.185:3300/0 conn(0x7fc98c144b90 0x7fc98c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.051 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.058+0000 7fc9918a1700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c12cca0 0x7fc98c12d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc97bfff700 1 -- 172.21.15.185:0/1529511440 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707358550 0 0) 0x7fc98c0b82e0 con 0x7fc98c0b4720 2020-11-16T22:23:20.052 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc97bfff700 1 -- 172.21.15.185:0/1529511440 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974003240 con 0x7fc98c0b4720 2020-11-16T22:23:20.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc99089f700 1 -- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c12cca0 msgr2=0x7fc98c12d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:20.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc99089f700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c12cca0 0x7fc98c12d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.053 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc99089f700 1 -- 172.21.15.185:0/1529511440 >> v1:172.21.15.185:6789/0 conn(0x7fc98c0b4720 legacy=0x7fc98c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc99089f700 1 -- 172.21.15.185:0/1529511440 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0b1ee0 con 0x7fc98c144b90 2020-11-16T22:23:20.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc99089f700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c144b90 0x7fc98c0b2a90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7fc97c00b9a0 tx=0x7fc97c006df0).ready entity=mon.1 client_cookie=e68f33b46cde0e4f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:20.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc97bfff700 1 -- 172.21.15.185:0/1529511440 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c009680 con 0x7fc98c144b90 2020-11-16T22:23:20.254 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc97bfff700 1 -- 172.21.15.185:0/1529511440 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0097e0 con 0x7fc98c144b90 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc97bfff700 1 -- 172.21.15.185:0/1529511440 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c009b60 con 0x7fc98c144b90 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 -- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c144b90 msgr2=0x7fc98c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c144b90 0x7fc98c0b2a90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7fc97c00b9a0 tx=0x7fc97c006df0).stop 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 -- 172.21.15.185:0/1529511440 shutdown_connections 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c12cca0 0x7fc98c12d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 --2- 172.21.15.185:0/1529511440 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c144b90 0x7fc98c0b2a90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.255 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 -- 172.21.15.185:0/1529511440 >> 172.21.15.185:0/1529511440 conn(0x7fc98c067860 msgr2=0x7fc98c067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:20.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 -- 172.21.15.185:0/1529511440 shutdown_connections 2020-11-16T22:23:20.256 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.062+0000 7fc9920a2700 1 -- 172.21.15.185:0/1529511440 wait complete. 2020-11-16T22:23:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 Processor -- start 2020-11-16T22:23:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- start start 2020-11-16T22:23:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc98c0b4720 0x7fc98c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.258 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 0x7fc98c12a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 0x7fc98c114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0aeaf0 con 0x7fc98c0b4720 2020-11-16T22:23:20.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c1149c0 con 0x7fc98c12cca0 2020-11-16T22:23:20.462 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c1536e0 con 0x7fc98c144b90 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc99089f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 0x7fc98c12a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 0x7fc98c114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc99089f700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 0x7fc98c12a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36168/0 (socket says 172.21.15.185:36168) 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc99089f700 1 -- 172.21.15.185:0/3035627797 learned_addr learned my addr 172.21.15.185:0/3035627797 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9910a0700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc98c0b4720 0x7fc98c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.463 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 -- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 msgr2=0x7fc98c12a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 0x7fc98c12a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 -- 172.21.15.185:0/3035627797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc98c0b4720 msgr2=0x7fc98c1531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc98c0b4720 0x7fc98c1531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.464 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 -- 172.21.15.185:0/3035627797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0b0a40 con 0x7fc98c144b90 2020-11-16T22:23:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9918a1700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 0x7fc98c114e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fc98400ed20 tx=0x7fc9840091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98400eee0 con 0x7fc98c144b90 2020-11-16T22:23:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98400cbb0 con 0x7fc98c144b90 2020-11-16T22:23:20.666 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c0b0dc0 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9840123f0 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.066+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c0ae6f0 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c0aedd0 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fc984022070 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9797fa700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 0x7fc964057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc98406e520 con 0x7fc98c144b90 2020-11-16T22:23:20.667 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9910a0700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 0x7fc964057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.070+0000 7fc9910a0700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 0x7fc964057520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc9880055a0 tx=0x7fc988009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:20.668 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.074+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fc984044780 con 0x7fc98c144b90 2020-11-16T22:23:20.871 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.278+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98c05d200 con 0x7fc98c144b90 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.278+0000 7fc9797fa700 1 -- 172.21.15.185:0/3035627797 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+2528 (secure 0 0 0) 0x7fc984016020 con 0x7fc98c144b90 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 msgr2=0x7fc964057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 0x7fc964057520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fc9880055a0 tx=0x7fc988009040).stop 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 msgr2=0x7fc98c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 0x7fc98c114e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fc98400ed20 tx=0x7fc9840091d0).stop 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 shutdown_connections 2020-11-16T22:23:20.872 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fc964055060 0x7fc964057520 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fc98c144b90 0x7fc98c114e20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc98c0b4720 0x7fc98c1531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 --2- 172.21.15.185:0/3035627797 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fc98c12cca0 0x7fc98c12a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 >> 172.21.15.185:0/3035627797 conn(0x7fc98c067860 msgr2=0x7fc98c129bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 shutdown_connections 2020-11-16T22:23:20.873 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:20.282+0000 7fc9920a2700 1 -- 172.21.15.185:0/3035627797 wait complete. 2020-11-16T22:23:20.874 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 109 2020-11-16T22:23:20.874 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:20.874 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":109,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:05.291578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:21.078 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2020-11-16T22:23:21.118 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:23:21.127+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:23:21.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.218+0000 7fb20c8a5700 1 Processor -- start 2020-11-16T22:23:21.210 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.218+0000 7fb20c8a5700 1 -- start start 2020-11-16T22:23:21.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb2080b6650 0x7fb208147f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb208144c60 0x7fb20813d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.211 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- --> v1:172.21.15.185:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb208063660 con 0x7fb2080b4d80 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080b2f80 con 0x7fb2080b6650 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080638c0 con 0x7fb208144c60 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fb208144c60 0x7fb20813d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb205d9b700 1 --1- >> v1:172.21.15.185:6789/0 conn(0x7fb2080b4d80 0x7fb208144550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.185:6789/0 says I am v1:172.21.15.185:57846/0 (socket says 172.21.15.185:57846) 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb205d9b700 1 -- 172.21.15.185:0/2558678030 learned_addr learned my addr 172.21.15.185:0/2558678030 (peer_addr_for_me v1:172.21.15.185:0/0) 2020-11-16T22:23:21.212 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20659c700 1 --2- 172.21.15.185:0/2558678030 >> v2:172.21.15.96:3300/0 conn(0x7fb2080b6650 0x7fb208147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20559a700 1 -- 172.21.15.185:0/2558678030 <== mon.1 v1:172.21.15.185:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055895549 0 0) 0x7fb208063660 con 0x7fb2080b4d80 2020-11-16T22:23:21.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20559a700 1 -- 172.21.15.185:0/2558678030 --> v1:172.21.15.185:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1ec003240 con 0x7fb2080b4d80 2020-11-16T22:23:21.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 -- 172.21.15.185:0/2558678030 >> v2:172.21.15.96:3300/0 conn(0x7fb2080b6650 msgr2=0x7fb208147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:21.214 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 --2- 172.21.15.185:0/2558678030 >> v2:172.21.15.96:3300/0 conn(0x7fb2080b6650 0x7fb208147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 -- 172.21.15.185:0/2558678030 >> v1:172.21.15.185:6789/0 conn(0x7fb2080b4d80 legacy=0x7fb208144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 -- 172.21.15.185:0/2558678030 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208063b60 con 0x7fb208144c60 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb206d9d700 1 --2- 172.21.15.185:0/2558678030 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb208144c60 0x7fb20813d710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb2080b63d0 tx=0x7fb2000091d0).ready entity=mon.1 client_cookie=122759bbe24b789e server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20559a700 1 -- 172.21.15.185:0/2558678030 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000b9c0 con 0x7fb208144c60 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20559a700 1 -- 172.21.15.185:0/2558678030 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20000bb20 con 0x7fb208144c60 2020-11-16T22:23:21.215 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20559a700 1 -- 172.21.15.185:0/2558678030 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000bea0 con 0x7fb208144c60 2020-11-16T22:23:21.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- 172.21.15.185:0/2558678030 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb208144c60 msgr2=0x7fb20813d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:21.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/2558678030 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb208144c60 0x7fb20813d710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fb2080b63d0 tx=0x7fb2000091d0).stop 2020-11-16T22:23:21.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- 172.21.15.185:0/2558678030 shutdown_connections 2020-11-16T22:23:21.418 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/2558678030 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb208144c60 0x7fb20813d710 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/2558678030 >> v2:172.21.15.96:3300/0 conn(0x7fb2080b6650 0x7fb208147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- 172.21.15.185:0/2558678030 >> 172.21.15.185:0/2558678030 conn(0x7fb2080649a0 msgr2=0x7fb208062e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- 172.21.15.185:0/2558678030 shutdown_connections 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.222+0000 7fb20c8a5700 1 -- 172.21.15.185:0/2558678030 wait complete. 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 Processor -- start 2020-11-16T22:23:21.419 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- start start 2020-11-16T22:23:21.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.430 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2080b6650 0x7fb208156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208144c60 0x7fb2081417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208146bd0 con 0x7fb208144c60 2020-11-16T22:23:21.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208156c00 con 0x7fb2080b4d80 2020-11-16T22:23:21.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb208146690 con 0x7fb2080b6650 2020-11-16T22:23:21.431 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36180/0 (socket says 172.21.15.185:36180) 2020-11-16T22:23:21.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 -- 172.21.15.185:0/4213933376 learned_addr learned my addr 172.21.15.185:0/4213933376 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:21.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb205d9b700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2080b6650 0x7fb208156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb206d9d700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208144c60 0x7fb2081417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.634 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 -- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2080b6650 msgr2=0x7fb208156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2080b6650 0x7fb208156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 -- 172.21.15.185:0/4213933376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208144c60 msgr2=0x7fb2081417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208144c60 0x7fb2081417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 -- 172.21.15.185:0/4213933376 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208146930 con 0x7fb2080b4d80 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20659c700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fb1fc00eba0 tx=0x7fb1fc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:21.635 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb1f67fc700 1 -- 172.21.15.185:0/4213933376 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00c9b0 con 0x7fb2080b4d80 2020-11-16T22:23:21.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb1f67fc700 1 -- 172.21.15.185:0/4213933376 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc00cb10 con 0x7fb2080b4d80 2020-11-16T22:23:21.636 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2080ac550 con 0x7fb2080b4d80 2020-11-16T22:23:21.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb1f67fc700 1 -- 172.21.15.185:0/4213933376 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00ce90 con 0x7fb2080b4d80 2020-11-16T22:23:21.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2080abef0 con 0x7fb2080b4d80 2020-11-16T22:23:21.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.226+0000 7fb1f67fc700 1 -- 172.21.15.185:0/4213933376 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fb1fc012600 con 0x7fb2080b4d80 2020-11-16T22:23:21.838 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb1f67fc700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 0x7fb1dc057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb1f67fc700 1 -- 172.21.15.185:0/4213933376 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb1fc06e130 con 0x7fb2080b4d80 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb205d9b700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 0x7fb1dc057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb205d9b700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 0x7fb1dc057520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fb208146d80 tx=0x7fb1f8009040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb21b336740 1 Processor -- start 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.230+0000 7fb21b336740 1 -- start start 2020-11-16T22:23:21.839 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 0x1ad02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1acaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 0x1acb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19224a0 con 0x180aba0 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19d6920 con 0x1ad07e0 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x18966e0 con 0x18787a0 2020-11-16T22:23:21.840 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 0x1ad02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 0x1acb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 0x1ad02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57864/0 (socket says 172.21.15.185:57864) 2020-11-16T22:23:22.042 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb20659c700 1 -- 172.21.15.185:0/1533395626 learned_addr learned my addr 172.21.15.185:0/1533395626 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb205d9b700 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1acaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 -- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 msgr2=0x1ad02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 0x1ad02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 -- 172.21.15.185:0/1533395626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 msgr2=0x1acaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1acaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 -- 172.21.15.185:0/1533395626 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1891590 con 0x1ad07e0 2020-11-16T22:23:22.043 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb205d9b700 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1acaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:22.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 0x1acb3f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fb2080b63d0 tx=0x7fb200016000).ready entity=mon.1 client_cookie=ad8d34bf6cd1fd11 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:22.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb1e37fe700 1 -- 172.21.15.185:0/1533395626 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000bea0 con 0x1ad07e0 2020-11-16T22:23:22.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb1e37fe700 1 -- 172.21.15.185:0/1533395626 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200021070 con 0x1ad07e0 2020-11-16T22:23:22.044 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb1e37fe700 1 -- 172.21.15.185:0/1533395626 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200011610 con 0x1ad07e0 2020-11-16T22:23:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 msgr2=0x1acb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 0x1acb3f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fb2080b63d0 tx=0x7fb200016000).stop 2020-11-16T22:23:22.246 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- 172.21.15.185:0/1533395626 shutdown_connections 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x1ad07e0 0x1acb3f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x18787a0 0x1ad02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 --2- 172.21.15.185:0/1533395626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1acaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- 172.21.15.185:0/1533395626 >> 172.21.15.185:0/1533395626 conn(0x1a46e30 msgr2=0x1acfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- 172.21.15.185:0/1533395626 shutdown_connections 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- 172.21.15.185:0/1533395626 wait complete. 2020-11-16T22:23:22.247 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 -- start start 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.234+0000 7fb21b336740 1 Processor -- start 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1ac9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 0x1addee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c36d0 con 0x18787a0 2020-11-16T22:23:22.248 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c3810 con 0x180aba0 2020-11-16T22:23:22.249 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x19c4a80 con 0x1ab5800 2020-11-16T22:23:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 10 client.0 ll_register_callbacks cb 0x19b9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 0x1addee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 0x1addee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57872/0 (socket says 172.21.15.185:57872) 2020-11-16T22:23:22.450 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb206d9d700 1 -- 172.21.15.185:0/2653724519 learned_addr learned my addr 172.21.15.185:0/2653724519 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1ac9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb205d9b700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 -- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 msgr2=0x1addee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 0x1addee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 -- 172.21.15.185:0/2653724519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 msgr2=0x1ab37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.451 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 -- 172.21.15.185:0/2653724519 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x19b4270 con 0x180aba0 2020-11-16T22:23:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb20659c700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1ac9730 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fb1fc0444f0 tx=0x7fb1fc01e040).ready entity=mon.1 client_cookie=304a79fe91f533e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:22.452 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc044700 con 0x180aba0 2020-11-16T22:23:22.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb1fc044860 con 0x180aba0 2020-11-16T22:23:22.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a8be00 con 0x180aba0 2020-11-16T22:23:22.654 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1fc043ff0 con 0x180aba0 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 10 client.8335 fetch_fsmap learned FSMap version 110 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x17e9fc0 con 0x180aba0 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1a8be00 con 0x180aba0 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 110) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb1fc046f50 con 0x180aba0 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 10 client.8335 fetch_fsmap finished waiting for FSMap version 110 2020-11-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 10 client.8335 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 20 client.8335 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ac5cf0 0x1adade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 4 client.8335 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x18b4c40 con 0x1ac5cf0 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb205d9b700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ac5cf0 0x1adade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb1e0ff9700 10 client.8335 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.238+0000 7fb205d9b700 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ac5cf0 0x1adade0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=9857958adad9c2fd server_cookie=6b4266477b89dd6f in_seq=0 out_seq=0 2020-11-16T22:23:22.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/2653724519 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x18b4c40 con 0x1ac5cf0 2020-11-16T22:23:22.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb1e0ff9700 10 client.8335 handle_command_reply: tid=0 2020-11-16T22:23:22.858 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 client.8335 shutdown 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 msgr2=0x1ac9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1ac9730 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7fb1fc0444f0 tx=0x7fb1fc01e040).stop 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 shutdown_connections 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ac5cf0 0x1adade0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1ac9730 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.859 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb1e0ff9700 0 client.8335 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1ab5800 0x1addee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 --2- 172.21.15.185:0/2653724519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 >> 172.21.15.185:0/2653724519 conn(0x1a46e30 msgr2=0x1acc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 shutdown_connections 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 1 -- 172.21.15.185:0/2653724519 wait complete. 2020-11-16T22:23:22.860 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.242+0000 7fb21b336740 20 client.8335 trim_cache size 0 max 16384 2020-11-16T22:23:22.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.258+0000 7fb21b336740 1 Processor -- start 2020-11-16T22:23:22.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.258+0000 7fb21b336740 1 -- start start 2020-11-16T22:23:22.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1aca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:22.861 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1a8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x199be30 con 0x180aba0 2020-11-16T22:23:23.062 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1adb870 con 0x18787a0 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ac60d0 con 0x1a46e30 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1aca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36204/0 (socket says 172.21.15.185:36204) 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1aca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57878/0 (socket says 172.21.15.185:57878) 2020-11-16T22:23:23.063 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 -- 172.21.15.185:0/1325813606 learned_addr learned my addr 172.21.15.185:0/1325813606 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb205d9b700 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1a8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 -- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 msgr2=0x1aca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1aca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 -- 172.21.15.185:0/1325813606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 msgr2=0x1a8ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.064 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1a8ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 -- 172.21.15.185:0/1325813606 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00081d0 con 0x18787a0 2020-11-16T22:23:23.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fb2080619f0 tx=0x7fb20001f070).ready entity=mon.1 client_cookie=52b3b5a9ddebe9d server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.266 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb1e3fff700 1 -- 172.21.15.185:0/1325813606 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20002b030 con 0x18787a0 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb1e3fff700 1 -- 172.21.15.185:0/1325813606 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20002c040 con 0x18787a0 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb1e3fff700 1 -- 172.21.15.185:0/1325813606 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000032b0 con 0x18787a0 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 msgr2=0x1acc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fb2080619f0 tx=0x7fb20001f070).stop 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- 172.21.15.185:0/1325813606 shutdown_connections 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x18787a0 0x1acc5c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.267 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1aca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- 172.21.15.185:0/1325813606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x180aba0 0x1a8ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- 172.21.15.185:0/1325813606 >> 172.21.15.185:0/1325813606 conn(0x7fb1d0002320 msgr2=0x1ab3590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- 172.21.15.185:0/1325813606 shutdown_connections 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- 172.21.15.185:0/1325813606 wait complete. 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- start start 2020-11-16T22:23:23.268 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 Processor -- start 2020-11-16T22:23:23.269 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.470 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ada750 con 0x18787a0 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ada9f0 con 0x180aba0 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x1adb640 con 0x1a46e30 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb21b336740 10 client.0 ll_register_callbacks cb 0x1a91df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.471 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb20659c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36208/0 (socket says 172.21.15.185:36208) 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb20659c700 1 -- 172.21.15.185:0/759595744 learned_addr learned my addr 172.21.15.185:0/759595744 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57888/0 (socket says 172.21.15.185:57888) 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.262+0000 7fb205d9b700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 -- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 msgr2=0x1acf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 -- 172.21.15.185:0/759595744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 msgr2=0x1ab27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.472 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 -- 172.21.15.185:0/759595744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1828710 con 0x180aba0 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb206d9d700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb20659c700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fb1fc047f20 tx=0x7fb1fc0478a0).ready entity=mon.1 client_cookie=93381f89c65f1e72 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc046390 con 0x180aba0 2020-11-16T22:23:23.473 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb1fc046f50 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb200021070 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1fc044700 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 10 client.8341 fetch_fsmap learned FSMap version 110 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x1833df0 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200021070 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 110) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fb1fc046650 con 0x180aba0 2020-11-16T22:23:23.474 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 10 client.8341 fetch_fsmap finished waiting for FSMap version 110 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 10 client.8341 resolve_mds: resolved ID 'a' to GID 7993 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 20 client.8341 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ab5800 0x1acfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 4 client.8341 mds_command: new command op to 7993 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,p,a,u,s,e,",}] 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 --> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] -- command(tid 0: { " p r e f i x " : " s c r u b p a u s e " }) v1 -- 0x18bbf40 con 0x1ab5800 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb205d9b700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ab5800 0x1acfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.475 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 10 client.8341 ms_handle_connect on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb205d9b700 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ab5800 0x1acfc20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=127a818907b2e135 server_cookie=d5411f7f77f4c497 in_seq=0 out_seq=0 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 1 -- 172.21.15.185:0/759595744 <== mds.0 v2:172.21.15.185:6834/1118138229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x18bbf40 con 0x1ab5800 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 10 client.8341 handle_command_reply: tid=0 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 client.8341 shutdown 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 msgr2=0x1adb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fb1fc047f20 tx=0x7fb1fc0478a0).stop 2020-11-16T22:23:23.476 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 shutdown_connections 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:6834/1118138229,v1:172.21.15.185:6835/1118138229] conn(0x1ab5800 0x1acfc20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb1e0ff9700 0 client.8341 ms_handle_reset on v2:172.21.15.185:6834/1118138229 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x180aba0 0x1adb150 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x1a46e30 0x1acf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 --2- 172.21.15.185:0/759595744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x18787a0 0x1ab27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.477 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 >> 172.21.15.185:0/759595744 conn(0x7fb1d0002320 msgr2=0x1a95a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 shutdown_connections 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 1 -- 172.21.15.185:0/759595744 wait complete. 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.266+0000 7fb21b336740 20 client.8341 trim_cache size 0 max 16384 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 msgr2=0x7fb1dc057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 0x7fb1dc057520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fb208146d80 tx=0x7fb1f8009040).stop 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 msgr2=0x7fb208146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.478 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7fb1fc00eba0 tx=0x7fb1fc006df0).stop 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 shutdown_connections 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fb1dc055060 0x7fb1dc057520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fb2080b4d80 0x7fb208146fc0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208144c60 0x7fb2081417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 --2- 172.21.15.185:0/4213933376 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fb2080b6650 0x7fb208156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 >> 172.21.15.185:0/4213933376 conn(0x7fb2080649a0 msgr2=0x7fb2080b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.479 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 shutdown_connections 2020-11-16T22:23:23.480 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:21.270+0000 7fb20c8a5700 1 -- 172.21.15.185:0/4213933376 wait complete. 2020-11-16T22:23:23.480 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:23.480 INFO:teuthology.orchestra.run.smithi185.stdout: "return_code": 0 2020-11-16T22:23:23.480 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:23.481 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:23.614 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 Processor -- start 2020-11-16T22:23:23.615 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- start start 2020-11-16T22:23:23.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 0x7efc4c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.616 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc4c0b0f90 0x7efc4c144300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c066f80 con 0x7efc4c0b37e0 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c1478d0 con 0x7efc4c148550 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c147b70 con 0x7efc4c0b0f90 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc4c0b0f90 0x7efc4c144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7efc4c0b0f90 0x7efc4c144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36220/0 (socket says 172.21.15.185:36220) 2020-11-16T22:23:23.617 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 -- 172.21.15.185:0/7393594 learned_addr learned my addr 172.21.15.185:0/7393594 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc4bfff700 1 --2- 172.21.15.185:0/7393594 >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 0x7efc4c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 -- 172.21.15.185:0/7393594 >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 msgr2=0x7efc4c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 --2- 172.21.15.185:0/7393594 >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 0x7efc4c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 -- 172.21.15.185:0/7393594 >> v1:172.21.15.96:6789/0 conn(0x7efc4c0b37e0 legacy=0x7efc4c141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 -- 172.21.15.185:0/7393594 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c147e10 con 0x7efc4c0b0f90 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc4bfff700 1 --2- 172.21.15.185:0/7393594 >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 0x7efc4c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-16T22:23:23.618 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc51676700 1 --2- 172.21.15.185:0/7393594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c0b0f90 0x7efc4c144300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7efc4c0657f0 tx=0x7efc44006eb0).ready entity=mon.1 client_cookie=a038974545c6ba7b server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc4b7fe700 1 -- 172.21.15.185:0/7393594 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44009890 con 0x7efc4c0b0f90 2020-11-16T22:23:23.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc4b7fe700 1 -- 172.21.15.185:0/7393594 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc440099f0 con 0x7efc4c0b0f90 2020-11-16T22:23:23.619 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc4b7fe700 1 -- 172.21.15.185:0/7393594 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44009d70 con 0x7efc4c0b0f90 2020-11-16T22:23:23.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- 172.21.15.185:0/7393594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c0b0f90 msgr2=0x7efc4c144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 --2- 172.21.15.185:0/7393594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c0b0f90 0x7efc4c144300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7efc4c0657f0 tx=0x7efc44006eb0).stop 2020-11-16T22:23:23.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- 172.21.15.185:0/7393594 shutdown_connections 2020-11-16T22:23:23.621 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 --2- 172.21.15.185:0/7393594 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c0b0f90 0x7efc4c144300 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 --2- 172.21.15.185:0/7393594 >> v2:172.21.15.96:3300/0 conn(0x7efc4c148550 0x7efc4c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- 172.21.15.185:0/7393594 >> 172.21.15.185:0/7393594 conn(0x7efc4c0604a0 msgr2=0x7efc4c0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- 172.21.15.185:0/7393594 shutdown_connections 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.626+0000 7efc530d9700 1 -- 172.21.15.185:0/7393594 wait complete. 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 Processor -- start 2020-11-16T22:23:23.622 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- start start 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc4c0b0f90 0x7efc4c0675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc4c0b37e0 0x7efc4c153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c099a20 con 0x7efc4c0b0f90 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1470a0 con 0x7efc4c148550 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c147340 con 0x7efc4c0b37e0 2020-11-16T22:23:23.623 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36224/0 (socket says 172.21.15.185:36224) 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 -- 172.21.15.185:0/3098641388 learned_addr learned my addr 172.21.15.185:0/3098641388 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc4bfff700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc4c0b37e0 0x7efc4c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc50e75700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc4c0b0f90 0x7efc4c0675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 -- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc4c0b37e0 msgr2=0x7efc4c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc4c0b37e0 0x7efc4c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.624 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 -- 172.21.15.185:0/3098641388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc4c0b0f90 msgr2=0x7efc4c0675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc4c0b0f90 0x7efc4c0675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 -- 172.21.15.185:0/3098641388 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c067ad0 con 0x7efc4c148550 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc51676700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7efc44006fd0 tx=0x7efc44015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44006be0 con 0x7efc4c148550 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4401d5d0 con 0x7efc4c148550 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c1475e0 con 0x7efc4c148550 2020-11-16T22:23:23.625 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4401d950 con 0x7efc4c148550 2020-11-16T22:23:23.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.630+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c099580 con 0x7efc4c148550 2020-11-16T22:23:23.626 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c153650 con 0x7efc4c148550 2020-11-16T22:23:23.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7efc44025030 con 0x7efc4c148550 2020-11-16T22:23:23.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc48ff9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 0x7efc28057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc4406f7b0 con 0x7efc4c148550 2020-11-16T22:23:23.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc50e75700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 0x7efc28057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.627 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.634+0000 7efc50e75700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 0x7efc28057520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7efc3c003920 tx=0x7efc3c007e60).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.630 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.642+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7efc44045ac0 con 0x7efc4c148550 2020-11-16T22:23:23.829 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.838+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc4c05d4e0 con 0x7efc4c148550 2020-11-16T22:23:23.830 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc48ff9700 1 -- 172.21.15.185:0/3098641388 <== mon.1 v2:172.21.15.185:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+2531 (secure 0 0 0) 0x7efc44023070 con 0x7efc4c148550 2020-11-16T22:23:23.830 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:23.831 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:22.141947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:23.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 msgr2=0x7efc28057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 0x7efc28057520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7efc3c003920 tx=0x7efc3c007e60).stop 2020-11-16T22:23:23.832 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 msgr2=0x7efc4c153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7efc44006fd0 tx=0x7efc44015000).stop 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 shutdown_connections 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7efc28055060 0x7efc28057520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7efc4c148550 0x7efc4c153e20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc4c0b0f90 0x7efc4c0675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.833 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 --2- 172.21.15.185:0/3098641388 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7efc4c0b37e0 0x7efc4c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 >> 172.21.15.185:0/3098641388 conn(0x7efc4c0604a0 msgr2=0x7efc4c13e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 shutdown_connections 2020-11-16T22:23:23.834 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.842+0000 7efc530d9700 1 -- 172.21.15.185:0/3098641388 wait complete. 2020-11-16T22:23:23.835 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 111 2020-11-16T22:23:23.844 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub status 2020-11-16T22:23:23.972 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 Processor -- start 2020-11-16T22:23:23.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 -- start start 2020-11-16T22:23:23.973 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 0x7fcdb8066ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcdb8068160 0x7fcdb813de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb8067780 con 0x7fcdb8060be0 2020-11-16T22:23:23.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb80604d0 con 0x7fcdb8068160 2020-11-16T22:23:23.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbd1e7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb8067a20 con 0x7fcdb80666f0 2020-11-16T22:23:23.974 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdb7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 0x7fcdb8066ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdbc9e6700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcdb8068160 0x7fcdb813de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdb7fff700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 0x7fcdb8066ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36232/0 (socket says 172.21.15.185:36232) 2020-11-16T22:23:23.975 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.982+0000 7fcdb7fff700 1 -- 172.21.15.185:0/2451837559 learned_addr learned my addr 172.21.15.185:0/2451837559 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.976 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb77fe700 1 -- 172.21.15.185:0/2451837559 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049697718 0 0) 0x7fcdb8067780 con 0x7fcdb8060be0 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb77fe700 1 -- 172.21.15.185:0/2451837559 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd98003240 con 0x7fcdb8060be0 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/2451837559 >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 msgr2=0x7fcdb8066ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/2451837559 >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 0x7fcdb8066ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/2451837559 >> v1:172.21.15.96:6789/0 conn(0x7fcdb8060be0 legacy=0x7fcdb80b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/2451837559 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8067cc0 con 0x7fcdb8068160 2020-11-16T22:23:23.977 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/2451837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8068160 0x7fcdb813de50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fcdb0009c90 tx=0x7fcdb00091d0).ready entity=mon.0 client_cookie=ce7b964deda6c0d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb77fe700 1 -- 172.21.15.185:0/2451837559 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000b980 con 0x7fcdb8068160 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb77fe700 1 -- 172.21.15.185:0/2451837559 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb000bae0 con 0x7fcdb8068160 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb77fe700 1 -- 172.21.15.185:0/2451837559 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000be60 con 0x7fcdb8068160 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/2451837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8068160 msgr2=0x7fcdb813de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/2451837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8068160 0x7fcdb813de50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fcdb0009c90 tx=0x7fcdb00091d0).stop 2020-11-16T22:23:23.978 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/2451837559 shutdown_connections 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/2451837559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8068160 0x7fcdb813de50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/2451837559 >> v2:172.21.15.185:3300/0 conn(0x7fcdb80666f0 0x7fcdb8066ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/2451837559 >> 172.21.15.185:0/2451837559 conn(0x7fcdb805ba80 msgr2=0x7fcdb805be90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/2451837559 shutdown_connections 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/2451837559 wait complete. 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 Processor -- start 2020-11-16T22:23:23.979 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- start start 2020-11-16T22:23:23.981 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8060be0 0x7fcdb80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdb8068160 0x7fcdb8146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb813ef90 con 0x7fcdb8060be0 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8146700 con 0x7fcdb8068160 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbd1e7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80b0c10 con 0x7fcdb80666f0 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.982 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57914/0 (socket says 172.21.15.185:57914) 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdaffff700 1 -- 172.21.15.185:0/1012029220 learned_addr learned my addr 172.21.15.185:0/1012029220 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdb8068160 0x7fcdb8146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.986+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8060be0 0x7fcdb80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 -- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdb8068160 msgr2=0x7fcdb8146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdb8068160 0x7fcdb8146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.983 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 -- 172.21.15.185:0/1012029220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8060be0 msgr2=0x7fcdb80b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8060be0 0x7fcdb80b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 -- 172.21.15.185:0/1012029220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80b0e80 con 0x7fcdb80666f0 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdaffff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fcda000c710 tx=0x7fcda0007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 -- 172.21.15.185:0/1012029220 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000e490 con 0x7fcdb80666f0 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 -- 172.21.15.185:0/1012029220 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000e5f0 con 0x7fcdb80666f0 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb80b1180 con 0x7fcdb80666f0 2020-11-16T22:23:23.984 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 -- 172.21.15.185:0/1012029220 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000e990 con 0x7fcdb80666f0 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb80b18e0 con 0x7fcdb80666f0 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 -- 172.21.15.185:0/1012029220 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fcda0019030 con 0x7fcdb80666f0 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 0x7fcd94057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb4ff9700 1 -- 172.21.15.185:0/1012029220 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcda006d020 con 0x7fcdb80666f0 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 0x7fcd94057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.985 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.990+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 0x7fcd94057520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7fcdb81482b0 tx=0x7fcda8008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.994+0000 7fcdcceda740 1 Processor -- start 2020-11-16T22:23:23.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.994+0000 7fcdcceda740 1 -- start start 2020-11-16T22:23:23.987 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2bfd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 0x2bfc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a894b0 con 0x2bfdda0 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b3d930 con 0x29df7b0 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x29fd6f0 con 0x2971bb0 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.988 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57920/0 (socket says 172.21.15.185:57920) 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 -- 172.21.15.185:0/642583415 learned_addr learned my addr 172.21.15.185:0/642583415 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdb7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2bfd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 0x2bfc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 -- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 msgr2=0x2bfd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2bfd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.989 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 -- 172.21.15.185:0/642583415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 msgr2=0x2bfc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 0x2bfc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 -- 172.21.15.185:0/642583415 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29f85a0 con 0x2971bb0 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 0x2bfc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdaffff700 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fcda00468d0 tx=0x7fcda0012000).ready entity=mon.2 client_cookie=b142543aae490b6c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdad7fa700 1 -- 172.21.15.185:0/642583415 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000ed40 con 0x2971bb0 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdad7fa700 1 -- 172.21.15.185:0/642583415 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0046430 con 0x2971bb0 2020-11-16T22:23:23.990 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdad7fa700 1 -- 172.21.15.185:0/642583415 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000eea0 con 0x2971bb0 2020-11-16T22:23:23.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 msgr2=0x2bfbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:23.993 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fcda00468d0 tx=0x7fcda0012000).stop 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- 172.21.15.185:0/642583415 shutdown_connections 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2bfd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2bfdda0 0x2bfc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 --2- 172.21.15.185:0/642583415 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbe90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- 172.21.15.185:0/642583415 >> 172.21.15.185:0/642583415 conn(0x2bade40 msgr2=0x2bfadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:23.994 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- 172.21.15.185:0/642583415 shutdown_connections 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- 172.21.15.185:0/642583415 wait complete. 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 -- start start 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:23.998+0000 7fcdcceda740 1 Processor -- start 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:23.995 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b2a6e0 con 0x2971bb0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b2a820 con 0x29df7b0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b2ba90 con 0x2bfdda0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 10 client.0 ll_register_callbacks cb 0x2b20ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdbc9e6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.996 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdbc9e6700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57930/0 (socket says 172.21.15.185:57930) 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36252/0 (socket says 172.21.15.185:36252) 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/2183134465 learned_addr learned my addr 172.21.15.185:0/2183134465 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 -- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 msgr2=0x2c453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 -- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 msgr2=0x2bfa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 -- 172.21.15.185:0/2183134465 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2b1b280 con 0x29df7b0 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdaffff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fcda0025000 tx=0x7fcda0014070).ready entity=mon.1 client_cookie=2dd60a56c729f2d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0032070 con 0x29df7b0 2020-11-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcda00460b0 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2bde6c0 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda0046270 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 10 client.8359 fetch_fsmap learned FSMap version 111 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x2950fd0 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2bde6c0 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 111) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fcda006ba00 con 0x29df7b0 2020-11-16T22:23:23.999 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 10 client.8359 fetch_fsmap finished waiting for FSMap version 111 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 10 client.8359 resolve_mds: resolved ID 'c' to GID 7830 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 20 client.8359 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2c45890 0x2c31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 4 client.8359 mds_command: new command op to 7830 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 --> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x2a1bc50 con 0x2c45890 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2c45890 0x2c31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcd8e7fc700 10 client.8359 ms_handle_connect on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.002+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2c45890 0x2c31140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=87866e13773222b2 server_cookie=6831df002389291f in_seq=0 out_seq=0 2020-11-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/2183134465 <== mds.1 v2:172.21.15.96:6834/477883541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2a1bc50 con 0x2c45890 2020-11-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcd8e7fc700 10 client.8359 handle_command_reply: tid=0 2020-11-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 client.8359 shutdown 2020-11-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 msgr2=0x2c41660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fcda0025000 tx=0x7fcda0014070).stop 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 shutdown_connections 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2c45890 0x2c31140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcd8e7fc700 0 client.8359 ms_handle_reset on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2bfdda0 0x2c453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.003 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2183134465 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x29df7b0 0x2c41660 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 >> 172.21.15.185:0/2183134465 conn(0x2bade40 msgr2=0x2bf9580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 shutdown_connections 2020-11-16T22:23:24.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 1 -- 172.21.15.185:0/2183134465 wait complete. 2020-11-16T22:23:24.004 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.006+0000 7fcdcceda740 20 client.8359 trim_cache size 0 max 16384 2020-11-16T22:23:24.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 Processor -- start 2020-11-16T22:23:24.015 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 -- start start 2020-11-16T22:23:24.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29df7b0 0x2bfc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.016 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bde6f0 con 0x29df7b0 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b02e40 con 0x2bade40 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b2a820 con 0x2971bb0 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36258/0 (socket says 172.21.15.185:36258) 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 -- 172.21.15.185:0/2765474809 learned_addr learned my addr 172.21.15.185:0/2765474809 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.017 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29df7b0 0x2bfc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 -- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 msgr2=0x2bfbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 -- 172.21.15.185:0/2765474809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29df7b0 msgr2=0x2bfc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.019 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29df7b0 0x2bfc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 -- 172.21.15.185:0/2765474809 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84008160 con 0x2bade40 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7fcda8014df0 tx=0x7fcda8014000).ready entity=mon.1 client_cookie=90fc41fa7eb0a22c server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdadffb700 1 -- 172.21.15.185:0/2765474809 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8014e20 con 0x2bade40 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdadffb700 1 -- 172.21.15.185:0/2765474809 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda8022c10 con 0x2bade40 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.026+0000 7fcdadffb700 1 -- 172.21.15.185:0/2765474809 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8029430 con 0x2bade40 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 msgr2=0x2bf85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.020 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7fcda8014df0 tx=0x7fcda8014000).stop 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/2765474809 shutdown_connections 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2bf85e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x29df7b0 0x2bfc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- 172.21.15.185:0/2765474809 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x2971bb0 0x2bfbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/2765474809 >> 172.21.15.185:0/2765474809 conn(0x7fcd84002320 msgr2=0x2c2c890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/2765474809 shutdown_connections 2020-11-16T22:23:24.021 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/2765474809 wait complete. 2020-11-16T22:23:24.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- start start 2020-11-16T22:23:24.022 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 Processor -- start 2020-11-16T22:23:24.024 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29df7b0 0x2c95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c33930 con 0x2971bb0 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c33b90 con 0x2bade40 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x2c42040 con 0x29df7b0 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 10 client.0 ll_register_callbacks cb 0x2c2d690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:24.025 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdaffff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29df7b0 0x2c95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36268/0 (socket says 172.21.15.185:36268) 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/3838531934 learned_addr learned my addr 172.21.15.185:0/3838531934 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29df7b0 msgr2=0x2c95970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.026 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29df7b0 0x2c95970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 msgr2=0x2bfd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 -- 172.21.15.185:0/3838531934 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x298f720 con 0x2bade40 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdbc9e6700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fcdb80637b0 tx=0x7fcdb0016000).ready entity=mon.1 client_cookie=35a5cdc75a3b4c7f server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.185:3300/0 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0022070 con 0x2bade40 2020-11-16T22:23:24.027 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mon.1 v2:172.21.15.185:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcdb001e740 con 0x2bade40 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcda0046430 con 0x2bade40 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb001eb10 con 0x2bade40 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 10 client.8365 fetch_fsmap learned FSMap version 111 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x299ae00 con 0x2bade40 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mon.1 v2:172.21.15.185:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0046430 con 0x2bade40 2020-11-16T22:23:24.028 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mon.1 v2:172.21.15.185:3300/0 5 ==== fsmap(e 111) v1 ==== 1489+0+0 (secure 0 0 0) 0x7fcdb0027070 con 0x2bade40 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 10 client.8365 fetch_fsmap finished waiting for FSMap version 111 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 10 client.8365 resolve_mds: resolved ID 'c' to GID 7830 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 20 client.8365 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2bfdda0 0x2c9f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 4 client.8365 mds_command: new command op to 7830 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 --> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x2a22f50 con 0x2bfdda0 2020-11-16T22:23:24.029 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.030+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2bfdda0 0x2c9f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcd8e7fc700 10 client.8365 ms_handle_connect on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdb7fff700 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2bfdda0 0x2c9f530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.1 client_cookie=88f8cc7e6892301c server_cookie=3da0225fc1e1d243 in_seq=0 out_seq=0 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcd8e7fc700 1 -- 172.21.15.185:0/3838531934 <== mds.1 v2:172.21.15.96:6834/477883541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x2a22f50 con 0x2bfdda0 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcd8e7fc700 10 client.8365 handle_command_reply: tid=0 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 client.8365 shutdown 2020-11-16T22:23:24.030 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 msgr2=0x2c95e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fcdb80637b0 tx=0x7fcdb0016000).stop 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 shutdown_connections 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:6834/477883541,v1:172.21.15.96:6835/477883541] conn(0x2bfdda0 0x2c9f530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x2bade40 0x2c95e60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x2971bb0 0x2bfd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 --2- 172.21.15.185:0/3838531934 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x29df7b0 0x2c95970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.031 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 >> 172.21.15.185:0/3838531934 conn(0x7fcd84002320 msgr2=0x2bf87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcd8e7fc700 0 client.8365 ms_handle_reset on v2:172.21.15.96:6834/477883541 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 shutdown_connections 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 1 -- 172.21.15.185:0/3838531934 wait complete. 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdcceda740 20 client.8365 trim_cache size 0 max 16384 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 msgr2=0x7fcd94057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 0x7fcd94057520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7fcdb81482b0 tx=0x7fcda8008040).stop 2020-11-16T22:23:24.032 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 msgr2=0x7fcdb8145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fcda000c710 tx=0x7fcda0007a90).stop 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 shutdown_connections 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fcd94055060 0x7fcd94057520 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0x7fcdb81482b0 tx=0x7fcda8008040).stop 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fcdb8068160 0x7fcdb8146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8060be0 0x7fcdb80b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 --2- 172.21.15.185:0/1012029220 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fcdb80666f0 0x7fcdb8145d20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.033 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 >> 172.21.15.185:0/1012029220 conn(0x7fcdb805ba80 msgr2=0x7fcdb80b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 shutdown_connections 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.034+0000 7fcdbd1e7700 1 -- 172.21.15.185:0/1012029220 wait complete. 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:23:24.034 INFO:teuthology.orchestra.run.smithi185.stdout: "4a97bcea-0e01-4b36-9db4-20d2bef38a21": { 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "4a97bcea-0e01-4b36-9db4-20d2bef38a21", 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:23:24.035 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:23:24.036 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:23:24.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 Processor -- start 2020-11-16T22:23:24.155 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- start start 2020-11-16T22:23:24.156 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 0x7fa1d8066e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d80b2180 con 0x7fa1d80b4200 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d8067360 con 0x7fa1d8099ea0 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d80b23e0 con 0x7fa1d8144ba0 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 0x7fa1d8066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.157 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57952/0 (socket says 172.21.15.185:57952) 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d6ffd700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 0x7fa1d8066e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36274/0 (socket says 172.21.15.185:36274) 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 -- 172.21.15.185:0/1104539506 learned_addr learned my addr 172.21.15.185:0/1104539506 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 -- 172.21.15.185:0/1104539506 >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 msgr2=0x7fa1d8066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 --2- 172.21.15.185:0/1104539506 >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 0x7fa1d8066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.158 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 -- 172.21.15.185:0/1104539506 >> v1:172.21.15.96:6789/0 conn(0x7fa1d80b4200 legacy=0x7fa1d8144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 -- 172.21.15.185:0/1104539506 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80b2680 con 0x7fa1d8144ba0 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d7fff700 1 --2- 172.21.15.185:0/1104539506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fa1c8006fd0 tx=0x7fa1c8006eb0).ready entity=mon.2 client_cookie=8c06fa610c265e37 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d67fc700 1 -- 172.21.15.185:0/1104539506 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8009810 con 0x7fa1d8144ba0 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d67fc700 1 -- 172.21.15.185:0/1104539506 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8009970 con 0x7fa1d8144ba0 2020-11-16T22:23:24.159 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1d67fc700 1 -- 172.21.15.185:0/1104539506 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8009cf0 con 0x7fa1d8144ba0 2020-11-16T22:23:24.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1104539506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 msgr2=0x7fa1d8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.161 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1104539506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fa1c8006fd0 tx=0x7fa1c8006eb0).stop 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1104539506 shutdown_connections 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1104539506 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d8144ba0 0x7fa1d8141250 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1104539506 >> v2:172.21.15.185:3300/0 conn(0x7fa1d8099ea0 0x7fa1d8066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1104539506 >> 172.21.15.185:0/1104539506 conn(0x7fa1d8063820 msgr2=0x7fa1d8061240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1104539506 shutdown_connections 2020-11-16T22:23:24.162 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.166+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1104539506 wait complete. 2020-11-16T22:23:24.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 Processor -- start 2020-11-16T22:23:24.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- start start 2020-11-16T22:23:24.163 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1d8099ea0 0x7fa1d809bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d8144ba0 0x7fa1d80ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d809ac80 con 0x7fa1d8144ba0 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d809a9e0 con 0x7fa1d8099ea0 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d809b6b0 con 0x7fa1d80b4200 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.164 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1d8099ea0 0x7fa1d809bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57956/0 (socket says 172.21.15.185:57956) 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 -- 172.21.15.185:0/1567229983 learned_addr learned my addr 172.21.15.185:0/1567229983 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d7fff700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d8144ba0 0x7fa1d80ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 -- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1d8099ea0 msgr2=0x7fa1d809bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1d8099ea0 0x7fa1d809bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.165 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 -- 172.21.15.185:0/1567229983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d8144ba0 msgr2=0x7fa1d80ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d8144ba0 0x7fa1d80ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 -- 172.21.15.185:0/1567229983 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d809b950 con 0x7fa1d80b4200 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1d6ffd700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fa1c000ebd0 tx=0x7fa1c0006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000b3a0 con 0x7fa1d80b4200 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c000c9b0 con 0x7fa1d80b4200 2020-11-16T22:23:24.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d813d0f0 con 0x7fa1d80b4200 2020-11-16T22:23:24.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000cd70 con 0x7fa1d80b4200 2020-11-16T22:23:24.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.170+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d809a6d0 con 0x7fa1d80b4200 2020-11-16T22:23:24.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d8062170 con 0x7fa1d80b4200 2020-11-16T22:23:24.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7fa1c0012600 con 0x7fa1d80b4200 2020-11-16T22:23:24.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1cf7fe700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 0x7fa1b4057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1d77fe700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 0x7fa1b4057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1c006e150 con 0x7fa1d80b4200 2020-11-16T22:23:24.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.174+0000 7fa1d77fe700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 0x7fa1b4057520 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fa1d0004250 tx=0x7fa1d0007f10).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.178+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143067 (secure 0 0 0) 0x7fa1c00443b0 con 0x7fa1d80b4200 2020-11-16T22:23:24.370 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1d805d4e0 con 0x7fa1d80b4200 2020-11-16T22:23:24.371 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1cf7fe700 1 -- 172.21.15.185:0/1567229983 <== mon.2 v2:172.21.15.185:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+2531 (secure 0 0 0) 0x7fa1c00479d0 con 0x7fa1d80b4200 2020-11-16T22:23:24.371 INFO:teuthology.orchestra.run.smithi185.stdout: 2020-11-16T22:23:24.371 INFO:teuthology.orchestra.run.smithi185.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-16T22:22:29.743585+0000","modified":"2020-11-16T22:23:22.141947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7993,"mds_1":7830,"mds_2":7821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7821":{"gid":7821,"name":"b","rank":2,"incarnation":105,"state":"up:active","state_seq":16,"addr":"172.21.15.96:6833/2921285835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2921285835},{"type":"v1","addr":"172.21.15.96:6833","nonce":2921285835}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_7830":{"gid":7830,"name":"c","rank":1,"incarnation":103,"state":"up:active","state_seq":15,"addr":"172.21.15.96:6835/477883541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":477883541},{"type":"v1","addr":"172.21.15.96:6835","nonce":477883541}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138297136906239,"flags":0},"gid_7993":{"gid":7993,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":16,"addr":"172.21.15.185:6835/1118138229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.185:6834","nonce":1118138229},{"type":"v1","addr":"172.21.15.185:6835","nonce":1118138229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-16T22:23:24.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 msgr2=0x7fa1b4057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 0x7fa1b4057520 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fa1d0004250 tx=0x7fa1d0007f10).stop 2020-11-16T22:23:24.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 msgr2=0x7fa1d813cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fa1c000ebd0 tx=0x7fa1c0006df0).stop 2020-11-16T22:23:24.373 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 shutdown_connections 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7fa1b4055060 0x7fa1b4057520 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7fa1d8099ea0 0x7fa1d809bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1d8144ba0 0x7fa1d80ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 --2- 172.21.15.185:0/1567229983 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7fa1d80b4200 0x7fa1d813cc00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 >> 172.21.15.185:0/1567229983 conn(0x7fa1d8063820 msgr2=0x7fa1d8067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 shutdown_connections 2020-11-16T22:23:24.375 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.382+0000 7fa1ddc36700 1 -- 172.21.15.185:0/1567229983 wait complete. 2020-11-16T22:23:24.375 INFO:teuthology.orchestra.run.smithi185.stderr:dumped fsmap epoch 111 2020-11-16T22:23:24.390 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b scrub status 2020-11-16T22:23:24.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.526+0000 7f52aefc7700 1 Processor -- start 2020-11-16T22:23:24.517 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.526+0000 7f52aefc7700 1 -- start start 2020-11-16T22:23:24.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.526+0000 7f52aefc7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 0x7f52a80b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.520 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 0x7f52a8142170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52a80639e0 0x7f52a813f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a8062f70 con 0x7f52a80640c0 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- --> v2:172.21.15.185:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a8146250 con 0x7f52a80639e0 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8062cd0 con 0x7f52a80b6d90 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52a80639e0 0x7f52a813f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.521 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 --2- >> v2:172.21.15.185:3300/0 conn(0x7f52a80639e0 0x7f52a813f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36288/0 (socket says 172.21.15.185:36288) 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 -- 172.21.15.185:0/1253175858 learned_addr learned my addr 172.21.15.185:0/1253175858 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52acd63700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 0x7f52a80b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f529ffff700 1 --2- 172.21.15.185:0/1253175858 >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 0x7f52a8142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 -- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 msgr2=0x7f52a80b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 0x7f52a80b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.522 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 -- 172.21.15.185:0/1253175858 >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 msgr2=0x7f52a8142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 --2- 172.21.15.185:0/1253175858 >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 0x7f52a8142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 -- 172.21.15.185:0/1253175858 --> v2:172.21.15.185:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8145f70 con 0x7f52a80639e0 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f529ffff700 1 --2- 172.21.15.185:0/1253175858 >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 0x7f52a8142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52acd63700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 0x7f52a80b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52ad564700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80639e0 0x7f52a813f4a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f52a8099410 tx=0x7f52a0009290).ready entity=mon.1 client_cookie=1c58bad219401714 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f529f7fe700 1 -- 172.21.15.185:0/1253175858 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000bbc0 con 0x7f52a80639e0 2020-11-16T22:23:24.523 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f529f7fe700 1 -- 172.21.15.185:0/1253175858 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a000bd20 con 0x7f52a80639e0 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f529f7fe700 1 -- 172.21.15.185:0/1253175858 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0011610 con 0x7f52a80639e0 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80639e0 msgr2=0x7f52a813f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80639e0 0x7f52a813f4a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f52a8099410 tx=0x7f52a0009290).stop 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- 172.21.15.185:0/1253175858 shutdown_connections 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80639e0 0x7f52a813f4a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.524 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- 172.21.15.185:0/1253175858 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80b6d90 0x7f52a80b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 --2- 172.21.15.185:0/1253175858 >> v2:172.21.15.96:3300/0 conn(0x7f52a80640c0 0x7f52a8142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- 172.21.15.185:0/1253175858 >> 172.21.15.185:0/1253175858 conn(0x7f52a80657b0 msgr2=0x7f52a8065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- 172.21.15.185:0/1253175858 shutdown_connections 2020-11-16T22:23:24.525 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- 172.21.15.185:0/1253175858 wait complete. 2020-11-16T22:23:24.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 Processor -- start 2020-11-16T22:23:24.527 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.530+0000 7f52aefc7700 1 -- start start 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 0x7f52a8144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 0x7f52a8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a809b4f0 con 0x7f52a80640c0 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81533e0 con 0x7f52a80b6d90 2020-11-16T22:23:24.528 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8153680 con 0x7f52a80639e0 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57968/0 (socket says 172.21.15.185:57968) 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52ad564700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 0x7f52a8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 -- 172.21.15.185:0/800265588 learned_addr learned my addr 172.21.15.185:0/800265588 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f529ffff700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 0x7f52a8144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 -- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 msgr2=0x7f52a8156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.529 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 0x7f52a8156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 -- 172.21.15.185:0/800265588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 msgr2=0x7f52a8144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 0x7f52a8144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 -- 172.21.15.185:0/800265588 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8154010 con 0x7f52a80639e0 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f529ffff700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 0x7f52a8144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52ad564700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 0x7f52a8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-16T22:23:24.530 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52acd63700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f52a400db60 tx=0x7f52a4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f529cff9700 1 -- 172.21.15.185:0/800265588 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009790 con 0x7f52a80639e0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f529cff9700 1 -- 172.21.15.185:0/800265588 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a40098f0 con 0x7f52a80639e0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a80aefe0 con 0x7f52a80639e0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f529cff9700 1 -- 172.21.15.185:0/800265588 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009c70 con 0x7f52a80639e0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.534+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a809b260 con 0x7f52a80639e0 2020-11-16T22:23:24.531 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.538+0000 7f529cff9700 1 -- 172.21.15.185:0/800265588 <== mon.2 v2:172.21.15.185:3301/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f52a4013480 con 0x7f52a80639e0 2020-11-16T22:23:24.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.538+0000 7f529cff9700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 0x7f5280057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.538+0000 7f529cff9700 1 -- 172.21.15.185:0/800265588 <== mon.2 v2:172.21.15.185:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52a406df50 con 0x7f52a80639e0 2020-11-16T22:23:24.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.538+0000 7f529ffff700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 0x7f5280057520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.532 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.538+0000 7f529ffff700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 0x7f5280057520 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f52a80b0870 tx=0x7f5298008040).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 Processor -- start 2020-11-16T22:23:24.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- start start 2020-11-16T22:23:24.537 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x151dbb0 0x17ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17e37f0 0x17de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16354b0 con 0x151dbb0 2020-11-16T22:23:24.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x16e9930 con 0x17e37f0 2020-11-16T22:23:24.538 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x15a96f0 con 0x158b7b0 2020-11-16T22:23:24.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57976/0 (socket says 172.21.15.185:57976) 2020-11-16T22:23:24.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 -- 172.21.15.185:0/2192216399 learned_addr learned my addr 172.21.15.185:0/2192216399 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f529ffff700 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x151dbb0 0x17ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.539 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 -- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17e37f0 msgr2=0x17de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17e37f0 0x17de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 -- 172.21.15.185:0/2192216399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x151dbb0 msgr2=0x17ddf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x151dbb0 0x17ddf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 -- 172.21.15.185:0/2192216399 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15a45a0 con 0x158b7b0 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52acd63700 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f52a400db30 tx=0x7f52a4018000).ready entity=mon.2 client_cookie=9463079b91872933 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.540 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f5285ffb700 1 -- 172.21.15.185:0/2192216399 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40471f0 con 0x158b7b0 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f5285ffb700 1 -- 172.21.15.185:0/2192216399 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4047350 con 0x158b7b0 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f5285ffb700 1 -- 172.21.15.185:0/2192216399 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40474c0 con 0x158b7b0 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 msgr2=0x17e3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f52a400db30 tx=0x7f52a4018000).stop 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- 172.21.15.185:0/2192216399 shutdown_connections 2020-11-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x17e37f0 0x17de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17e3300 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 --2- 172.21.15.185:0/2192216399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x151dbb0 0x17ddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- 172.21.15.185:0/2192216399 >> 172.21.15.185:0/2192216399 conn(0x1759e40 msgr2=0x17e2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- 172.21.15.185:0/2192216399 shutdown_connections 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- 172.21.15.185:0/2192216399 wait complete. 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 -- start start 2020-11-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.542+0000 7f52bda58740 1 Processor -- start 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17c6440 0x17f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d66e0 con 0x17c6440 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d6820 con 0x158b7b0 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x16d7a90 con 0x151dbb0 2020-11-16T22:23:24.543 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 10 client.0 ll_register_callbacks cb 0x16ccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57982/0 (socket says 172.21.15.185:57982) 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 -- 172.21.15.185:0/2621679285 learned_addr learned my addr 172.21.15.185:0/2621679285 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f529ffff700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52ad564700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17c6440 0x17f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 -- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 msgr2=0x17f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 -- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17c6440 msgr2=0x17f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17c6440 0x17f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 -- 172.21.15.185:0/2621679285 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16c7280 con 0x151dbb0 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52acd63700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f52a4047a00 tx=0x7f52a401afc0).ready entity=mon.2 client_cookie=b412f62241719360 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4046180 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x179ee10 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52a402fc40 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a402fe00 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 10 client.7460 fetch_fsmap learned FSMap version 111 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x14fcfd0 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x179ee10 con 0x151dbb0 2020-11-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 111) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f52a4006450 con 0x151dbb0 2020-11-16T22:23:24.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 10 client.7460 fetch_fsmap finished waiting for FSMap version 111 2020-11-16T22:23:24.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 10 client.7460 resolve_mds: resolved ID 'b' to GID 7821 2020-11-16T22:23:24.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 20 client.7460 populate_metadata read hostname 'smithi185' 2020-11-16T22:23:24.750 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17d8ec0 0x17ee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 4 client.7460 mds_command: new command op to 7821 tid=0[{,",p,r,e,f,i,x,",:, ,",g,e,t,_,c,o,m,m,a,n,d,_,d,e,s,c,r,i,p,t,i,o,n,s,",}] 2020-11-16T22:23:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 --> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] -- command(tid 0: { " p r e f i x " : " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x15c7c50 con 0x17d8ec0 2020-11-16T22:23:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f529ffff700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17d8ec0 0x17ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f5276ffd700 10 client.7460 ms_handle_connect on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:24.751 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.546+0000 7f529ffff700 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17d8ec0 0x17ee050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=78ca6f37816f50b0 server_cookie=f67086e2d190f58a in_seq=0 out_seq=0 2020-11-16T22:23:24.756 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f5276ffd700 1 -- 172.21.15.185:0/2621679285 <== mds.2 v2:172.21.15.96:6832/2921285835 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x15c7c50 con 0x17d8ec0 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f5276ffd700 10 client.7460 handle_command_reply: tid=0 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 client.7460 shutdown 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 msgr2=0x17c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f52a4047a00 tx=0x7f52a401afc0).stop 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 shutdown_connections 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x17c6440 0x17f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.757 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17c7030 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17d8ec0 0x17ee050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 --2- 172.21.15.185:0/2621679285 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f5276ffd700 0 client.7460 ms_handle_reset on v2:172.21.15.96:6832/2921285835 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 >> 172.21.15.185:0/2621679285 conn(0x1759e40 msgr2=0x17dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 shutdown_connections 2020-11-16T22:23:24.758 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 1 -- 172.21.15.185:0/2621679285 wait complete. 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.550+0000 7f52bda58740 20 client.7460 trim_cache size 0 max 16384 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 Processor -- start 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- start start 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x17a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x151dbb0 0x17a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x16aee40 con 0x1759e40 2020-11-16T22:23:24.759 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x17c8600 con 0x151dbb0 2020-11-16T22:23:24.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x17dd400 con 0x158b7b0 2020-11-16T22:23:24.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.760 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57990/0 (socket says 172.21.15.185:57990) 2020-11-16T22:23:24.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 -- 172.21.15.185:0/694192777 learned_addr learned my addr 172.21.15.185:0/694192777 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:24.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f529ffff700 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x151dbb0 0x17a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52acd63700 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x17a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:24.962 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 -- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x151dbb0 msgr2=0x17a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x151dbb0 0x17a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 -- 172.21.15.185:0/694192777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 msgr2=0x17a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x17a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 -- 172.21.15.185:0/694192777 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0081d0 con 0x158b7b0 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52ad564700 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f52a8146390 tx=0x7f52a0016000).ready entity=mon.2 client_cookie=41101aa68c4c30f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52867fc700 1 -- 172.21.15.185:0/694192777 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0009200 con 0x158b7b0 2020-11-16T22:23:24.963 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52867fc700 1 -- 172.21.15.185:0/694192777 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a0011610 con 0x158b7b0 2020-11-16T22:23:24.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52867fc700 1 -- 172.21.15.185:0/694192777 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a001d660 con 0x158b7b0 2020-11-16T22:23:24.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 msgr2=0x17a8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:23:24.964 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f52a8146390 tx=0x7f52a0016000).stop 2020-11-16T22:23:25.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- 172.21.15.185:0/694192777 shutdown_connections 2020-11-16T22:23:25.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x158b7b0 0x17a8fe0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:25.166 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x17a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 --2- 172.21.15.185:0/694192777 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x151dbb0 0x17a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- 172.21.15.185:0/694192777 >> 172.21.15.185:0/694192777 conn(0x7f527c002320 msgr2=0x17c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.570+0000 7f52bda58740 1 -- 172.21.15.185:0/694192777 shutdown_connections 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 -- 172.21.15.185:0/694192777 wait complete. 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 -- start start 2020-11-16T22:23:25.167 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 Processor -- start 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x1843590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x17c72c0 con 0x1759e40 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x17a49c0 con 0x158b7b0 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x17e28c0 con 0x151dbb0 2020-11-16T22:23:25.168 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52bda58740 10 client.0 ll_register_callbacks cb 0x17c8770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f529ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:57996/0 (socket says 172.21.15.185:57996) 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 -- 172.21.15.185:0/1329058973 learned_addr learned my addr 172.21.15.185:0/1329058973 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f529ffff700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17e2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36322/0 (socket says 172.21.15.185:36322) 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52ad564700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x1843590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:23:25.169 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 -- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 msgr2=0x17e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 -- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 msgr2=0x1843590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x1843590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52acd63700 1 -- 172.21.15.185:0/1329058973 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x153b720 con 0x151dbb0 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.574+0000 7f52ad564700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x1843590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52acd63700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f52a406c480 tx=0x7f52a402d000).ready entity=mon.2 client_cookie=ef270a099b6f8ab1 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:23:25.170 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 10 client.0 ms_handle_connect on v2:172.21.15.185:3301/0 2020-11-16T22:23:25.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40471f0 con 0x151dbb0 2020-11-16T22:23:25.171 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mon.2 v2:172.21.15.185:3301/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52a4044410 con 0x151dbb0 2020-11-16T22:23:25.374 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52a4047350 con 0x151dbb0 2020-11-16T22:59:59.195 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a40474c0 con 0x151dbb0 2020-11-16T22:59:59.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 10 client.7466 fetch_fsmap learned FSMap version 111 2020-11-16T22:59:59.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x1546e00 con 0x151dbb0 2020-11-16T22:59:59.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mon.2 v2:172.21.15.185:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4047350 con 0x151dbb0 2020-11-16T22:59:59.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mon.2 v2:172.21.15.185:3301/0 5 ==== fsmap(e 111) v1 ==== 1489+0+0 (secure 0 0 0) 0x7f52a40056d0 con 0x151dbb0 2020-11-16T22:59:59.196 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 10 client.7466 fetch_fsmap finished waiting for FSMap version 111 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 10 client.7466 resolve_mds: resolved ID 'b' to GID 7821 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 20 client.7466 populate_metadata read hostname 'smithi185' 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17c6440 0x183f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 4 client.7466 mds_command: new command op to 7821 tid=0[{,",p,r,e,f,i,x,",:, ,",s,c,r,u,b, ,s,t,a,t,u,s,",}] 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 --> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] -- command(tid 0: { " p r e f i x " : " s c r u b s t a t u s " }) v1 -- 0x15cef50 con 0x17c6440 2020-11-16T22:59:59.197 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f529ffff700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17c6440 0x183f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f529ffff700 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17c6440 0x183f3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.2 client_cookie=c18b4bc93a0f1a1e server_cookie=da7017e780a5cff8 in_seq=0 out_seq=0 2020-11-16T22:59:59.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 10 client.7466 ms_handle_connect on v2:172.21.15.96:6832/2921285835 2020-11-16T22:59:59.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 1 -- 172.21.15.185:0/1329058973 <== mds.2 v2:172.21.15.96:6832/2921285835 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x15cef50 con 0x17c6440 2020-11-16T22:59:59.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 10 client.7466 handle_command_reply: tid=0 2020-11-16T22:59:59.198 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 client.7466 shutdown 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 msgr2=0x17a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f52a406c480 tx=0x7f52a402d000).stop 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 shutdown_connections 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x1759e40 0x1843590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x151dbb0 0x17a3990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.96:6832/2921285835,v1:172.21.15.96:6833/2921285835] conn(0x17c6440 0x183f3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.199 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 --2- 172.21.15.185:0/1329058973 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x158b7b0 0x17e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 >> 172.21.15.185:0/1329058973 conn(0x7f527c002320 msgr2=0x17a8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f5276ffd700 0 client.7466 ms_handle_reset on v2:172.21.15.96:6832/2921285835 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 shutdown_connections 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 1 -- 172.21.15.185:0/1329058973 wait complete. 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52bda58740 20 client.7466 trim_cache size 0 max 16384 2020-11-16T22:59:59.200 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 msgr2=0x7f5280057520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 0x7f5280057520 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f52a80b0870 tx=0x7f5298008040).stop 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 msgr2=0x7f52a8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f52a400db60 tx=0x7f52a4008040).stop 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 shutdown_connections 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f5280055060 0x7f5280057520 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f52a80b6d90 0x7f52a8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.201 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f52a80639e0 0x7f52a8153b20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 --2- 172.21.15.185:0/800265588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52a80640c0 0x7f52a8144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 >> 172.21.15.185:0/800265588 conn(0x7f52a80657b0 msgr2=0x7f52a80b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:59:59.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 shutdown_connections 2020-11-16T22:59:59.202 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:23:24.578+0000 7f52aefc7700 1 -- 172.21.15.185:0/800265588 wait complete. 2020-11-16T22:59:59.202 INFO:teuthology.orchestra.run.smithi185.stdout:{ 2020-11-16T22:59:59.203 INFO:teuthology.orchestra.run.smithi185.stdout: "status": "PAUSED (0 inodes in the stack)", 2020-11-16T22:59:59.203 INFO:teuthology.orchestra.run.smithi185.stdout: "scrubs": { 2020-11-16T22:59:59.203 INFO:teuthology.orchestra.run.smithi185.stdout: "4a97bcea-0e01-4b36-9db4-20d2bef38a21": { 2020-11-16T22:59:59.203 INFO:teuthology.orchestra.run.smithi185.stdout: "path": "/d1/d2/d3", 2020-11-16T22:59:59.203 INFO:teuthology.orchestra.run.smithi185.stdout: "tag": "4a97bcea-0e01-4b36-9db4-20d2bef38a21", 2020-11-16T22:59:59.204 INFO:teuthology.orchestra.run.smithi185.stdout: "options": "recursive,force" 2020-11-16T22:59:59.204 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:59:59.204 INFO:teuthology.orchestra.run.smithi185.stdout: } 2020-11-16T22:59:59.204 INFO:teuthology.orchestra.run.smithi185.stdout:} 2020-11-16T22:59:59.205 INFO:teuthology.orchestra.run.smithi185:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-16T22:59:59.209 INFO:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:59:59.210 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:23:27.103+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2020-11-16T22:59:59.210 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:29:59.998+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 3 MDSs report damaged metadata 2020-11-16T22:59:59.414 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:39:59.995+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 3 MDSs report damaged metadata 2020-11-16T22:59:59.414 INFO:tasks.ceph.mon.b.smithi096.stderr:2020-11-16T22:49:59.996+0000 7f95efcf1700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 3 MDSs report damaged metadata 2020-11-16T22:59:59.420 INFO:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:59:59.422 INFO:teuthology.orchestra.run.smithi185:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-16T22:59:59.545 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 Processor -- start 2020-11-16T22:59:59.546 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 -- start start 2020-11-16T22:59:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f50900b76f0 0x7f50900673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.547 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50900b3ad0 con 0x7f5090060130 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5090141dd0 con 0x7f50900b76f0 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5090141fc0 con 0x7f5090060c10 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:58068/0 (socket says 172.21.15.185:58068) 2020-11-16T22:59:59.548 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 -- 172.21.15.185:0/1067252122 learned_addr learned my addr 172.21.15.185:0/1067252122 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508e59c700 1 --2- 172.21.15.185:0/1067252122 >> v2:172.21.15.96:3300/0 conn(0x7f50900b76f0 0x7f50900673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 -- 172.21.15.185:0/1067252122 >> v2:172.21.15.96:3300/0 conn(0x7f50900b76f0 msgr2=0x7f50900673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 --2- 172.21.15.185:0/1067252122 >> v2:172.21.15.96:3300/0 conn(0x7f50900b76f0 0x7f50900673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 -- 172.21.15.185:0/1067252122 >> v1:172.21.15.96:6789/0 conn(0x7f5090060130 legacy=0x7f509013ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 -- 172.21.15.185:0/1067252122 --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5090142260 con 0x7f5090060c10 2020-11-16T22:59:59.549 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508ed9d700 1 --2- 172.21.15.185:0/1067252122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f508000ba20 tx=0x7f5080006eb0).ready entity=mon.2 client_cookie=d3b0192ccdccb7a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:59:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508d59a700 1 -- 172.21.15.185:0/1067252122 <== mon.2 v2:172.21.15.185:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50800097d0 con 0x7f5090060c10 2020-11-16T22:59:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508d59a700 1 -- 172.21.15.185:0/1067252122 <== mon.2 v2:172.21.15.185:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5080009930 con 0x7f5090060c10 2020-11-16T22:59:59.550 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.560+0000 7f508d59a700 1 -- 172.21.15.185:0/1067252122 <== mon.2 v2:172.21.15.185:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080009cb0 con 0x7f5090060c10 2020-11-16T22:59:59.552 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/1067252122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 msgr2=0x7f509013f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- 172.21.15.185:0/1067252122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f508000ba20 tx=0x7f5080006eb0).stop 2020-11-16T22:59:59.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/1067252122 shutdown_connections 2020-11-16T22:59:59.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- 172.21.15.185:0/1067252122 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f5090060c10 0x7f509013f3f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- 172.21.15.185:0/1067252122 >> v2:172.21.15.96:3300/0 conn(0x7f50900b76f0 0x7f50900673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.553 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/1067252122 >> 172.21.15.185:0/1067252122 conn(0x7f509005ba80 msgr2=0x7f509005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/1067252122 shutdown_connections 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/1067252122 wait complete. 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 Processor -- start 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- start start 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5090060130 0x7f50900ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.554 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5090060c10 0x7f50900b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50900b76f0 0x7f50900b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50901461b0 con 0x7f5090060c10 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900ae800 con 0x7f5090060130 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- --> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50901448f0 con 0x7f50900b76f0 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5090060130 0x7f50900ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50900b76f0 0x7f50900b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.555 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5090060c10 0x7f50900b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 --2- >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5090060130 0x7f50900ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3300/0 says I am v2:172.21.15.185:36398/0 (socket says 172.21.15.185:36398) 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508ed9d700 1 --2- >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50900b76f0 0x7f50900b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.185:3301/0 says I am v2:172.21.15.185:58078/0 (socket says 172.21.15.185:58078) 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 -- 172.21.15.185:0/4189933213 learned_addr learned my addr 172.21.15.185:0/4189933213 (peer_addr_for_me v2:172.21.15.185:0/0) 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 -- 172.21.15.185:0/4189933213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50900b76f0 msgr2=0x7f50900b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 --2- 172.21.15.185:0/4189933213 >> [v2:172.21.15.185:3301/0,v1:172.21.15.185:6790/0] conn(0x7f50900b76f0 0x7f50900b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 -- 172.21.15.185:0/4189933213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5090060c10 msgr2=0x7f50900b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-16T22:59:59.556 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 --2- 172.21.15.185:0/4189933213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5090060c10 0x7f50900b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-16T22:59:59.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 -- 172.21.15.185:0/4189933213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5090144b90 con 0x7f5090060130 2020-11-16T22:59:59.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508e59c700 1 --2- 172.21.15.185:0/4189933213 >> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] conn(0x7f5090060130 0x7f50900ae310 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f508800ef80 tx=0x7f50880091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-16T22:59:59.762 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f50867fc700 1 -- 172.21.15.185:0/4189933213 <== mon.1 v2:172.21.15.185:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800cc90 con 0x7f5090060130 2020-11-16T22:59:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f50867fc700 1 -- 172.21.15.185:0/4189933213 <== mon.1 v2:172.21.15.185:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508800cdf0 con 0x7f5090060130 2020-11-16T22:59:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f50867fc700 1 -- 172.21.15.185:0/4189933213 <== mon.1 v2:172.21.15.185:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088012600 con 0x7f5090060130 2020-11-16T22:59:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/4189933213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5090144e30 con 0x7f5090060130 2020-11-16T22:59:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.564+0000 7f508f59e700 1 -- 172.21.15.185:0/4189933213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50900b4850 con 0x7f5090060130 2020-11-16T22:59:59.763 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.568+0000 7f508f59e700 1 -- 172.21.15.185:0/4189933213 --> [v2:172.21.15.185:3300/0,v1:172.21.15.185:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509005d200 con 0x7f5090060130 2020-11-16T22:59:59.765 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.568+0000 7f50867fc700 1 -- 172.21.15.185:0/4189933213 <== mon.1 v2:172.21.15.185:3300/0 4 ==== mgrmap(e 5) v1 ==== 66261+0+0 (secure 0 0 0) 0x7f5088022070 con 0x7f5090060130 2020-11-16T22:59:59.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.568+0000 7f50867fc700 1 --2- 172.21.15.185:0/4189933213 >> [v2:172.21.15.185:6832/15035,v1:172.21.15.185:6833/15035] conn(0x7f506c055060 0x7f506c057520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-16T22:59:59.766 INFO:teuthology.orchestra.run.smithi185.stderr:2020-11-16T22:59:59.568+0000 7f50867fc700 1 -- 172.21.15.185:0/4189933213 <== mon.1 v2:172.21.15.185:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f508806e4c0 con 0x7f5090060130